Next Article in Journal
Problem-Solving Design-Platform Model Based on the Methodological Distinctiveness of Service Design
Previous Article in Journal
The Identification of Opportunities for Innovations through Collecting Problems from Citizens

J. Open Innov. Technol. Mark. Complex. 2019, 5(4), 77; https://doi.org/10.3390/joitmc5040077

Article
Critical Factors Affecting Sustainable Success of Social Service Systems
1
Seoul Business School, Seoul School of Sciences and Technologies (aSSIST), Seoul 03767, Korea
2
Business School Lausanne (BSL), 1022 Chavannes, Switzerland
*
Author to whom correspondence should be addressed.
Received: 13 August 2019 / Accepted: 27 September 2019 / Published: 29 September 2019

Abstract

:
Increasing demand for various social services in accordance with the global social change in time requires a systematic and efficient system based on a user-oriented policy operation method beyond an administrative operation method of social services. Therefore, this study aimed to present a specific direction for the development of the social service system by empirically deriving and discussing the critical factors that can lead to sustainable success of the social service system. To this end, 12 factors in 3 areas were critically analyzed through a questionnaire survey and analysis on the basis of the Analytic Hierarchy Process (AHP) method for the social service workers and users of the 50 Plus Foundation, an organization that helps the elderly in Seoul to continue their social participation. The analysis results confirmed that the service delivery system was the most important factor among the service delivery system, service administration, and service network, and, in particular, factors for users’ active system access, such as accessibility and participation possibility, were the most influential factors for sustainable success of the social service system. Ultimately, unlike in the past, system development and direction should be sought by considering service users rather than a one-sided policy approach of the government for the development and operation of the social service system.
Keywords:
social service system; sustainable success; critical factor; analytic hierarchy process

1. Introduction

As new crises that directly affect the quality of life of people emerge, alongside an aging society and changes in family structure, the demand for social services, which are difficult to address by social insurance or public assistance, has rapidly increased [1]. The explosive increase in the demand for social services is represented by the deepening of social alienation, which is reflected through the rapid growth in the aging global population in recent years. Many impacts, such as the increasing divorce rate, suicide rate, and gap between the rich and poor, as well as instability of labor and employment, have also led to the demand for a service provision system that can respond to such changes [2,3].
Social democratic governments have been caught in a difficult dilemma because of the contemporary globalization process, as they have been confronted with the dual responsibility of how to satisfy the rising demands of their citizenry for better welfare provision, while at the same time, resisting the pressures of the global market for cuts in their welfare states [4]. Globalization has been blamed largely for undermining the capacity of the nation-state to determine its own social welfare agenda. The contemporary globalization process in relation to social welfare system has also been traced to neo-liberal ideological dominance, as neo-liberalism privileges market provision that has limited redistributive effects rather than state provision [5].
In these contexts, welfare restructuring movements in Europe after 1990 have led to governments privatizing certain public utilities—transferring part of the welfare burden to the private sector, increasing charges for some services, making eligibility criteria more difficult, reducing the generosity of benefits, introducing new management techniques designed to reduce expenditure, making employment in public sectors less advantageous, and encouraging communities to take more responsibility for the everyday running of services [6]. On the other side, an increase in the diverse social service demands is associated with systematic problems because the demands for social welfare services, such as childbirth, childcare, economic problems, labor, psychological counseling, healthcare, and elderly support, tend to become increasingly complex. Therefore, service delivery systems categorized by a target group or need area can effectively address complex service needs and are thus positioned within the growing interest around systematic service delivery [7,8].
Furthermore, as the demand for social services with complex needs grows because of the occurrence of various social problems, an integrated and user-centered approach to social services is required in accordance with the changing social environment. It is necessary to provide more extended social services through systematic and integrated system development and management by escaping from the past policy and administrative perspective, instead considering an opportunity to invest in new service development through effective and efficient service management [9,10,11].
In the past, the social welfare system has been discussed from the perspective of service providers in terms of how to improve efficiency, which was planned by public structures. However, more and more social welfare services are delivered by local or private social welfare organizations and operated between service providers and consumers. For example, the importance of a service delivery system is highlighted by the European Commission’s Social Investment Package (SIP), which calls for member states to adapt their social models to achieve smart, sustainable, and inclusive growth. Aside from stabilizing the economy and strengthening social investments, this would also require the improvement of access to social services. In another way, the main social services for the 50 plus generation—‘Link Age Plus’ in the United Kingdom, ‘The Tokyo Foundation for Employment Services’ in Japan, and ‘The Seoul 50 Plus Project’ in the South Korea—show that the service delivery system has developed into a recipient-centered and integrated service system with many stakeholders.
As the importance of social services has been widely recognized, research on the social services system is also increasing. However, most studies related to social services mainly focus on the social service policy and status of service, and studies on the theory and system are insufficient when compared to those on the social security system. This is because the priority of social services in a social welfare area is lower than that of social insurance and public assistance, and, thus, the institutional framework of social services is quite weak. This is also because the approach to the relationship with other social service methods and systems is poor, as the discussion on social services is limited mainly to micro approaches [12,13]. Therefore, this study aimed to empirically analyze and derive the critical factors for the successful and sustainable system maintenance and operation of management of a social service-focused social service system, which especially targets middle-aged people in a social service area, providing a concrete discussion on the developmental approach of social services.

2. Literature Reviews

2.1. Social Service

Social service differs in conceptual definition according to a country’s level of institutional development and scholars. In the United Kingdom, the concept of personal social service is used as a concept of consultation rather than social service, referring to support and care services that meet the needs of social care [14]. Previously in the United Kingdom, personal social service was used as a comprehensive term to refer to the duties of local social service agencies, including child service, education service, housing service, welfare service, local health service, national health service, probation and aftercare service, supplementary benefit service, and welfare service for the Department of Social Security’s war pension recipients [15]. However, national social services centering on income security, health, education, and housing have developed, and in the first half of 1970, personal social service was added as a fifth form of social service [16].
In the United States, the concept of social service refers to a consulting-based service that supports the socially disadvantaged, such as social care, disability, and disease, excluding income security, education, healthcare and culture, and human service. In South Korea, the concept of social service is broadly defined as a “service provided socially to improve the welfare of an individual or society as a whole and to improve the quality of life” [12] in the process of promoting social service policy. Social service commonly refers to a system that ensures a form of living worthy of human beings in the fields of welfare, healthcare, education, employment, housing, culture, and environment for all citizens who need help from the national and local governments and private sector, helping to improve the quality of life of people through counseling, rehabilitation, care, information provision, utilization of related facilities, competency development, and social participation support.
However, scholars are attempting a relatively specific definition of social service. Kahn and Kamerman [17] defined personal social service as an individual, immaterial, and socio-psychological service. Examples of the specific categories presented by them include childcare; family service and counseling; local elderly welfare service; elderly care service; housekeeping assistance and housekeeping support services; community centers; daycare; family camps for children, the elderly, the disabled, and the general public; information and referral services; group food and food car services; self-help and mutual aid activities for the disabled and vulnerable groups; adult counseling programs; institutional care for adolescents; and institutional care for adults and children. Kim [18] pointed out that social service generally has a characteristic of public service because of the nature of personal service and the intervention of national or public resources at a certain level.
The concept of social service differs depending on the viewpoints and on the scholars discussing the topic; however, social service can be identified as a public welfare service for which government resources are used to improve the quality of life of people by providing services depending on the need for collective response to individual needs. Jang [19] defined social welfare service by focusing on the ex-post nature that provides assistance to families and individuals suffering from loss or facing problems with personal functioning and adaptation. Lee [20] and Park [21] defined program activities of social welfare facilities and institutions as social service. Furthermore, Nam and Cho [22] considered social service as an activity to resolve an individual’s problem using professional knowledge and methods, thereby enabling the individual to return as a normal member of society. Lee et al. [12] considered social welfare service as a service provided by a society for social adaptation, focusing on members’ non-material social relationship needs. Meanwhile, Lee and Kim [23] defined social service as a “social welfare aspect of the service” and a society’s collective, non-profit-seeking, and personal intervention activity to improve the quality of life (welfare) of an individual and increase the opportunity for daily participation. They stated that social service is a concept that encompasses both “social service in consultation” defined by the Social Security Act and “social service” that has a strong welfare nature as a social job.
On the basis of the above discussion, in this study, social service can be defined as a service provided through intervention for the needs of individual service users, that is, social risks that cannot be resolved by direct cash payment, and problems that may arise in interaction with individuals, families, and the environment surrounding them. Such social service can be classified in various approaches. Gilbert [15] classified the needs of the continuum concept, as well as social service in terms of legal relationship, which serves as the basis for providing benefits at the vertical level. Hansenfeld [24] presented social service as human identification, human maintenance, and human change services in accordance with the purpose of the social welfare service. Human identification service refers to a service function that evaluates to provide service. In other words, the selection of beneficiaries is called human identification service. Human maintenance service is a service that restores and prevents residual functions such that one’s present situation does not deteriorate further. It enables one to maintain a certain level, such as in hospices, for example. Lastly, human change service is a service aimed at improving the various social functions of an individual to a more desirable state from the present state, unlike the two aforementioned services.
However, Mrazrek and Haggerty [25] classified social service into post-treatment service aimed at problem-solving and preventative service aimed at intervening in the cause of the problem. Furthermore, by combining these classifications, Kim et al. [26] presented an analysis framework of the social service model (Figure 1). As shown in Figure 1, social service is sub-divided as human maintenance service, and human change service classifications are placed on the horizontal axis and preventative (universal) and post-treatment (selective) service classifications are placed on the vertical axis.

2.2. System and Principles of Social Service

The social service system can be examined in association with the concept of social welfare administration. Social welfare administration is defined as the management of all the activities of organizations in the process of converting an abstract and general policy into a concrete service [7]. In this regard, the social service system is a structural arrangement of organizations in the process of converting a policy into a service. Gilbert et al. [27] described the social service system as an organizational arrangement that exists between a service provider and consumer in the context of a local community.
The social service system is important in that the criteria, which are generally applied when social responsibility is required for social service, question the effectiveness or efficiency of an individual organizational unit, and there are many cases where an organization’s individual unit is not effective in terms of the system even when the organization is effective as a whole. That is, although organizations can achieve effectiveness by devoting themselves to the provision of service for their clients with the given resources, clients may be missing or the service may be duplicated if the system for inter-organizational collaboration or collaboration does not work appropriately [8]. Kim [28] argues that a systematic approach is important, one that analyzes the existence of coordination and integration of the relationship and role from an integrated perspective rather than individually grasping the organizations that constitute the system, identifying whether the service needs of the community or clients are satisfied effectively.
Kahn and Kamerman [16] define an intervention program as a service system integration that resolves a fragmented service system problem and is a specialized service and holistic and systematic task involved in resolving the inconsistency between the problems or the needs of individuals and families in a community. Strategic methods for integrating the social service system can be divided into system-oriented and service-oriented methods. A system-oriented strategy seeks to achieve a high level objective close to reform, aiming at developing or changing the basis of the social service system. Therefore, a system-oriented strategy generally aims to change the manner in which institutions organize and fund the programs to achieve the objectives of developing a new system or reorganizing the existing organization, developing new services to fill the gaps in the existing service system or responding to client needs that are unmet in the community, and reducing the inconsistency or contradictory procedures between the existing service programs.
This system-oriented social service is, in general, a concept that aggregates different service organizations or programs into one place (or near) through physical reorganization of organizations or service programs. Of course, service integration cannot be achieved instantly by external pressure [29]. The common view of the academic world is that social service system integration is a phased development process on a continuum, progressing step-by-step over time [30]. In this respect, even if physical system integration is an ultimate goal, service coordination should be conducted through a relationship between the related factors [31,32,33,34]. Therefore, discussion on the integration and coordination of the social service system can be divided into three main approaches: service delivery system, service administration, and service network.

2.2.1. Service Delivery System

The social service delivery system comprehensively refers to an organization and the labor that enable the transition of the abstract social welfare law and welfare policy into a specific welfare service that is delivered to those in need of the service. According to Gilbert et al. [15], the delivery system is one of the most important issues that needs to be fundamentally considered when establishing welfare policy and service.
Walsh [34] stated that the delivery system can be divided into the public sector delivery system and private sector delivery system, which is an appropriate and necessary process for welfare recipients, and that the delivery system is an organized arrangement among the administrative organization, deliverer, and welfare recipient. They emphasized the administrative organization of the delivery system and classified the composition into public and private sectors. Noveck [35] classified the subjects of the delivery system into social welfare customer or service provider and recipient, and the components into public and private sectors by defining the delivery system as a systematic device that connects a service provider, including public and private sectors, to a service recipient, in order to provide service to a social welfare customer.
With regard to the principles of the social service delivery system, Bennett and Weisinger [36] and Gilbert and Terrell [15] argued for inclusiveness, accessibility, continuity, non-fragmentation (integration), accountability, integration, and accountability. Previous researchers [37,38,39,40,41] argued for accessibility, sustainability, suitability, inclusiveness, integration, impartiality, adequacy, professionality, equality, and accountability. Burkhardt and Brass [39] mainly emphasized the organizational structure and systemicity of the operation (service provision) in terms of function. In addition, on the basis of a long-term research on the public welfare delivery system, Brown and Potoski [1] and Park and Kim [42] conducted an analysis with connectivity, efficiency, community participation, complementarity, adequacy, professionality, evaluation, and immediateness. In these studies, the four influencing factors of the service delivery system were used, namely, the ‘integration’, ‘accessibility’, ‘functional systemicity’, and ‘participation possibility’ of the service system.

2.2.2. Service Administration

The importance of administrative aspects of the social service system is emphasized because citizens’ level of satisfaction with social service affects government trust. The government should combine human and material production factors to develop an appropriate administrative service and deliver it to the people quickly and accurately. Today, administrative service is considered as the most important factor that determines the quality of life of residents, as the level of satisfaction for administrative service is directly related to quality of life.
Bovaird and Löffler [43] emphasize citizens’ fulfillment of their right to know about an administrative agency’s policies, decision-making content, and decision-making processes utilized to achieve citizen-centered administrative service. Furthermore, Stewart emphasizes administrative agencies’ discussions with citizens regarding important matters of the project and accurate delivery of the policy direction, citizens’ right to participate in the policy-making process, and their right to participate as an evaluator of policy outcome. If an administrative agency does not function appropriately in terms of the content and quantity of administrative service, it will ultimately lose confidence in terms of competitiveness and efficiency [44]. Therefore, factors of service administration, which operate and manage the system, need to be considered for the social service system.
As an important factor of service administration, Bachman [45] presented the interest in accountability, which resulted from an increase in privatization of social services, compliance with operational regulations, professionalism of the personnel, and compliance with accounting regulations. As the factors of service administration, Kettner and Martin [46] presented the customer’s level of functional development, communication satisfaction, and spread of consensus. Shakespeare [47] and many other researchers [48,49,50,51] emphasized usability, feasibility, accuracy, adequacy, meaningfulness, validity, verifiability, connectivity, and sensitiveness. In addition, Burns et al. [52] and Kwak [53] presented organization’s mission, expertise, experience, and stakeholder coordination, confidence recovery, representative democracy healing, and problem-solving skills. In these backgrounds, we selected the four factors, ‘accountability’, ‘problem-solving ability’, ‘connectivity’, and ‘professionality’, as sub-factors of service administration on the social system.

2.2.3. Service Network

Networking between social welfare agencies is conducted as an essential strategy in the rapidly changing organizational environment [54]. In particular, effective service delivery, uncertainty of budget securing, and diversification tendency of social welfare providers has brought attention to networking. In addition, in terms of social welfare practice and clients, networks between organizations have an important value [55]. In general, for clients with diverse psychosocial needs, individual social welfare organizations are limited in providing only limited resources and services. Therefore, activation of inter-organizational relationships is important for the provision of integrated service [56]. As part of the reorganization of the social welfare delivery system, governments are actively attempting to revitalize network cooperation between private and public institutions, that is, activating networking [57,58,59,60].
Alter and Hage [61] emphasized the importance of a cooperative relationship between individual organizations, cooperation process, degree of network participation, and structure of individual organizations. Marsden [62] went beyond individual organizational dimensions to address key factors in terms of inter-organizational network level or overall network level. From a stakeholder’s perspective, Weech-Maldonado [63] pointed out factors such as role, influence, and responsibility among groups that substantially affect and are affected by a particular issue of stakeholders or organizations, whereas Scott and Davis [64] and other related literature [65,66,67,68] emphasized the importance of stakeholder specification, information gathering, utilization, and clear objective and definition. On the basis of these suggestions, we defined the four influencing factors of service network as ‘reliability’, ‘shared decision-making’, ‘clarity of role-sharing’, and ‘information sharing and communication’.

3. Research Methods

3.1. Analytic Hierarchy Process (AHP)

The analytic hierarchy process (AHP), as a new decision-making methodology that captures the knowledge, experience, and intuition of the evaluator through a pairwise comparison between elements that constitute the hierarchy of decision-making, was developed by Thomas Saaty, a professor at the University of Pennsylvania, in the early 1970s [69]. The AHP has the characteristics of simplicity of theory, clarity, application, and generality; therefore, it is widely used to derive key factors, establish policy alternatives, and formulate strategies [70]. In particular, the AHP methodology needs to obtain a comprehensive ranking of the alternatives of the subject of evaluation by compiling the relative importance of factors that are calculated by several evaluators [71,72]. Therefore, this study used AHP to assess the importance of factors in determining what factors are important for successful and sustainable maintenance of the social service system for the elderly.
There are two ways to compile the results of several evaluators. The first is a geometric mean of pairwise comparison method that recalculates the relative importance of an element from the comprehensive pairwise comparison matrix, constructed by calculating the geometric mean of each element of the pairwise comparison matrix for each evaluator. The second is an arithmetic mean of importance weight method that calculates the arithmetic mean for each element in the relative importance of each element by the evaluator. Although the analysis results of the two methods do not significantly vary, in this study, the relative importance of multiple decision-making alternatives was integrated using the commonly-used geometric mean of pairwise comparison method [73]. Moreover, since AHP is a type of a probabilistic method that evaluates the uncertainty between input variables and set model output variables, the weight was set to establish the calculation basis of the input variables and the reliability of the results.
First, the Delphi technique was used to verify the reliability and validity of the elements derived through previous studies. The Delphi technique is used to identify the priority and importance of various opinions through intensive questions centering on experts regarding a fundamentally important issue, or to coordinate or integrate group opinions for improvements as a sequencing technique [74]. In this study, prior to the composition of the AHP questionnaire, opinions were obtained on the reliability and validity of the elements of the AHP questionnaire for five subjects, including two professors and three experts with more than 15 years of work experience in social service, for 2 weeks in May 2019, from which the final evaluation factors were determined.

3.2. Research Framework and Variables

As shown in Figure 2, the research framework was designed for constructing sustainable success factors of the social service system for the elderly on the basis of previous studies on the component and success factors of the social service system. The main evaluation areas consisted of three categories.
First, the “service delivery system” refers to the influencing factors in the process of providing service to the subject in the social service system for the elderly. On the basis of the precedence of Burkhardt and Brass [39] and Park and Kim [42], in this study, integration, accessibility, functional systemicity, and participation possibility of the service system, which may be needed in the process of operating and providing the social service system, were designed as sub-factors. Second, “service administration” was formed through sub-factors on the accountability of the system operator, conflict resolution mechanism, continuous intervention, and professionality, by reflecting on the nature of the involvement of administrative activities, such as government or public institutions in the operation of the social service system based on the research by Bachman [45], Kettner and Martin [46], and Moulton [49]. Lastly, as “service network” is characterized by network-based activities with various stakeholders, ranging from social enterprises, private organizations, and NGOs(non-governmental organizations) to companies, in the case of the social service system, the effects of factors, such as reliability within the relationship, shared decision-making, clarity of role-sharing, and information sharing and communication on sustainable success, were examined on the basis of previous studies by Provan et al. [45], Scott and Davis [64], and Wedel and Colston [66] (Table 1).

3.3. Research Process and Data Collection

To derive the critical factors that influence sustainable success of the social service system in an objective and empirical manner, this study first developed a questionnaire using the AHP technique on the basis of the research framework. The questionnaire consisted of a total of 24 questions, including 1 short-answer question and 23 multiple-choice questions using a pairwise comparison scale, and pilot tests were conducted on five experts, directors, and managers in a social welfare service center and foundation in South Korea.
The research subjects included social service staff and recipients of the Seoul 50 Plus, who formed a professional group with high experience and understanding of the social service system. The Seoul 50 Plus Foundation promotes a private sector-linked 50 plus social service policy that helps to transform the crisis of early retirement into a new opportunity in association with private institutions and civil society. It aims to provide a sustainable social participation opportunity for early retiree seniors, moving beyond employment support policy. Therefore, a survey was conducted for a total of 28 people, including 13 service providers, who experienced the social service system for the typical elderly, and 15 service users.
The survey was conducted for two months from June to July 2019, and an individual one-to-one face-to-face interview survey was conducted for about 40–60 minutes for each individual. Finally, data analysis reviewed the significance and reliability of the questionnaire results, which was based on the questionnaire completed by 24 subjects with more than 7 years of experience in the public service system, including 12 service providers and 12 service users. As shown in Table 2, the final data sample showed that the ratio of females was greater than that of males, and that females accounted for 66.7% and males for 33.3%, respectively. Those in their 50s and 60s accounted for 41.7% and 25%, respectively, while those over 50 represented 66.77%. Those with 10–20 years of work experience in public service accounted for 54.3%, and thereby more than 87.5% of the subjects had more than 10 years of experience. MS Excel software was used for the analysis. The consistency ratio (CR) of the respondents, which shows how consistently the comparison subjects responded, was 0.061–0.0259, and all were less than 1, and therefore were significant.

4. Analysis Results

4.1. Comparison of Evaluation Variables

As shown in Table 3, the weight of the service delivery system, service administration, and service network was found to be 0.549, 0.246, and 0.205, respectively. Thus, the service delivery system was found to be the most important factor for sustainable success of the social service system. Considering each area, accessibility (0.284) and participation possibility (0.284) were found to be more important factors than functional systemicity (0.228) and integration (0.204) among the four factors of the service delivery system. With regard to the factors for service administration, accountability (0.401) was found to be the most important factor, followed by professionality (0.377), connectivity (0.116), and problem-solving ability (0.106). Lastly, with regard to the area of service network, reliability (0.422) was presented as the most important factor, followed by information sharing and communication (0.241), clarity of role-sharing (0.178), and shared decision-making (0.158).
Furthermore, considering the overall ranking of the 12 factors rather than the ranking by area, the factors that had the biggest effect on the success of the social service system were found to be accessibility (0.156) and participation possibility (0.156), followed by functional systemicity (0.125) and integration (0.112). As a result, the service delivery system is shown to be more important than service administration or service network for sustainable success of the social service system. Moreover, the factors that constitute the service delivery system were confirmed to be serving as overwhelmingly important factors as compared to the other factors.

4.2. Comparison of Evaluation Areas between Service Providers and Service Users

As shown in Table 4, the comparison analysis result of the evaluation areas between a service provider and service user suggested that the service delivery system was the most important factor in both groups. In terms of a service provider, the service delivery system was the most important factor with 0.577, followed by service administration with 0.224, and service network with 0.199. In addition, in terms of a service user, the ranking result was the same, as the service delivery system was the most important factor with 0.515, followed by service administration with 0.271, and service network with 0.214.

4.3. Comparison of Evaluation Factors between Service Providers and Service Users

The comparison analysis result of the evaluation factors between a service provider and service user suggested that functional systemicity (0.183) was considered as the most important factor for service providers, whereas participation possibility (0.183) was selected as the most important factor for service users. Therefore, the needs pertaining to the provider and user side were reflected. Accessibility was chosen as the second most important factor in both service providers (0.167) and service users (0.135), and, thus, system accessibility is a factor that needs to be considered in any situation in case with regard to the social service system (see Table 5).
Considering the results of the 3–5 ranking analysis of other factors, participation possibility (0.126) was derived as the most important factor, followed by integration (0.101), and accountability (0.086) on the service provider side, whereas integration (0.118) was derived as the most important factor, followed by accountability (0.113), and reliability (0.110) on the service user side. Moreover, it was found that integration and accountability were considered important factors in both groups.

5. Conclusions

In general, the field of welfare service is operated as a center of social service oriented toward active welfare through job creation, as it responds to new social problems, such as low birthrate, aging, and increasing participation of women in economic activities, along with selective welfare services aimed at supporting the existing vulnerable social groups [3]. Furthermore, the needs of families and individuals addressed by social service are diverse and usually overlap. For example, many families in the community need various social services or welfare benefits, such as childcare, work training, healthcare, and counseling, to be successful in the labor market. For the concept of simplifying and streamlining the process by which clients in the community are exposed to such complex services or benefits, establishment of a redundant, interconnected, uncoordinated, and confusing service system in the community has previously been suggested as a solution [55,75]. Rather than individually grasping the organizations that constitute the system, an integrated approach is important to analyze the existence of coordination and integration of the relationship and role, and to systematically understand whether the service needs of the community or service users are satisfied effectively.
To this end, this study aimed to present key areas of the social service system in terms of the delivery system, administration and network, and critical factors for successful operation, and to explore the developmental direction for management of the social service system by objectively presenting the critical factors. In particular, AHP was used for the prioritizing processes and pairwise comparison of factors that comprise the importance of the system success and sustainability.
The analysis results showed that, firstly, the delivery system is more important than the administration or network in the social service system. As mentioned in many previous studies [58,59,60], the fact that the service delivery system is important in the area of social service or welfare service has been verified in this study as well. However, in this study, as compared to previous studies focusing on the integration of the delivery system, accessibility and participation possibility were considered the most important factors of the delivery system. Therefore, accessibility and participation are considered more important factors for service users. This means that although the physical system for the integrated and functional structure is important, accessibility and participation must be considered in terms of interaction by which one uses the system.
Moreover, the analysis results pointed out the importance of administrative accountability. The results highlighted the importance of considering accountability as an important factor in terms of relationship, rather than administrative tasks such as problem-solving ability and connectivity. As mentioned by Jung and Ko [76], as social service has been addressed from the government support policy perspective, there has been a lack of discussion on the quality, post-implementation assessment, or sustainability of social service. The characteristic of social service is that it results in an irrational system in which the issue of accountability becomes ambiguous in the event of a problem or lack of support, and consequential damage is therein experienced by service users. In this respect, the need for an integrated process approach was ultimately confirmed, which addresses the pre- and post-social service system and the strengthening of subsequent problem-solving and accountability. In addition, a fair distribution of service should be conducted such that recipients can be aware of equity, and the responsibility and sense of duty of social workers who provide service should be promoted.
On the basis of the analysis results in this study, four implications can be presented. First, it is necessary to identify the needs and desires of service users rather than of the government or public institutions, and to make efforts for interactivity to benefit the development of the social service system. User usability is of paramount importance for the operation and management of the system beyond the objective of building the system itself, and it is necessary to establish a system operational process such that the accountability of the government and the institution (that users perceive) and the reliability between the stakeholders can be well-formed. Furthermore, it is necessary to find clients in the blind-spot and provide services by expanding the services that are directly provided rather than services for which users are for which users are on board the characteristics of beneficiaries.
Second, there is a need to determine ways in which to increase the professionality of the service system itself. It is necessary to efficiently establish a welfare service processing system that plans, delivers, and evaluates the service. The growing literature [77,78] on service integration has documented several advantages of well-designed integrated approaches, such as tailor-made, flexible, and responsive policy options; cost effectiveness and efficiency by sharing knowledge, expertise, and resources across cooperating actors; and capacity building and innovation, among others, along with considerable challenges in how to reach these outcomes [79]. Concerning the social service system and delivery, one of the main challenges is the coordination and balancing of the support provision with service incentives and other forms of activation.
Third, there is a need to consider a direction for the establishment of a system aimed at service accessibility rather than the establishment and operation of the social service system itself. If service accessibility is increased, economical effects by the time and cost of accessing the service can be sought, along with the simplification process. In this respect, service systems and processes that consider the accessibility of users’ rather than the provider’s position should be established. Social service management may change skill requirements in several levels and positions of the system [80]. If these skills are missing or inadequate, there is a need for capacity building at the beginning of the implementation stage. Retraining may be necessary, even if existing staff are highly trained. The pace of the reform process also matters. Piloting in a small scale, expanding gradually, and allowing time for detecting and correcting problems, along with adjusting staff capacities and skills, can further improve outcomes, even if the initial plan was already good enough to guarantee success. In addition, this process should be formed by experts. Continual education and training should be provided to help social workers and service personnel improve their process understanding and expertise.
Fourth, a direction for network system development in accordance with the expansion of the social service stakeholders needs to be discussed in more detail. Today, social service is evolving in the form of sharing and synergy, moving beyond the level of simple support or benefit. Therefore, an expanded network should be established through associations with various private organizations, social organizations, and corporations, beyond public and non-profit organizations. Social service systems require more intense cooperation between public agents. Systems will work smoothly and efficiently if the roles are clearly divided and allocated on the right level of government and in related institutions [81].
Nevertheless, this study has the following limitations. First, this study has factor analysis limitations in that it conducted an analysis by defining 3 areas and a total of 12 factors for the delivery system, administration, and network of the social service system. Considering various academic approaches and previous studies related to social welfare, various factors, such as contingency, consensus, and impartiality, are influential, and thus advanced research is required from a broader perspective. Moreover, this study has limitations in terms of generalization of the analysis results, given the limited scope of the study that focused on Seoul 50 Plus Foundation personnel and users in South Korea. Since social welfare systems address a diverse range of projects that target a wide range of users, including children, women, and the disabled, in addition to the 50 Plus Foundation project for the elderly, further research that expands the scope of research subjects may be needed to derive more objective and empirical factors. Lastly, although this study presented the factor analysis results focused on an AHP-based interactive hierarchy comparison, it needs to empirically clarify the causal relationship regarding how the factors actually affect sustainable success of the social service system.

Author Contributions

Funding acquisition, M.L.; methodology, B.K.; resources, M.L.; supervision, B.K. and M.M.; writing—original draft, B.K.; writing—review and editing, M.M.

Funding

This research received no external funding.

Acknowledgments

This research was supported by aSSIST (Seoul School of Integrated Sciences and Technologies).

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Brown, T.L.; Potoski, M. Contract-management capacity in municipal and county governments. Public Adm. Rev. 2003, 63, 153–164. [Google Scholar] [CrossRef]
  2. Reid, W.J. Service effectiveness and the social agency. Adm. Soc. Work 1988, 11, 41–58. [Google Scholar] [CrossRef]
  3. Peat, B.; Costley, D.L. Effective contracting of social services. Nonprofit Manag. Leadersh. 2001, 12, 55–74. [Google Scholar] [CrossRef]
  4. Geyer, R. Globalization and the (Non-) defence of the Welfare State. West Eur. Politics 1998, 21, 77–102. [Google Scholar] [CrossRef]
  5. May, C. Stages in the International Political Economy—Retreat or Transition? Rev. Int. Political Econ. 1998, 5, 157–163. [Google Scholar]
  6. Yeates, N. Globalization and Social Policy; SAGE Publications Ltd.: Thousand Oaks, CA, USA, 2001. [Google Scholar]
  7. Gooden, V. Contracting and negotiation: Effective practices of successful human service contract managers. Public Adm. Rev. 1998, 58, 499–509. [Google Scholar] [CrossRef]
  8. Gronbjerg, K.A. Understanding Nonprofit Funding: Managing Revenues in Social Services and Community Development Organizations; Jossey-Bass: San Francisco, CA, USA, 1993. [Google Scholar]
  9. DeHoog, R.H. Evaluating human services contracting: Managers, professionals and politicos, state and local. Gov. Rev. 1986, 18, 37–46. [Google Scholar]
  10. Hasenfled, Y. Human Services as Complex Organizations; Sage: Thousand Oaks, CA, USA, 2009. [Google Scholar]
  11. Van Slyke, D.M. The mythology of privatization in contracting for social services. Public Adm. Rev. 2003, 63, 296–326. [Google Scholar] [CrossRef]
  12. Lee, H.J.; Kang, H.K.; Seo, M.H.; Jung, K.H.; Yoo, D.C.; Jung, J.H.; Lee, S.K.; Roh, E.J.; Hyun, M.L. System Analysis and Reorganization Measure of Public Assistance and Social Welfare Service; Korea Institute for Health and Social Affair: Seoul, Korea, 2003. [Google Scholar]
  13. Nam, C.S. Retrospect and prospect on the Korean welfare state: Welfare reform in the period of productive welfare and its legacies. J. Crit. Soc. Welf. 2011, 32, 133–180. [Google Scholar]
  14. Plant, R.; Lesser, H.; Taylor-Gooby, P. Political Philosophy and Social Welfare: Essays on the Normative Basis of Welfare Provision; Routledge & Kegan Paul Ltd.: London, UK, 1980. [Google Scholar]
  15. Gilbert, N.; Specht, H.; Terrell, P. Dimension of Social Welfare Policy, 3rd ed.; Prentice-Hall: Englewood Cliffs, NJ, USA, 1993. [Google Scholar]
  16. Kang, H.K. Employment policy in the social service sector. Hyg. Welf. Forum 2008, 144, 34–54. [Google Scholar]
  17. Kahn, A.J.; Kamerman, S.B. Social Services in International Perspective: The Emergence of the Sixth System; Transaction Books: New Brunswick, NB, Canada, 1980. [Google Scholar]
  18. Kim, E.J. Changes in financial supporting flow on social service areas and quality-related policy issues. Soc. Welf. Policy 2008, 35, 141–168. [Google Scholar]
  19. Jang, I.H. Introduction of Social Service; Seoul National University Press: Seoul, Korea, 1990. [Google Scholar]
  20. Lee, C.H. Quantification of social welfare service program. Soc. Welf. Study 1991, 3, 169–200. [Google Scholar]
  21. Park, K.I. A case study on the welfare QC activity as a management technique for improving quality in social welfare service. Korean J. Soc. Welf. 2000, 40, 97–130. [Google Scholar]
  22. Nam, S.J.; Cho, H.S. Korean Social Welfare; Nanam Press: Seoul, Korea, 1995. [Google Scholar]
  23. Lee, B.J.; Kim, M.G. The issues and future of Korean social welfare provision system. Academy of Korean. In Proceedings of the Social Welfare Administration Conference, Seoul, Korea, 25 July 2007; pp. 1–25. [Google Scholar]
  24. Hasenfeld, Y. Human Service Organizations; Columbia University Press: New York, NY, USA, 1983. [Google Scholar]
  25. Mrazrek, P.J.; Haggerty, R.J. Reducing Risk for Mental Disorders: Frontiers for Preventive Intervention Research; National Academy of Medicine Press: Washington, DC, USA, 1994. [Google Scholar]
  26. Kim, S.K.; Choi, I.S.; Choi, S.J.; Cho, H.S.; Kim, H.R.; Lee, B.J.; Koo, I.H.; Kang, S.K.; Ahn, S.H. Introduction of Social Welfare; Nanam Press: Seoul, Korea, 2010. [Google Scholar]
  27. Gilbert, N.; Terrell, P. Dimensions of Social Welfare Policy, 5th ed.; Allyn and Bacon: Boston, MA, USA, 2002. [Google Scholar]
  28. Kim, Y.J. Social Welfare Administration; Hakjisa Press: Seoul, Korea, 2010. [Google Scholar]
  29. GAO (U.S. General Accounting Office). Welfare Reform: Interim Report on Potential Ways to Strengthen the Federal Oversight of State and Local Contracting; GAO-02-245; U.S. General Accounting Office: Washington, DC, USA, 2002; Volume 26–31.
  30. Sanger, M.B. The Welfare Marketplace: Privatization and Welfare Reform; Brookings Institution Press: Washington, DC, USA, 2003. [Google Scholar]
  31. Kettner, P.M.; Martin, L.L. Contracting for services: Is politics a factor? N. Engl. J. Hum. Serv. 1989, 9, 15–23. [Google Scholar]
  32. Kettner, P.M.; Martin, L.L. Performance, accountability and purchase of service contracting. Adm. Soc. Work 1993, 17, 61–79. [Google Scholar] [CrossRef]
  33. Kramer, P.M.; Grossman, B. Contracting for social services: Process management and resource dependencies. Soc. Serv. Rev. 1987, 61, 32–55. [Google Scholar] [CrossRef]
  34. Walsh, K. Quality and public administration. Public Adm. 1991, 69, 508–513. [Google Scholar] [CrossRef]
  35. Noveck, B. Wiki Government: How Technology Can Make Government Better, Democracy Stronger, and Citizens More Powerful; Brookings Institution Press: Washington, DC, USA, 2010. [Google Scholar]
  36. Bennett, E.C.; Weisinger, M. Evaluation: Alternative Models; Prentice Hall: Upper Saddle River, NJ, USA, 1977. [Google Scholar]
  37. Moon, S.Y.; Yoon, K.C. An analysis of the performance of social welfare service: Focusted on the methods of DEA and SERVQUAL. Korean Assoc. Gov. Stud. 2004, 38, 201–224. [Google Scholar]
  38. Park, N.H.; Min, S.Y. An exploratory study on the delivery system of social welfare service: Focusing on the community care services for the elderly. Korean J. Soc. Welf. Adm. 2008, 10, 57–83. [Google Scholar]
  39. Burkhardt, M.E.; Brass, D.J. Changing patterns or patterns of change: The effect of a change in technology on social network structure and power. Adm. Sci. Q. 1990, 42, 339–365. [Google Scholar] [CrossRef]
  40. Ammons, D.N. Overcoming the inadequacies of performance measurement in local government: The case of libraries and leisure services. Public Adm. Rev. 1995, 55, 37–47. [Google Scholar] [CrossRef]
  41. Martin, L.; Kettner, P. Measuring the Performance of Human Service Programs; Sage Publications: Thousand Oaks, CA, USA, 1996. [Google Scholar]
  42. Park, G.K.; Kim, Y.I. A study on the status and development plan of local public service delivery system. Korean J. Policy Anal. Eval. 2009, 19, 35–55. [Google Scholar]
  43. Bovaird, T.; Löffler, E. Understanding Public Management and Governance; Taylorfrancis.com; Routledge: Abingdon-on-Thames, UK, 2004. [Google Scholar]
  44. Hwang, S.S.; Ahn, J.S. Community mapping to promote the regional informatization. Reg. Informatiz. 2013, 79, 106–109. [Google Scholar]
  45. Bachman, S.S. Contracting for Mental Health Services: Six State Experiences. Interdisciplinary Doctoral Dissertation, Brandeis University, Waltham, MA, USA, 1994. Unpublished work. [Google Scholar]
  46. Kettner, P.M.; Martin, L.L. Issues in the development of monitoring systems for purchase of service contracting. Adm. Soc. Work 1985, 9, 69–82. [Google Scholar] [CrossRef]
  47. Shakespeare, T. The Social Relation of Care. In Rethinking Social Policy; Lewis, G., Gewirtz, S., Clarke, J., Eds.; Sage: London, UK, 2002. [Google Scholar]
  48. Rowe, G.; Frewer, L.J. Public participation methods: A framework for evaluation. Sci. Technol. Hum. Value 2002, 25, 3–29. [Google Scholar] [CrossRef]
  49. Moulton, S. Putting together the publicness puzzle: A framework for realized publicness. Public Adm. Rev. 2009, 69, 889–900. [Google Scholar] [CrossRef]
  50. Anderson, S. Public, private, neither, both? Publicness theory and the analysis of healthcare organizations. Soc. Sci. Med. 2012, 74, 313–322. [Google Scholar] [CrossRef] [PubMed]
  51. Fox, A. The New Social Care: Strength Based Approach; Shared Lives Plus: Liverpool, UK, 2013. [Google Scholar]
  52. Burns, T.R.; Roszkowska, E.; Corte, U.; Des Johansson, N.M. Linking group theory to social science game theory: Interaction grammars, group subcultures and games for comparative analysis. Soc. Sci. 2017, 6, 107. [Google Scholar] [CrossRef]
  53. Kwak, H.G. The study of residential influential factors on local community social capital. Korean Assoc. J. Public Manag. 2013, 27, 237–267. [Google Scholar]
  54. Provan, K.G.; Milward, H.B. A preliminary theory of network effectiveness: A comparative study of four community mental health systems. Adm. Sci. Rev. 1995, 40, 1–33. [Google Scholar] [CrossRef]
  55. Provan, K.G.; Fish, A.; Sydow, J. Interorganizational networks at the network level: A review of the empirical literature and whole networks. J. Manag. 2007, 33, 479–516. [Google Scholar] [CrossRef]
  56. Woo, J.J. A study on the management factor in the collaboration among social service organizations. Asian Soc. Work Policy Rev. 2006, 58, 37–63. [Google Scholar]
  57. Ahuja, G. Collaborative networks, structural holes, and innovation: A longitudinal study. Adm. Sci. Q. 2000, 45, 425–455. [Google Scholar] [CrossRef]
  58. Lee, E.E.; Kang, H.J.; Ahn, H.J. Word-of-mouth of cultural products through institutional social networks. Sustainability 2017, 9, 917. [Google Scholar] [CrossRef]
  59. Shin, C.H.; Park, J.K. Classifying social enterprises with organizational culture, network and socioeconomic performance: Latent profile analysis approach. J. Open Innov. Technol. Mark. Complex. 2019, 5, 17. [Google Scholar] [CrossRef]
  60. Kilduff, M.; Tsai, W. Social Networks and Organizations; Sage: Thousand Oaks CA, USA, 2003. [Google Scholar]
  61. Alter, C.; Hage, J. Organizations Working Together; Sage: Newbury Park, CA, USA, 1993. [Google Scholar]
  62. Marsden, P.V. Network data and measurement. Annu. Rev. Sociol. 1990, 16, 435–463. [Google Scholar] [CrossRef]
  63. Weech-Maldonado, R.; Benson, K.J.; Gamm, L.D. Evaluating the effectiveness of community health partnerships: A stakeholder accountability approach. J. Health Hum. Serv. Adm. 2003, 26, 59–92. [Google Scholar]
  64. Scott, W.R.; Davis, G.F. Organization and Organizing; Pearson: Upper Saddle River, NJ, USA, 2007. [Google Scholar]
  65. Borgatti, S.P.; Foster, P.C. The network paradigm in organizational research: A review and typology. J. Manag. 2003, 29, 991–1013. [Google Scholar]
  66. Wedel, K.R.; Colston, S.W. Performance contracting for human services: Issues and suggestions. Adm. Soc. Work 1998, 12, 73–87. [Google Scholar] [CrossRef]
  67. Galaskiewicz, J. Has network theory of organizational behavior lived up to its promise? Manag. Organ. Rev. 2007, 3, 1–18. [Google Scholar] [CrossRef]
  68. Provan, K.G.; Milward, H.B. Do networks really work? A framework for evaluating public-sector organizational networks. Public Adm. Rev. 2001, 61, 414–423. [Google Scholar] [CrossRef]
  69. Udo, G.G. Using analytic hierarchy process to analyze the information technology outsourcing decision. Ind. Manag. Data Syst. 2000, 100, 421–429. [Google Scholar] [CrossRef]
  70. Yang, C.; Huang, J.B. A decision model for IS outsourcing. Int. J. Inf. Manag. 2000, 20, 225–239. [Google Scholar] [CrossRef]
  71. Chan, A.H.; Kwok, W.Y.; Duffy, V.G. Using AHP for determining priority in a safety management system. Ind. Manag. Data Syst. 2004, 104, 430–445. [Google Scholar] [CrossRef]
  72. Hafeez, K.; Zhang, Y.B.; Malak, N. Determining key capabilities of a firm using analytic hierarchy process. Int. J. Prod. Econ. 2002, 76, 39–51. [Google Scholar] [CrossRef]
  73. Yoo, S.K.; Kim, B.Y. A decision-making model for adopting a cloud computing system. Sustainability 2018, 10, 2952. [Google Scholar] [CrossRef]
  74. Boyatzis, R.E. The Competent Manager: A Model for Effective Performance; John Wiley & Sons: Hoboken, NJ, USA, 1982. [Google Scholar]
  75. Provan, K.G.; Isett, K.R.; Milward, H.B. Cooperation and compromise: A network response to conflicting institutional pressures. Nonprofit Volunt. Sect. Q. 2004, 33, 489–514. [Google Scholar] [CrossRef]
  76. Jung, Y.H.; Ko, S.J. Social Conflict Index and Economic Growth: Evidence from a Cross-Section of 7 Countries; Health and Welfare Policy Forum: Seoul, Korea, 2015. [Google Scholar]
  77. Tauqeer, M.A.; Bang, K.E. Servitization: A model for the transformation of products into services through a utility-driven approach. J. Open Innov. Technol. Mark. Complex. 2018, 4, 60. [Google Scholar] [CrossRef]
  78. Kim, S.J.; Kim, E.M.; Suh, Y.K.; Zheng, Z.K. The effect of service innovation of R&D activities and government support systems: The moderating role of government support systems in Korea. J. Open Innov. Technol. Mark. Complex. 2016, 2, 6. [Google Scholar] [CrossRef]
  79. Nam, K.; Kim, B.Y.; Carnie, B.W. Service open innovation: Design elements for the food and beverage service business. J. Open Innov. Technol. Mark. Complex. 2018, 4, 53. [Google Scholar] [CrossRef]
  80. Lee, K.Y.; Jung, K.G. Exploring institutional reform of Korean civil service pension: Advocacy coalition framework, policy knowledge and social innovation. J. Open Innov. Technol. Mark. Complex. 2018, 4, 14. [Google Scholar] [CrossRef]
  81. Kim, A. Social Exclusion of Multicultural Families in Korea. Soc. Sci. 2018, 7, 63. [Google Scholar] [CrossRef]
Figure 1. Social service type classification model [26].
Figure 1. Social service type classification model [26].
Joitmc 05 00077 g001
Figure 2. The research framework.
Figure 2. The research framework.
Joitmc 05 00077 g002
Table 1. Evaluation factors and definition.
Table 1. Evaluation factors and definition.
Evaluation AreaEvaluation FactorsFactor DefinitionRelated References
Service delivery systemIntegrationIntegrated system and operating system of the social service systemBurkhardt and Brass [39];
Park and Kim [42]
AccessibilityEase of access to or participation in the social service system
Functional systemicityEfficient and effective organization and role of the social service system
Participation possibilityHigh participation possibility of the social service system
Service administrationAccountabilityResponsible attitude of administrative staff or agency operating the social service systemBachman [45];
Kettner and Martin [46];
Moulton [49]
Problem-solving abilityProblem-solving ability and initiative of administrative staff or agency operating the social service system
ConnectivityLogical and connected operating system of the social service system
ProfessionalityProfessional competence of administrative staff or agency operating the social service system
Service networkReliabilityCooperation and reliability among stakeholders within the network related to the social service systemProvan et al. [55];
Scott and Davis [64];
Wedel and Colston [66]
Shared decision- makingFree discussion culture as well as decision-making and process optimized within the network related to the social service system
Clarity of role-sharingClear and systematic role-sharing among stakeholders in the network related to the social service system
Information sharing and communicationActive information sharing and communication of network stakeholders involved in the social service system
Table 2. Demographic information.
Table 2. Demographic information.
CharacteristicsFrequencyRatio (%)
GenderMale833.3
Female1666.7
Age30s312.5
40s520.8
50s1041.7
60s625
Work experience
in the public service
7–10 years312.5
10–20 years1354.2
Over 20 years833.3
Professional areaService provider1250
Service user1250
Table 3. Weights and priority of evaluation variables.
Table 3. Weights and priority of evaluation variables.
Evaluation AreasThe Weights of AreasEvaluation FactorsThe Weights of Evaluation Factors
LocalLocalPriorityGlobalPriority
Service delivery system0.549Integration0.20440.1124
Accessibility0.28410.1561
Functional systemicity0.22830.1253
Participation possibility0.28410.1561
Service administration0.246Accountability0.40110.0995
Problem-solving ability0.10640.02612
Connectivity0.11630.02911
Professionality0.37720.0936
Service network0.205Reliability0.42210.0877
Shared decision-making0.15840.03210
Clarity of role-sharing0.17830.0379
Information sharing and communication0.24120.0498
Total1.0000 3.0000 1.0000
Table 4. Comparison analysis result on evaluation areas.
Table 4. Comparison analysis result on evaluation areas.
Evaluation AreasThe Weights of Areas
Service ProviderService User
LocalPriorityLocalPriority
Service delivery system0.57710.5151
Service administration0.22420.2712
Service network0.19930.2143
Total1.0000 1.0000
Table 5. Comparison analysis result on evaluation factors.
Table 5. Comparison analysis result on evaluation factors.
Evaluation FactorsThe Weights of Evaluation FactorsPriority of Factors
(by Global)
LocalGlobal
Service ProviderService UserService ProviderService UserService ProviderService User
Integration0.1750.2280.1010.11843
Accessibility0.2900.2620.1670.13522
Functional systemicity0.3170.1540.1830.08017
Participation possibility0.2180.3550.1260.18331
Accountability0.3850.4160.0860.11354
Problem-solving ability0.1380.0800.0310.0221012
Connectivity0.1130.1170.0250.0321211
Professionality0.3640.3870.0820.10566
Reliability0.3270.5130.0650.11075
Shared decision-making0.1490.1610.0300.0341110
Clarity of role-sharing0.1960.1560.0390.03399
Information sharing and communication0.3280.1690.0650.03688
3.00003.00001.00001.0000

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Back to TopTop