Assessment of Digital Co-Creation for Public Open Spaces: Methodological Guidelines

Abstract

The accessibility and quality of public open spaces (e.g., parks, gardens, squares and plazas) are critical for cultural identity development as they provide important gathering points in the urban fabric and offer a place for social activities, enabling interaction among people of different generations and ethnicities. Public open spaces enhance the urban environment by providing important ecological processes and ecosystem services. The current research generates knowledge about co-creation approach to be used to merge the application of information and communication technologies (ICT) with these essential functions of the public spaces. It explores new dynamics of open spaces as a trusted service for the community and expands our understanding of how meditated public open spaces function, paying attention to stakeholders, local context and different social groups. The paper presents the design of Digital Co-Creation Index and methodological guidelines for applying Digital Co-creation monitoring technique for evaluation of co-creation processes in designing attractive, inclusive and responsive public open spaces.

1. Introduction

Creative and smart cities, streets and other public spaces are the most attractive urban spots that galvanize urban development through creative and smart economic processes and cultural identity. Public open spaces are crucial parts of all cities and exist in a great variety of types, forms and sizes (from squares, streets to playgrounds, parks, riverbanks and urban forests), each one providing different environmental and social services for all inhabitants. To increase the quality of urban life, public open spaces deserve priority attention, as they affect the townscape, provide ecological diversity, have relevance for healthy citizens and societal well-being and deliver important economic value. The list of benefits of public open spaces is long and strongly related to different types of spaces and their features in the urban fabric. Besides accessibility for all, they include a wide range of functions supporting the quality of environment, health and wellbeing, from ecosystem services to social cohesion sense of place and identity, possibility of choice and responsiveness to variety of needs related to everyday outdoor use and activities. The list of important aspects of public space includes the multiple stories and the body of narratives that are exercised on a place by users and the values they assign to it. These narratives and connections can be exploited to activate public open space and engage citizens in their improvements.

Another factor to consider is the relentless development of digital technology. In the last decades, ICT (information and communication technologies) and mobile devices have profoundly affected multiple aspects of our daily life—the way we work, learn and communicate with other, and how we spend our free time. The relationship between ICT and public open spaces (POS) is a growing challenge for ICT experts, spatial planners, social scientists and decision-makers [1]. There are different examples of blended digital/public open spaces, e.g., digital displays in cities, Wi-Fi provision in parks and squares, on-the-spot tourist information, broadcasting and interactive art performances, urban games, etc. Several projects, activities and initiatives take up aspects of interaction among users, ICT and social behavior (e.g., CyberParks [2], MobileCity [3], Cyberbullying [4], GreenKeys [5]); others set up ICT systems for spatial analyses and planning methodologies (People Friendly Cities [6]), as well as for a series of social networks and pervasive urban gaming. Participation of residents and participants of activities in the design process of public spaces of the city invite a more dynamic urban life. Another relevant aspect of ICT lies in their ability to enhance communication with (potential) users, transforming the production of public open spaces into an interactive process and enabling wide community participation and empowerment. Thus, ICT represent a valuable source of information that could be used in the production of a more responsive and inclusive urban environment. However, no past or ongoing projects or research initiatives tackle in a systematic approach to the involvement of different users in the production of ICT-enhanced public open spaces.

To address all these issues comprehensively, professional support is needed to structure the issues in a user-friendly way for non-professionals, enabling better-focused discussions and exchange for co-creation and help to develop practical solutions for real place and time implementation. The current research project analyses how ICT and spaces are used together and from there come up with ideas on how to provide this service in a more efficient way and more specifically tuned to the local context and different community members’ needs. The main goal of the paper is to present the design of Digital Co-Creation Index and to offer methodological guidelines for applying Digital Co-creation monitoring technique [7] for evaluation of co-creation processes by designing attractive, inclusive and responsive public open spaces. The application procedure is illustrated with the evaluation results of four European Living Labs—Aukštamiestis Living Lab (Vilnius), Alvalade Living Lab (Lisbon), Città Studi Living Lab (Milano) and Zuid park Living Lab (Ghent). The Living Labs were selected due to their diversity, size, importance as active centers, availability and vibrant involvement of local communities during the implementation of C3Places project.

The paper is structured as follows. Section 1 is the introductory section, in which the main idea of interaction between ICT and public open spaces is analyzed. Section 2 focuses on methodology for designing indices for social phenomena. Section 3 presents procedures applied in the development of digital co-creation assessment tool. Section 4 presents methodological guidelines and results of experimental evaluation of four case studies through Digital Co-creation Index (DCCI) dimensions. Discussion and limitation are outlined in Section 5 and conclusions in Section 6.

2. Designing Indices for Social Phenomena

Scientific literature [8,9,10,11,12] defines a (complex) index as an instrument for qualitative or quantitative assessment of a certain domain that is composed of individual sub-indicators and is used to compare various analyzed subjects. In other words, such indexes are quantities the values of which are determined by applying statistical methods and using statistical data and may, in their turn, be used as input data in the analysis of an observed phenomenon. The index method is typically used to incorporate separate statistical values that may be described using different scales and numeric characteristics, into a certain measurement system. Typically, indexes are sensible, where phenomena depending on numerous variables (e.g., country, economy’s or organization’s competitiveness, market integration, development of knowledge society, etc.) that, due to complexity of the subject or structure of the phenomenon, may not be unambiguously described by a single index. Indexes are aimed to give the estimated subject an accurate and, at the same time, exhaustive rating to correctly describe the general state of the subject. Applied in homogeneous streaks or in any other regular scale, indexes may indicate phenomenon’s long term development trends or short term changes in a state or region and subsequently help adopt political, economic or other administrative decisions. The application of indexes to evaluate social phenomena can qualitatively describe the evolution of such phenomena within a certain territorial unit or any other category (e.g., a group citizens chosen under certain criteria). Major advantages and drawbacks of the use of composite indexes may be found in a manual compiled by Saisana and Tarantola [13]. One of the key problems in the construction of indexes of social phenomena is frequent uncertainty of what exactly has to be measured by means of composite indexes. Such uncertainty constitutes an essential reason preconditioning the complexity of the process of evaluation of social phenomena. According to Foa and Tanners [14], one of the most important tasks in the construction of composite indexes is to decide what data shall be used. The construction of a composite index depends upon whether the provided two or three components should correspond to the evolution of an observed phenomenon or more components are necessary to parameterize and characterize the phenomenon. In the latter case, a problem of accessibility to the necessary data is faced. The traditional index construction methodology incorporates three different levels of structural elements: dimensions, components and indicators. As indexes and indicators describe properties of analyzed subjects by various values, it is necessary to adopt a correct assessment procedure granting an opportunity to accurately compare individual indexes with each other [15]. Therefore, in combining the individual variables into a common index, the value of each indicator is normed [16,17].

3. Digital Co-Creation Index

As mentioned in the previous chapter, “the Index is a numerical value that expresses the statistical relationship between amounts relating to the same phenomenon. The numerical value is precisely what gives us an insight on the phenomenon we hope to analyze and measure” [17]. The proposed Digital Co-creation Index (DCCI) methodology focuses on facilitating the framework to evaluate the co-creation processes for designing attractive, inclusive and responsive public open spaces (POS). The framework summarizes the current research progress on the topic and was developed as a part of C3Places project [7]. According to the concept of the composite index introduced in Section 2, DCCI was developed consisting of three Sub-Indices [18]:

• POS Quality Index evaluates the physical and social aspects of the observed public space that are forming its quality;
• Digital Inclusiveness Index explains technological readiness of the initiative for enabling co-creation and measures preconditions for the inclusiveness of public places;
• Social Responsiveness Index refers to the co-creative maturity of actors (stakeholders and community members) in responding to the social challenges and in generating the public value.

The index construction methodology is a constituent part of DCCI research methodology which fully complies with the system approach to the analyzed subject. On the basis of the composite index construction experience [19], the following stages were distinguished in modelling the Digital Co-creation Index.

• Theoretical review, construction of the conceptual model. The analysis of previous research efforts captured the theoretical influences and provided the basis for the selection of framework dimensions. The POS Quality dimension was developed in combining the Project for Public Places [20] and Quality of Experience frameworks [21], which identified four qualities determining its attractiveness: uses and activities; comfort and image; access and linkages; sociability by evaluating thousands of public spaces globally. The Social Networking Adoption Model [22], which helps the public organizations to weigh the benefits and risks associated with the use of ICT and social networking applications, formed the base for Digital Inclusiveness pillar. Social Responsiveness dimension was adapted from the Collective Intelligence Index [23]. The second step of the process was the expert interviews. The in-depth knowledge provided by the experts on the key evaluation points was particularly suited for broadening the theoretical framework. Nine purposively sampled semi-structured face-to-face expert interviews were conducted to check and improve the theoretical model.
• Selection of evaluation criteria and proposal of assessment guidelines. The qualitative data collected during the interviews were analyzed in the context of respondents’ ideas, arguments and opinions in order to deepen the researchers’ understanding of the analyzed issues. At this stage the methods for data collection were chosen and described.
• Collection of data in the Living Labs (Vilnius, Lisbon, Ghent, Milano). The experimental evaluation of Living Labs involved the use of a newly constructed measurement instrument. In the course of the experiment, the measurement scales were adjusted and improved. The values of the indicators are of a qualitative nature; therefore indicators underwent a qualitative evaluation and were ascribed numeric values that corresponded to their quantitative weight: 0, 0.5 or 1. All calculated indexes depend on the logic-categorical variables that determine the results of survey.
• The values of answers to questions were transformed into a numeric scale in accordance with the following procedure (keeping the property of monotonicity of function and according to the intuitive reasoning). The function f, describing this procedure is defined by following approach: yes—1; no—0. Other categorical variables were transformed into a numeric scale applying the same approach: high—1; medium—0.5; low—0. To ascribe the numeric values, the variables underwent transformation f, which retained the intuitive order of the values of the categorical variables in the set of non-negative real numbers. To preserve measurability features, a set of non-negative numbers has been chosen. If the questions had no responses too often, their corresponding indicators were excluded from the index. If the interview failed to produce data only in several cases, the corresponding indicator was attributed the most frequently recurring value. Such attribution is sufficient for the purposes of the experiment as more complex cases were absent; usually, when frequently recurring numbers include several values, the problem of missing data is addressed by ascribing the missing position the arithmetic mean of the recurrent values.
• Transformation f was also supplemented by rating of indicator values (since the values (and scales) are chosen from the range (0, 1)):
  • We assume that the weighted coefficients of each indicator inside each category is equal;
  • K_i is the estimate of weighted coefficient of i-th category, $K_i=\frac{1}{m_i}{{\displaystyle \sum }}_{j=1}^{m_i}{\widehat{I}}_{ji}$;
  • ${\widehat{I}}_{ji}$ is the transformed estimate of j-th indicator of i-th category using formula ${\widehat{I}}_{ji}=f(I_{ji})$;
  • m_i is the number of variables (indicators) of i-th category;
  • n is the number of categories, defining the Digital Co-Creation Index.

The values of all three composite indices are identified by means of corresponding formulas specified further. Values of the indices fall into the range of real numbers (0, 1). To improve the user perception, the obtained values of the composite indices were transformed into a more attractive scale by multiplying the obtained values by, for example, 100. As the indices have just been introduced, any additional transformations are impossible, until they empirically prove to match the actual data. When the actual data and the values of their indices (or their evolution) differ essentially (in accordance to the corresponding criteria), changes of index defining formulas are necessary to lay down other leverage coefficients of the indicators (first type structural change) or include new indicators (second type structural change).

POS (public open space) Quality Index (POS QI) has 14 variables divided in four categories, which are used to define the attractiveness of open public space. We assumed that the categories were equally significant based on our theoretical insights and all variables used in these categories have the equal weight. POS Quality Index is calculated by applying the following formula for categories (also see

Table A1. The structure of POS Quality Index (QI).

Category	Indicator (Exogenic Variable)
Access and linkages	Level of readability/orientation/wayfinding for all (not only visual types)
	Level of convenience for movement
	Interlinking level
	Level of accessibility
Comfort and image	Level of captivation
	Level of comfort and cleanness
	Level of safety
Uses and activities	Level of equipment
	Level of activities
	Variety of activities
Sociability	Level of welcoming
	Level of publicness
	Level of interactivity
	Level of diversity

in Appendix A) (where AL—Access and linkage; CI—Comfort and image; UA—Uses and activities; S—Sociability):

$$POSQI=\frac{AL+CI+UA+S}{4}$$

The estimates of weighted coefficients of category are estimated by expert assessment. As no numeric data have been collected until the present experiment, there are no possibilities to carry out statistical research and identify statistical significance of each indicator necessary to construct the indexes. Therefore, leverage coefficients of the indicators (or categories) are determined in view of the acquired empirical experience in defining indicator correlation significance.

Digital Inclusiveness Index (DII) has, in total, seven exogenic variables, divided into five categories, which are used to determine the Digital inclusiveness Index (

Table A2. The structure of Digital Inclusiveness Index.

Category	Indicator (Exogenic Variable)
Risk-related technologies	Security and privacy assurance technologies
Expansion-related technologies	External and internal networking—provision
Social value creating technologies	Data collection and data access technologies
	Sharing/creating knowledge technologies
	Decision-making technologies
Pervasiveness of ICT	Pervasiveness of digital technologies
Appropriateness of ICT	Appropriateness of ICT regarding target group

in Appendix A). We assumed that categories are equally significant based on our theoretical insights and all variables used in these categories have equal weight. DII was calculated by applying the following formula for categories (where RRI—Risk related technologies; ERT—Expansion-related technologies; SVT—Social value creating technologies; PT—Pervasiveness of ICT; AT—Appropriateness of ICT):

$$DII=\frac{RRI+ERT+SVT+PT+AT}{5}$$

The estimates of weighted coefficients of the category are estimated by expert assessment. As no numeric data have been collected until the present experiment, there are no possibilities to carry out statistical research and identify statistical significance of each indicator necessary to construct the indexes. Therefore, leverage coefficients of the indicators (or categories) were determined in view of the acquired empiric experience in defining indicator correlation significance.

Social Responsiveness Index (SRI). In total, 11 exogenic variables, divided into five categories, were used to determine the Social Responsiveness Index (

Table A3. The structure of POS QI.

Category	Indicator (Exogenic Variable)
Dynamism, openness and flexibility	Degree of interaction and engagement
	Degree of adequate supply of critical mass (“swarm effect”)
	Degree of diversity in the spatial interaction
Transparency	Degree of development of transparent structure and culture
	Degree of independence
Decentralization and self-organization	Degree of decentralization and self-organization
Social impact and engagement	Degree of social impact
	Degree of social motivation
	Degree of social orientation
Generated public value	Efficiency of problem-solving
	New qualities in form of ideas, structured opinions, competencies, etc.

). We assumed that categories were equally significant based on our theoretical insights and all variables used in these categories have the equal weight. Thus, the SRI value is determined applying the formula as follows (where DOF—Dynamism, openness and flexibility; T—Transparency, applicable for the communities only; DS—Decentralization and self-organization, applicable for communities only; SI—Social impact and engagement; GPV—Generated public value):

$$SRI=\frac{DOF+T+DS+SI+GPV}{5}$$

The estimates of weighted coefficients of category are estimated by expert assessment. As no numeric data on the observed phenomena have been collected until the present experiment, it was not possible to carry out statistical research and identify statistical significance of each indicator necessary to construct the indexes. Therefore, leverage coefficients of the indicators (or categories) were determined in view of the acquired empirical experience in defining indicator correlation significance.

Digital Co-Creation Index (DCCI). The Digital Co-Creation Index is designed around three different indices: POS Quality Index, Digital Inclusiveness Index and Social Responsiveness Index. The Digital Co-Creation Index is numerical value that expresses the mean of these three Indices. The Digital Co-Creation Index formula is the following:

$$DCCI=\frac{POS QI+DII+SRI}{3}$$

At the current stage of the research, we assume that three indices are equally significant.

4. Assessment Results of ICT Supported Co-Creation for Open Public Spaces

Digital Co-creation Index calculation methodology was applied for experimental evaluation of four Living Labs (Lisbon, Ghent, Vilnius, Milano) during the implementation of the C3 Places project, mentioned in the introduction (January–May 2019). The proposed methodology provides evaluators with assessments’ criteria, definitions of dimensions, evaluation indicators, guiding questions and evaluation guidelines [24]. The empirical data for evaluation were collected using mixed-method approach (semi-structured interviews with stakeholders, a questionnaire survey, digital monitoring and analysis of available secondary data, mobile application and experience of the visitors when visiting the park, a set of thematic workshops, interviews, mapping and evaluation of public spaces in the area, field observation, etc.). Descriptive statistics of the Digital Co-Creation Index is presented in

Table 1. Statistical analysis of four Living Labs according to public open spaces (POS) Quality Index, Digital Inclusiveness Index, Social Responsiveness Index and Co-Creation Index in total.

Dimensions		POS Quality Index	Digital Inclusiveness Index	Social Responsiveness Index	Digital Co-Creation Index
N	Valid	4	4	4	4
	Missing	0	0	0	0
Mean		68.7500	15.8333	43.2292	42.6042
Median		68.2292	5.0000	54.7917	42.6042
Mode		51.04 a	0.00	0.00 a	29.17 a
Standard Deviation		15.88897	25.44056	29.12373	12.31140
Minimum		51.04	0.00	0.00	29.17
Maximum		87.50	53.33	63.33	56.04

^a Multiple modes exist. The smallest value is shown.

and Figure 1:

Analysis of the data indicates that the lowest value of POS Quality Index is 51.04, whereas the highest is 87.50. The highest value of Digital Inclusiveness Index is 53.33, the lowest is 0.00. The highest value of Social Responsiveness Index is 63.33, the lowest is 0.00 (

Table A4. Indexes of 4 Living Labs analyzed during the experiment.

Living Lab	POS Quality Index	Digital Inclusiveness Index	Social Responsiveness Index	Digital Co-Creation Index $\mathit{D}\mathit{C}\mathit{C}\mathit{I}=\frac{\mathit{P}\mathit{O}\mathit{S}\mathit{Q}\mathit{I}+\mathit{D}\mathit{I}\mathit{I}+\mathit{S}\mathit{R}\mathit{I}}{3}$
Vilnius	75	10	63.33	49.44
Lisbon	51.04	N/A	56.25	35.76
Ghent	61.45	53.33	53.33	56.04
Milano	87.5	0	0	29.16
TOTAL	68.75	21.11	43.22	42.60

). Comparison of the indexes reveals that the lowest mean has been found in Digital Co-Creation Index—42.6042, with the lowest standard deviation—12.3114. This shows that the distribution of data is the smallest, i.e., the values of the Digital Co-Creation Index within four Living Labs is less dispersed. The biggest dispersion has been observed in the Social Responsiveness Index and Digital Inclusiveness Index, i.e., 29.12373 and 25.44056, respectively. Comparison of the indexes also reveals that the highest mean has been found in POS Quality Index—68.75 with the standard deviation—15.88897.

In

Table 2. Correlation analysis of POS Quality Index, Digital Inclusiveness Index, Social Responsiveness Index and Digital Co-Creation Index of 4 Living Labs.

Living Labs		POS Quality Index	Digital Inclusiveness Index	Social Responsiveness Index	Digital Co-Creation Index
POS Quality Index	Pearson Correlation		−0.268	−0.713	−0.317
	Significance (two-tailed)		0.663	0.177	0.603
	N		5	5	5
Digital Inclusiveness Index	Pearson Correlation	−0.268		0.331	0.831
	Significance (two-tailed)	0.663		0.586	0.081
	N	5		5	5
Social Responsiveness Index	Pearson Correlation	−0.713	0.331		0.711
	Significance (two-tailed)	0.177	0.586		0.178
	N	5	5		5
Digital Co-Creation Index	Pearson Correlation	−0.317	0.831	0.711
	Significance (two-tailed)	0.603	0.081	0.178
	N	5	5	5

Pearson correlation coefficient r for the POS Quality Index, Digital Inclusiveness Index and Social Responsiveness Index was calculated. The relationship is stronger if (r) value is closer to 1. If r > 0, this indicates positive relationship between variables, when one random value is increasing, other values are growing as well. If r < 0, this indicates negative relationship, when one random value is increasing, other random values are decreasing. A significant theoretical correlation between Digital Inclusiveness Index and Digital Co-Creation Index has been determined (r = 0.831). On the other hand, a moderate statistically significant linear relationship has been found between Social Responsiveness Index and Digital Co-Creation Index, as r = 0.771. However, there is no correlation between Social Responsiveness Index Digital Inclusiveness Index (r = 0.311) (as (Sig. 2 tailed) p = 0.586, p > 0.01).

5. Discussion and Limitations

The discussion of the results is limited to the comparison of the European Living Labs composing the research sample. In the absence of an index that was equally tested in other territorial contexts, the comparative value of the outcomes of this research can be established only between four Living Labs. The assessment results offer insights about the state of different socio-technological indicators and dynamic of co-creation processes inside the community. However, the comparison of the evaluation results of the case studies is not the subject of this research paper. The numerical values are provided to illustrate the functionality of the assessment tool. An increase of empirical data will condition an increase of research data reliability and the validity of a newly constructed instrument. On the other hand, almost all the data are categorical; therefore, the natural limitations of quantitative analysis of qualitative data arise. Other limitations of this research are related to the lack of historical data; it is impossible to determine statistically the relevance of used parameters and their statistical significance. In addition, statistical inference and reliability of indicators measuring digital co-creation has not been analyzed enough due to objective reasons. It is expected that more scientific results will appear in this direction during the implementation of the C3Places project and other researchers’ activities. The findings and their implications should be discussed and tested in the broadest context possible. Future research directions could be the identification of compatibility and significance of individual components, since obviously, not all indicators are equally important.

6. Conclusions

In this paper, we presented the design of a Digital Co-Creation Index and methodological guidelines for applying Digital Co-creation monitoring technique for the evaluation of co-creation processes by designing public open spaces. The application procedure is illustrated with the evaluation results of four European Living Labs—Aukštamiestis Living Lab (Vilnius), Alvalade Living Lab (Lisbon), Città Studi Living Lab (Milano) and Zuid park Living Lab (Ghent). The methodology is based on POS functionality and identifies the basic characteristics of attractive, inclusive and responsive urban initiatives. Measuring them could be useful in predicting the performance of the co-creation ecosystem as a whole. While this study has certain limitations, they offer opportunities for future research. By testing the proposed framework on real-life case studies and employing a more quantitative approach to evaluation of co-creative initiatives, the methodology can be transformed into useful self-assessment tool. The Digital Co-creation Index is expected to facilitate urban architects, IT developers, social scientists, business designers and user communities to identify the main problem areas, to decide on scientific supported organizational, communicational and technological solutions and to maximize the benefit that the community and other stakeholders could receive from the collaboration. Tools and processes should support community activities that contribute to its identity. Nevertheless, taking into account the interdisciplinary nature of co-creation processes and complexity of public open spaces, future work could include combining the proposed methodology with the relevant findings of different research fields. The experimental evaluation results pointed the demand to find new methods for understanding dynamics and complexity on the co-creation phenomenon, the need to revise, upgrade and supplement the Digital Co-creation Index calculation methodology and to collect supporting empirical evidence for model validation.