Evidence-Based Public Policy Decision-Making in Smart Cities: Does Extant Theory Support Achievement of City Sustainability Objectives?

Abstract

Evidence-based decision making is promoted as offering efficiency and effectiveness; however, its uptake has faced barriers such as underdeveloped supporting culture, limited access to evidence, and evidence that is not fully relevant. Smart city conceptualizations offer economic and environmental sustainability and better quality of life through evidence-based policy decision-making. We wondered whether smart city theory and practice has advanced the knowledge of evidence-based decision-making. We searched major databases for literature containing a mention of smart cities, decision-making, and policy. We identified relevant literature from a range of disciplines and supplemented these by following backwards and forwards citations. Evidence-based decision-making was found mostly in literature regarding the theory and practice of smart city operations, and, to lesser extents, the articles regarding policy decisions and tactical decisions. Better decision-making which supported the achievement of city sustainability objectives was reported in some articles; however, we found significant obstacles to the further achievement of city objectives in the areas of underachievement in collaborative decision-making, privileging of big data evidence, and artificial intelligence agents as decision-makers. We assembled a definition of smart city decision-making and developed an agenda of research which will support city governments, theorists, and practitioners in better achieving sustainability through improved decision-making.

1. Introduction

The literature on smart cities prescribes a wide-range of actions, including decision-making process changes [1], typically to achieve objectives such as economic performance, citizen-centric governance, sustainability, efficient government, and quality of life [1,2,3]. Whether the smart city is driven by innovation and creativity or is characterized by pervasive and ubiquitous ICT, data capture and analysis is required to underpin evidence-informed policy development [4]. Smart city prescriptions as to decision-making are dispersed across the literature from many fields, for example, business, urban governance, economics, public management, transportation, computing, and information. The rationale behind a government adopting evidence-based policy as an organizational strategy is that policy will be based on the most accurate and relevant evidence [5]. This approach to decision-making in the smart city context seeks to put into effect conceptualization of smart city governance and policy decision-making as a holistic approach aiming for balanced public value and which rejects the prevalent focus of the literature on smart cities upon ‘smartness,’ as demonstrated by the extent to which ICTs are applied [6]. What was not clear was whether the literature on smart cities explains how decisions which more effectively achieve sustainable city objectives can be made.

Such fresh insights might address the criticisms and limited take-up of evidence-based decision making across public administration [7,8,9]. This paper reports painstaking research which assembled the scattered fragments of smart city decision-making literature, and then proposes and justifies an agenda of research focused on increasing knowledge as to how smart city governments can promote more effective decision-making through an evidence-based approach.

Our research was guided by the following research question:

What does the literature report about the role of evidence-based decision-making in smart city government policy-making?

Whilst our foundational search of major databases was focused on literature containing a mention of smart cities, decision-making, and policy, we found that following backwards and forwards citations led to our assembling an exceptionally wide body of evidence as to decision-making for different purposes, not only in the smart city. Many of the conceptualizations and applications of decision-making and evidence-based decision-making are about immediate and other operational decisions, mostly with regards to mobility systems. We found a lesser amount of literature about decision-making on policy or strategic topics and a small amount regarding tactical or managerial matters. We contribute to the knowledge on smart cities by separating out the knowledge according to the purposes of decisions and by separating out the types of evidence and the actors who the literature specifies as the decision-maker. We set a precise decision as to what constitutes smart city decision-making, unpack three substantial barriers to the achievement of sustainable city objectives through better decision-making, and propose an agenda of research to address these barriers.

Our paper, next, explains our approach to the research. Then, we unpack the literature regarding evidence-based decision-making in public administration as a foundation for our interrogation of the data. We then set out the themes within the literature on smart cities that emerged. We then discuss the importance of our findings and finish with our overall conclusions and an agenda for research.

2. Literature Review

Our exploratory search, to inform the determination of our research plan, identified articles spread across publications from diverse disciplines, for example, information technologies, urban planning, and public management which, at the least, mentioned smart city evidence-based decision-making. Accordingly, we determined that a deep and wide search for all relevant articles would be the foundational method for our research. That search and subsequent review of identified articles was followed by an analysis of the included articles for the purpose of drawing out the themes from the substantial body of data which was progressively discovered.

2.1. Methodology Description

We applied the PRISMA data base search process [10], commencing with searches of the Web of Science (Core Collection), ProQuest Central, and EBSCOhost (Academic Search Premier) data bases. The search terms ‘smart cit*’ AND ‘decision-making’ AND ‘polic*’ were applied. The search was not narrowed further to search only for ‘evidence-based,’ as we believed that understanding what the literature on smart cities says about decision-making, evidence-based or otherwise, is necessary. We limited the search to peer-reviewed items published in English during 1999 to 2020, as the literature review of Meijer & Bolívar (2016) reported that the first item regarding smart city governance, which would encompass decision-making by city government, was published in 1999 [3].

The database search, full-text in the instances of Web of Science and EBSCOhost, and abstract in the case of ProQuest Central, identified a total of 210 items, 193 with the removal of duplicates. Subsequently, two reviewers with intimate knowledge of public policy-making independently screened each article. The exclusion criteria were as follows:

• Did the article advance the body of knowledge with respect to evidence-based policy decisions applied to cities? If not, go to question 2.
• Did the article provide relevant background material with respect to evidence-based policy decisions applied to cities? If not, then exclude the article.

Interestingly, most usages of ‘decision-making’ were incidental and of no assistance to answering the research question. There were a minority of items for which the independent reviewers were in initial conflict; however, this was resolved through discussion. Ultimately, 37 articles sourced from academic databases offered potential for contribution to the research. The entire review process is depicted diagrammatically in Figure 1.

2.2. Analysis

The analysis of the data from these items proceeded iteratively, influenced by the rich body of evidence that was unearthed though following forwards and backwards citations. A further body of decision-making literature, numbering 47 articles, mostly in the context of big data in cities, emerged. This focus on big data-informed decision-making by city governments dominated the smart city decision-making literature.

The interrogation of the articles encountered two pervading grey areas. Firstly, the role of the actor in the decision-making. Secondly, the level of the decision within the activities of a city government, for example whether the decision is a strategic matter, or whether it is purely operational, merely implementing an established policy, for example buses being privileged over private cars.

3. Evidence-Based Decision-Making in Public Administration

Our intent was to both be clear as to the meaning of evidence-based decision-making and to understand the key aspects of evidence-based decision-making in public administration. We first assemble the knowledge as to evidence-based decision-making across all sectors. Secondly, we set out perceptions as to what constitutes evidence. Thirdly, we set out the knowledge as to evidence-based policy decision-making in the public sector, including city government.

3.1. Evidence-Based Decision-Making

Evidence-based decision-making, in an all-sectors management context, is described by Baba and HakemZadeh (2012) as ‘…a dynamic process through which evidence is obtained, interpreted, and used as a basis of decision making.’ The evidence has to be generated through a rigorous process, and must be relevant to the context where it is invoked [11].

Evidence-based decision-making in management is at an early stage of development, with the available examples being derived through formal research, in particular from medicine [12]. Rousseau (2006) explains evidence-based management as an approach that seeks to enhance the quality of decisions and practice by adopting principles from external, systematic research. Rousseau and McCarthy (2007) explain practice as the detailed, operational workings of organizations [13].

3.2. Defining Evidence

The literature [5,7] is focused upon the sourcing of evidence for public policy decision-making from academic research with a strong emphasis upon applying scientific methods and procedures to produce evidence [14]. Blockages to greater use of evidence in policy-making are disagreement among stakeholders as to what constitutes evidence, and the interpretation to be placed on the information [15].

Baba and HakemZadeh (2012) found that the literature did not provide guidance as to what constitutes evidence, observing that, to meet the needs of decision-makers, evidence of different types must be obtained from a range of sources with regard to the circumstances of the intended decision [11]. Baba and HakemZadeh (2012) provide the following examples: financial information, surveys, public opinion, practical experience, and internal research [11].

In the absence of a definition of what constitutes evidence for evidence-based policy decision-making, our research was alert for all information and data that were portrayed as evidence used in any type of decision-making.

3.3. Role of Evidence in Policy Decision-Making

The design and implementation of public administration policy is explained by Osborne (2021) as a bi-directional process involving service users, and requiring evidence-based, incremental development. For our research, we took public policy to be the enactment of a societies’ values, beliefs, and norms by a government [16]. The rationale behind the adoption of evidence-based policy as a strategy is that policy would be based on the most accurate evidence relevant to achieving policy objectives [5]. Evidence-based decision-making is seen by some authors as providing benefits, and by others as having disadvantages and limitations.

Benefit is achieved because the rationality inherent in evidence-based decisions avoids, or at least reduces, the policy failures which are rooted in the ideological dimension of the policy process [14]. The evidence-based approach is held to ensure that policies respond to the real needs of citizens, that issues and problems requiring immediate attention are addressed, and that information sharing amongst different stakeholders is enabled [17]. In the context of public administration, the benefits of making policy decisions using evidence are held to be wide, direct, and indirect. Burkšienė and Dvorak (2020) describe the benefits as being:

Help to make better decisions. Evidence-based policy is needed by policy planners, politicians, and stakeholders who care how resources are allocated to competing policy goals. They formulate questions for researchers such as: how best to use budget resources; what resource use alternatives will benefit people more; what the relationship is between costs and benefits.

Improve and enhance social determinants. Scientific evidence is positioned as the reason for changes in policy-making, management, or implementation strategies. Evidence can be gathered during preparation for, or execution of, a decision. Ongoing provision of evidence assists those responsible for the quality of implemented policy programs to be aware of the changing needs of stakeholders and to legitimize policies.

Learning. Evidence-based policies play an educational or training role by providing information on how the policy program works, what the program should have been, and what the impact has been. This approach translates to a behavioral transformation of government service delivery organizations [18].

Critics have identified limitations of evidence-based decision-making which they characterize as relating to the realities of governance in the government context. The socio-political nature of government policy decision-making invariably requires political decisions involving trade-offs between competing social values [7], with only a small proportion of the decisions being suited to technical evidence [9]. Concilio and Pucci (2021) observe that the literature generally presents that socio-political factors should be given the advantage by decision-makers [14]. Further, Neylan (2008) identifies the privileging of evidence in the form of statistics as resulting in evidence that could be challenged as to relevance and accuracy [8]. This dominance by statistical data as evidence is attributed by Neylan (2008) to the bureaucratic environment favoring impersonal decision-making devices, being technology-laden, and craving standardization at the expense of relevance and accuracy [8].

Subsequently, Head (2010) noted an increasing use of qualitative evidence such as data concerning values, attitudes, and perceptions of stakeholders, typically together with statistics [19].

Head (2010) specified the following factors as necessary to establish robust and effective evidence-based public policy decision-making systems:

• High-quality information bases on topic areas;
• Capacity in the form of cohorts of professionals contributing skills in data analysis, monitoring, and policy evaluation;
• Political and organizational incentives for utilizing evidence-based analysis and advice;
• Substantial mutual understanding across the demand and supply roles, namely decision-makers, policy professionals, and researchers [19].

Newman, Cherney, and Head (2017) acknowledge that evidence-based policy may be an unattainable ideal in public administration [5]. Notwithstanding, we wonder whether the quite positive conceptualizations from the literature on smart cities address Head’s (2010) factors and concerns as to evidence-based decision-making [8,19].

4. Smart City Decision-Making

Smart cities are smart, not only in the way in which their governments harness technology [17,20] but in the way that they monitor, analyze, plan [2], and govern the city [21,22]. Evidence-based decision-making is identified by Gil-Garcia et al. (2016) as a key dimension of smartness, being enabled and supported by real-time data collection and sharing, analytics, and city business processes [23]. Narrowing in on smart governance, the decision-making process is prominent as one of the six defining elements [1] and is key to Scholl and Al Alawadi’s (2016) definition of smart governance as ‘the capacity of employing intelligent and adaptive acts and activities of looking after and making decisions about something’ [24].

The decision-making process is explained as: the city government facilitating decision-making between city government, external organizations, and citizens which is highly collaborative, transparent, and efficiency-focused [25]; supported by citizen-centric ICT-mediated governance systems [26]; using network technologies [27] and different decision-making methods [28]; collecting ‘all sorts of data and information’ by sensor networks [29].

The importance of decision-making capacity compared to the ICT focus was explored by Bolivar and Meijer (2016), who surveyed European smart city practitioners to validate a model of smart city governance which positions changes to the city government organization as the first order outcome [1]. One of the five categories of changes is decision-making processes and all five were rated as more important than use of technology, per se, as an outcome [1,2,3].

These articles advocate changes at the system level but do not explicate an important detail, namely the differing purposes of the decisions, e.g., operational or strategic/policy, or whether the types of evidence might extend beyond the transactional and statistical aspects which invariably accompany the prescription of ICT, or who precisely is the decision-maker. In short, we found that change to decision-making and policy-making processes in smart cities is advocated, but by many authors, from many disciplines, and in a way that was characterized by Meijer and Bolivar (2016) as fragmented [3].

Our research then sought to answer: who, what, and why. When was clear, namely, when decisions are being made. In the following sub sections, we report the literature as to why (the purpose of the decisions), what (the types of evidence), and who (the decision-makers).

4.1. Why—Purpose of Decisions

The literature on smart cities [30,31] presents conceptualizations of decision making that do not specify the purpose of the decision-making activity. Not all decisions are about policy making, many are about policy implementation, be that immediate, real-time actions, or management decisions over the term envisaged in a policy decision. In this sub section, we report the evidence from the literature as to the differing purposes of the decisions which the authors described. To distinguish between these purposes, we apply the structure of categories of purpose of municipal decision-making, namely operational, tactical, and policy, established by Ribeiro et al. (2019) in their model of the ideal strategic digital city [32].

4.2. Operational

The literature on smart cities presents a strong foundation of literature regarding decisions for an operational purpose. Harrison et al. (2010) conceptualise a smart city as using sensors and personal devices, a networked communication and information platform, and complex analytics to make better operational decisions [30]. The city is smart if the city has the right real-time information at the right place and on the right device to make real-time city-related decisions with ease [33]. An example is the city government ensuring the security of the water supply through real-time monitoring [34].

The dominant application of smart city ICT prescriptions to operational decision-making is for traffic and mobility where the objective is to increase efficiency whilst immediately remedying blockages and advancing environmental sustainability [33,35]. Welch and Widita (2019) examined 81 public transportation articles, finding that, whilst some were about informing both tactical and policy decisions, most were directed at operational decision-making, analysing schedule-optimization, delays, level of service, crowding and weather, and providing real-time information to assist passengers [35].

Typically, real-time decision-making systems for traffic, waste collection, or public transit commence with the gathering of data by static and vehicular sensors, cameras, georeferenced tools, and in-vehicle (windscreen) and smart devices, which transmit the data by way of the IoT to an aggregation platform, and then to a range of applications for data interpretation and decision-making by citizens or the service provider [33]. City government decision-makers, making real-time decisions for traffic operations, are typically located in municipal operations centres. Examples are the municipal operations centres of Rio De Janeiro, Porto Alegre, and Belo Horizonte, which Pereira et al. (2017) found to be massively leveraging ICT, formal protocols, and data mining capabilities to enable information sharing between city agencies such as traffic, separate municipalities, police, ambulance, military, and civil defense [36]. These centers bring appropriate decision-makers together to form real-time collaborative operational decisions based on shared best available data. Pereira et al. (2017) found that such data-based decision-making at this operational level led to better outcomes from decisions [36].

The capacity of smart city real-time, operational decision-making systems to deliver immediacy is invariably characterized as a positive force for the city and its citizens. The exception is Kitchin (2019), who raises concerns as to the compression and merging of timescales that were traditionally separate, namely past, present, and future [37]. An example of compression of timescales provided by Kitchin (2019) is Dublin, where workflow apps allow managerial and operational staff to make real-time decisions out in the field and implement these, radically compressing the timescale in which the activity and decisions are made [37]. An example of the merging of timescales is the flow of big data into urban control centers and intelligent systems (and back out to citizens) which Kitchin (2019) argues leads to the ‘past present’ becoming a ‘perpetual present’ [37]. For Kitchin (2019), there is a substantial risk of automated predictive systems such as those used in traffic control center vehicle flow predictions deciding the ‘future present’ [37]. Kitchin (2019) applies a timescale perspective to the pressing issue of the negative impacts of decision-making by algorithm-based agents, but in the specific context of operational decisions [37]. In the Decision-makers sub-section, we report the concerns of a number of authors as to other problems associated with AI agents making decisions.

4.3. Tactical

Tactical decision-making, which applies the data collected for real-time operational decision-making, is carried out by the agencies that are partners in the Brazilian operations centers [36]. For example, informants explained ‘future-focused’ and ‘preventative measures’ where recent data is applied to address traffic congestion by modelling the timing of varying mobile public transport lines and data regarding robberies to inform decision-making as to lighting. Use of data captured in real-time for tactical decision-making presented as a persistent theme in both the theoretical and empirical literature. Engin et al. (2020) assert that using real-time evidence provides opportunities for better decisions in managing public assets and delivering services. They describe a short time, typically a day, week, or, at most, a month, between data capture and decision [38].

The extent of the coverage of the making of tactical decisions is not as large as that regarding real-time operational decisions, yet it appears to be increasing. Again, public transport and traffic-related research dominates. Public transportation analyses utilizing big data were found by Welch and Widita (2019) to largely focus on topics related to understanding and improving travel itself, with the travel behaviour category dominating [35]. The next most frequent category, management, was dominated by papers regarding infrastructure maintenance, e.g., condition-based maintenance. Examples of the decisions are: public transport route planning, fleet performance, traffic scheduling, rerouting, and stop changes. In summary, Welch and Widita (2019) identified a distinct body of articles about using big data to understand and make decisions about the existing transportation system [35]. Similarly, Basilio, Periera, and Brum (2019) developed a multi-criteria methodology to analyse big data such as to crime in a territory to provide evidence to assist managers in decision-making regarding the objective of reducing criminal activity [39]. Basilio, Periera, and Brum’s (2019) methodology was tested in the context of Rio de Janeiro, where it created knowledge suitable to provide both intelligence for use in tactical activities and for longer-term, planning decisions [39].

4.4. Policy

Smart cities are conceptualized as planning for the future [31], for ‘the long run’ [31], and as using big data for policy decision-making [37].

Literature reporting empirical evidence as to the practice of smart city policy decision-making was limited. Policy as to the introduction of bus services between midnight and 5.00 a.m. in Seoul was decided by the Seoul Metropolitan Government in 2013 with regard to evidence in the form of historical data as to five million late-night taxi journeys and three billion mobile phone accesses between 12 midnight and 5.00 a.m. [40]. The evidence was also used to design the route that a late-night bus might best take [40]. The investigation of Hong, Kim, Kim, and Park (2019) concluded that new late-night service resulted in the average number of late-night passengers during 2015–2016 being 23 to 33% greater than daytime bus services [40]. Whilst Hong, Kim, Kim, and Park (2019) concluded that big data evidence-based policymaking made decision-making more citizen-focused by more accurately deciphering citizen preferences, they also concluded that their own, largely big data-based methods did not allow them to understand whether policy makers had considered matters of equity such as the extent of provision of late-night services to low-income neighborhoods [40].

A further demonstration of smart city policy decision is provided by the consideration of the practicalities of changing the Beijing taxi fleet to electric vehicles. Evidence in the form of data from disparate sources, e.g., GIS, mobile phone, and transit system transactions was brought together to support decisions as to the positioning of possible charging station locations for hybrid electric vehicles [41], and subsequently fully electric vehicles [42].

4.5. What—Smart City Evidence

Literature on smart cities emphasizes the importance of data-as-evidence to the making of robust and reliable decisions. Indeed, the UN Statistical Commission (2017) has recognized that quality and timely data is vital to governments in making informed decisions [43]. Much of the literature on smart cities comes from the perspective of evidence for decision-making coming from electronic sources. The highly influential article of Harrison et al. (2010) foresaw data as necessary for city management making better decisions, and describes data as typically extracted and communicated in near-real-time from sensors within various city services by way of an interconnected enterprise computing platform [30]. This prediction has come to pass, and more, metamorphizing into the IoT-linked system of systems that presents evidence for decision-making by governments, service providers, and citizen service users within and beyond the boundaries of the city [33]. This evidence is sometimes explained as high-level conceptualizations, and other times with great specificity, but often in the context of big data. We were mindful of the concerns of authors from the wider literature on decision-making [8,19] that statistical data is strongly privileged over other, non-statistical forms of data. We now set out our findings as to types of evidence, sharing of evidence, and the quality of decisions.

4.6. Types of Evidence

The privileging of data in the form of statistics in evidence-based public policy decision-making has been found to be so extensive that the evidence could be challenged as to relevance and accuracy [8,9]. Parkhurst (2017) observed that only a small proportion of government decision-making was suited to technical evidence. For these reasons, we were alert to distinguishing the type of evidence being advocated or reported in each article [9].

Statistical evidence, mostly derived from transactional data, was dominant in the literature on smart cities. For example, Gil-Garcia et al. (2016) position evidence-based decision making as a key dimension of smartness in city government, yet specify a conceptualization of evidence that is limited to ‘...the intensive use of data, through ubiquitous sensing, advanced metering, and integrated applications...’ [23].

Despite our being alert to the possibility of the description of decision-making using alternate terminology, we found only one smart city article that described decision-making using non-statistical data. This was the Brazilian operations centres where government organizations gather images and messages regarding traffic congestion, disasters, and incidents from citizen contributions to social media on their smartphones [36].

Statistical data examples in the literature mostly depict the smart city vision, and growing reality, of streams of data flowing between physical objects, actors, and government, private, and social institutions that inhabit and govern cities [2,31]. This mass of data has made possible the concept of ‘big data’, increasingly facilitated by the IoT [33], utilizing computational capacity larger than that on which the real-time data is created [44] and for which standard statistical software is not adequate [45]. For the purposes of this research, we take the approach that big data includes the interrogation of data using analytics, artificial intelligence, and data mining [38,46].

Data in the big-data context is characterized by its immediate availability, even if it is not used immediately, and the frequency of collection. The most prominent examples of big-data-derived evidence are found in transport studies [35,38]. The two most frequently used sources of data in the 81 big data analyses examined by Welch and Widita (2019) were smart card and automated data [35]. Welch and Widita (2019) found that such data and their analysis were not fully fit-for-purpose [35]. The techniques used in the analyses were found by Welch and Widita (2019) to be mostly those originally adopted for historically deliberately collected data [35]. Welch and Widita (2019) conclude by identifying a significant opportunity for smart city transportation scholars to capture value from big data by applying the more advanced statistical methods that are commonly associated with big data in other fields [35]. The parameters of the data utilized are seen to be inappropriately driven by their availability from transactions, as distinct from the dimensions of the precise research question [35], as is the traditional approach which leads to deliberately collected data for transport evaluation [2]. This absence of a preceding, precise research question as a characteristic of big data-based research causes Engin et al. (2020) to caution those evaluating decisions to establish a careful understanding of the provenance of the big data evidence so that the possibly of poor-quality research and bias is diminished [38].

The limitations of a big data-only strategy for analysis were commonly acknowledged by the authors of studies reviewed by Welch and Widita (2019), who offered that the analysis derived from emerging big data needed to be complemented or validated using conventional travel surveys, thus confirming the notion of the complementary forms of data for evidence-based decisions [35].

4.7. Sharing of Evidence

The high-level smart city conceptualization of smartness or smart governance being achieved through co-operation and collaboration between organizations [4,31,47,48,49] was clearly evident in the articles regarding obtaining the data necessary to form opposite evidence for city decisions, operational, tactical, and policy decisions. Much of the inter-organizational collaboration requires the sharing of data between organizations, leading to the literature on smart cities [35,38,50] presenting a distinct theme of much of the data required for decision-making by the city administration being controlled by another entity, including citizens.

Notwithstanding a strong history of all levels of the Denmark public sector publishing data for public consumption through open data portals, valuable city data are locked up within private and public organizations where the case for its relevance has not been made or heard [51]. This is consistent with the earlier experience of Malmo [52] where renewable energy projects were frustrated by issues around aggregating energy data by sector and balancing public and private interests. However, Giest (2017) found that, in the smart city cases of Copenhagen, London, Malmo, and Oxford, obstacles to data exchange to support carbon emission reduction decision-making stemmed from incompatibilities in formats, standards, access, individual organizations developing their own hardware, software and data applications, and institutional complexity [53]. For these reasons, several years of effort had not produced any sharing of data within each city’s ecosystem, and resulted only in local experiments as to data sharing. The obstacles to data exchange and compatibility across datasets stemmed less from the connection to private stakeholders and more from cross-departmental and cross-level (government) links. The close focus on forging these links and sharing and integrating data sets led to only limited attention being paid to establishing capacity to utilize the data as evidence [53]. This may be because the heavy reliance by city administrations upon outsourcing to private companies had led internal knowledge as to its use as evidence being not adequate, as was found by Giest (2017) [53].

Problems and challenges of data sharing where there is a shortfall in trust between organizations which constitute smart city IOT-based systems have been addressed by Cao et al. (2016), who proposed a model of the processes necessary for trust-based data sharing which encompasses system architecture and trust data usage control policies [50]. The core of the model is the assembly of the extant data sharing policies of all stakeholders with the intent to harmonize policies. Consequent upon application of this mostly technical model to a case study smart city topic, Cao et al. (2016) identified the need for further development of the model by providing for the involvement of multiple stakeholders in defining data sharing policies and in determining trust and usage policies as well as in determining incentives such as tax rebates to data stakeholders, including citizens [50]. The impact of other dimensions of trust, and other factors that facilitate or constrain sharing of smart city data, appears to be very underdeveloped.

4.8. Quality of Decisions

Because the raison d’être for emphasis upon decision-making being based on evidence is achieving a better outcome from the decision [11,12,19], we separated out the evidence as to decision outcomes for the city according to the purpose of the decision, namely operational, tactical, or policy.

4.8.1. Operational Decision-Making

Operational decision-making supported by evidence assembled through smart city ICT’s in respect to motor vehicle traffic congestion, emergency services, and transit across cities was found to result in increased quality of public decisions made by the operations centers of three Brazilian case study cities [36].

In contrast, Kitchin (2019) describes the phenomenon of increasing real-time, immediate operational decision-making as a fixation, arguing that there is an erasure of time for thoughtful decision-making, emphasizing that the reality is that now is not always the right time to act because governmentality and quality of life may be sacrificed [37].

4.8.2. Tactical Decision-Making

Applying Ribeiro et al.’s (2019) concept of tactical decision-making being those managerial decisions taken to implement a policy decision, the article of Hong, Kim, Kim, and Park (2019) reports their evaluation of the 2015–2016 efficiency of the late-night bus service in Seoul [32,40]. Their evidence drawn from big data was that the late-night operations attracted 23 to 33% more passengers than did daytime bus services, providing evidence to inform further managerial tactical decisions as to timing and route changes to enhance efficiency or service quality.

4.8.3. Policy Decision-Making

Better policy decision-making through evidence formed of city data was prominent in the literature on smart city transportation and urban management. Largely through the application of big data concepts, innovative approaches to relating smart city data sets to form evidence are reported as being used to make decisions as to policy which have led to better outcomes.

An example is the innovative approach to forming evidence to support decision-making as to the policy of whether and where a late night (midnight to 5.00 a.m.) bus in Seoul should run. Big data were used to decide the frequency and routes of new services which were found by Hong, Kim, Kim, and Park (2019) to have attracted average passenger traffic that was 23 to 33% greater than for daytime bus services, indicating that the evidence-based policy decision led to a high-quality outcome [40].

Contrary experiences, where the objective of the decision-making process has not been achieved or, due to limitations as to the type of data, the data offered have been judged to have not provided fully relevant evidence, are also reported. An examination of carbon emission initiatives of four case study European smart cities [53] revealed that the cities were not able to acquire evidence as to carbon emissions. No evidence to inform decisions could be obtained by the responsible city organizations because of the proliferation of expert external contractors, the silos internal to and between government bodies, and a disproportionate emphasis upon technological aspects of big data integration, to such an extent that the opportunity to integrate big data into decision-making was missed. Evidence not being fully relevant is asserted by Kitchin (2014) to be derived from city administrations adopting an algorithmic process approach to city governance because such an approach centralizes power with a select set of offices that make the decisions and perpetuates reliance on this form of evidence to the exclusion of other forms of evidence [4].

This limiting of the forms of evidence used to make policy decisions in the smart city is held to be unnecessary, requiring remedy by way of robust empirical evidence so that better policy and practice decisions can be made [4]. Missed opportunities for better outcomes of decisions caused by unnecessary limitations upon the evidence used were identified by Welch and Widita (2019) in their review of the literature on public transportation [35]. Only 8 of 81 articles regarding the use of big data had reported research which applied advanced methods/techniques such as machine learning. The remaining 90% of articles may have unnecessarily limited the possibility of insights by remaining with the historical, deliberately collected fields of data, were limited to single modes of transit, or should have been gathered over a longer duration [35].

4.9. Who—Decision-Makers

Who are in fact the decision-makers in smart-city conceptualizations of decision-making is not at all clear. For example, Albino et al. (2015: 12) explain that “smart governance means various stakeholders are engaged in decision-making and public services,” then proceed to argue that ICT-mediated smart governance has a spirt which is citizen-centric and citizen-driven [26]. Yet, Albino et al. (2015) do not specify whether the spirit imbues individual citizens with decision-making powers or, indeed, which actors within the ‘various stakeholders’ omnibus conceptualization is to be the smart city decision-makers [26]. Similarly, Nam and Pardo (2011) explain the role of information, saying ‘a smarter city infuses information into its physical infrastructure to improve conveniences, facilitate mobility, add efficiencies, conserve energy, improve the quality of air and water, identify problems and fix them quickly, recover rapidly from disasters, collect data to make better decisions, deploy resources effectively, and share data to enable collaboration across entities and domains’ [54]. But who is to be making these better decisions?

To order the evidence as to who are smart city decision-makers, in theory and in practice, we categorized the evidence according to the themes of actors involved, namely, the city government, city government with other organizations, citizens, and artificial intelligence (AI) agents.

4.10. City Government

The foundation of most smart city conceptualizations is an infrastructure which gathers, integrates, analyzes, and optimizes data to enable better operational decisions [30]. Yet, the decisions made by a city government are often not about topics within that government’s sole jurisdictional and geographic boundaries. Modern cities are often agglomerations of local governments which reflect their past as towns and agricultural hubs for which higher levels of government have created city-wide or regional service delivery organizations for topics such as transport, health, and utilities. In this sub-section, we lay out the evidence where the city government alone is the decision-maker. In the following sub-section, the evidence regarding the city government and other organization as shared decision-makers is set out.

Put simply, decision-making internal to a city government is informed by evidence drawn from the systems explained in Harrison et al.‘s (2010) influential model, yet the literature on smart cities presents a persistent theme of the departments within the city administration frustrating both the decision-making of the city and the implementation of decisions [30]. A silo approach to decision-making and management by departments has been identified by Alhusban (2015), Pereira et al. (2017), and van Winden et al. (2016) in the case of Amsterdam, Alawadhi et al. (2012) in the case of four north American smart cities [36,55,56,57]. Pierce and Anderson (2017), in the case of 10 European smart cities, provide empirical evidence as to the dysfunction within and between city government departments [58]. All of these authors believe that, for smart city objectives to be achieved, this dysfunctional, siloed approach must be resolved through collaboration between departments.

4.11. City Government and Other Organizations

Collaboration between city governments and other organizations is a very strong theme in the smart city theory and empirical articles. Shared decision-making is an essential element of authentic collaboration [59]. Whilst the full range of collaborations within the smart city can be separated out into a framework of five categories [60] for the purpose of examining decision-making; the evidence as to joint decision-making will be considered to be one category, namely the city government and other organizations, be that a private corporation, another local government, or another level of government or not-for-profit.

The simple conceptualization of service-based systems supporting city management in their near real-time operational decision-making laid out by Harrison et al. (2010) has evolved into an all-encompassing sociotechnical system which addresses the whole need in an area, such as housing, transportation, energy, waste management, healthcare, telecommunications, and education [30]. Collaboration across these systems is seen to promote information sharing, situation awareness and, in turn, sustainable collaborative decision-making and joint actions [36]. At the project level, Gil-Garcia et al., (2015) explain the smartness of smart city e-government projects as involving collaborative decision-making where more than one set of stakeholders share responsibility or authority for decisions about operation, policies, or actions of government [47].

The complexity of the multi-decision-maker model is broadening exponentially. City governments have traditionally had to make decisions by way of cooperation and collaboration with a myriad of other decision-makers [4,31,48,49] because of distinct and numerous geographical and jurisdictional boundaries [36,55]. An example is the situation of municipal operations centers in Rio de Janeiro, Porto Alegre, and Belo Horizonte, where Pereira et al. (2017) found the overall governance model to be characterized by a collaborative decision-making process involving differing mixes of city departments, agencies from other levels of government, and services which, in each case, led to better outcomes for citizens, business, and governments [36]. This complexity is increasing because of heightened expectations by legislators and the public that systems previously considered holistic but separate, such as transportation and telecommunications, now be perceived as a single system, for example a mobility system [31].

This need for city governments to achieve shared decision-making in a highly complex context has proven problematic. One example is the failure of European cities to implement the technological components of the smart city paradigm in such a way that decisional capabilities across the whole city tourism system were created [61]. In another example, four European smart cities seeking to leverage big data to inform decision-making to achieve carbon emission targets were found by Giest (2017) to be fully unsuccessful because of obstacles categorized as: the number of stakeholders, their interconnectedness, and trial and error or localized experimentation [53]. The number of stakeholders expanded beyond the municipalities that comprised the city, the utility companies, other interested levels of government, and industry bodies through to international companies such as IBM, Hitachi Consulting, Siemens, and project teams. For the city governments, the problems as to collaboration were not with the private stakeholders but rather of not building cross-departmental and cross-governments collaboration and coordination. Data needed for decision-making as to energy consumption were owned by utility companies, often private companies. These multi-decision-maker complexity problems resulted in there being constraints upon data exchange, and even compatibility of data issues, and, ultimately, little actual decision-making [53].

In these case studies, the presence of the international companies and, indeed, the engagement of other private companies for collecting and analyzing data was attributed to each of the city governments having limited expertise [53]. Giest (2017) argued that the inevitable consequence of a city government deciding upon a big data strategy for forming evidence was that city governments are forced to include additional stakeholders into the decision-making process [53]. The need to engage a range of contractors from narrow specialties compounds the complex challenge of achieving collaboration between the departmental silos within city administration and with other levels of government and the community [3]. This is because the city governments had either opted for a centralized data center or had located data professionals into the departments that own the data which reinforced the existing silos of people, IT, and data [53].

A second criticism of smart city government adoption of a big data strategy is the impact of international corporations upon the ongoing decision-making process. Kandt and Batty (2021) see corporations as applying their international network influence to ensure their data-system is purchased by city government [31,62]. In doing so, the corporations, think tanks, and advisers seek to shape the systems of the client city government, including the decision points and how and who makes decisions. Kandt and Batty (2021) concede that these ideologies are resisted by city governments which typically respond to the reality of their political mandates but remain concerned with the present conceptualization of big data, based decision-making [31].

4.12. AI Agents

We report AI agents as a category of decision-makers because authors [63,64] have characterized and named them as decision-makers and because scholars from social science and other disciplines have raised strong concerns as to the ethics of the decision-making effected by AI agents and their machine learning capabilities [63,65,66]. Kitchin (2019) asserts that smart cities are ceding control to algorithmic systems, creating a form of technocratic governance [37].

Gabriel (2020), writing on the wider philosophy of cognitive science, defines artificial intelligence (AI) as “… the design of artificial agents …” within “… computer systems and models…” “…that perceive their environment and make decisions to maximize the chances of achieving a goal” [63]. Gabriel (2020) is adamant that it is the software, the AI ‘agent’, that is making the decision [63]. Iphofen and Kritikos (2021) explore the personhood of algorithmic machines and whether ‘they’ have the potential to become moral beings and make ethical decisions [66].

This algorithm-based decision-making in the smart city context is examined in the literature on urban management [38], urban planning [14], and transportation and mobility [35,67]. However, this literature on smart cities offers only a limited exploration of the negative impacts of AI-based decision-making leading us to direct our research to the wider literature. Iphofen and Kritikos (2021) characterize algorithmic agents as opaque black boxes which have had adverse impacts in decisions about credit, healthcare, human welfare, and employment [66]. In addition, Iphofen and Kritikos (2021) anticipate AI agents becoming autonomous moral agents, causing harmful social and political consequences [66]. They call for the urgent review of regulation to address legal and ethical issues, asserting that scientists who invent AI and robots ‘... do not necessarily confront the moral issues and the political consequences...’ [66] of the autonomous moral agents they create. To provide for these moral issues, Gabriel (2020) argues for alignment of the technical aspects of the AI agent with the values surrounding the decision being made, advocating that scholars develop fair processes for determining which values to encode [63]. Gabriel (2020) suggests theory conceptualized at a high level and does not proceed to the exploration of the theory and practical application to smart cities, let alone the context of an individual smart city [63]. For Gabriel (2020), who has the right to make the decisions as to what norms, values, and principles are encoded in AI is an unresolved but simmering societal problem [63].

In the smart city context, Engin et al. (2020) observe that conversational AI systems, for example chatbots and ‘intelligent’ assistants, are increasingly utilized in urban management as key interaction tools, and in doing so are making decisions [38]. Through the agent’s machine learning capacity, each interaction with a human informs subsequent decision-making by the agent and can be formed into evidence for operational, tactical, and policy decision-making. These interactions of the chatbox form of AI agent have been categorized into three levels, namely service information provision, targeted assistance, and service negotiation, with the latter two categories of functions typically requiring the chatbox to decide between options to be put to the system user with regard to their individual circumstances, present and emerging [64,68,69]. Makasi et al. (2020) explored the concept of the values specified within the technology being aligned with those values required of public managers charged with the design and delivery of public services [64]. Makasi et al. (2020) developed, defined, and justified a comprehensive taxonomy of public values [64] expected of a human agent for application by public service chatbot designers.

4.13. Citizens

A citizen-centric approach to smart city governance is prominent in a significant number of models [6,70,71,72,73]. The intent is that the citizen is to be the focus of collaboration between city government and external organizations and citizens, not only to achieve increased efficiency, effectiveness, and transparency, but also to facilitate nongovernment entities’ participation in decision-making [25,73]. This approach to the smart city presents the city as an ongoing process, not a static objective, through which cities become more livable and resilient by way of citizens and other relevant stakeholders being involved, notwithstanding such involvement being particularly hard to achieve due to bureaucratic reasons in local governments [74]. However, whether citizens are being proposed as decision-makers in city governance, or merely provided with two-way communication, transparency of information, and decision-making processes [25] is not clear. We discovered two themes: the first regarding the role of the citizen with respect to the governance of smart city services and innovation projects, and the second regarding the role in making decisions in real-time regarding the use of city services, and providing information that informs the decisions of others.

Smart city conceptualizations position ICTs as enhancing citizen engagement through political discourse and decision-making, with including decision-making by citizens in the creation of services as a desirable element of governance [7]. Alawadhi et al. (2012) observed that smart city initiatives typically have inter-sectoral relationships across which citizens are encouraged to participate in decision-making, monitoring of services, and providing feedback [56]. Delving further into the more recent literature, we found descriptions of smart cities as having participative governance where the citizen is involved in city management [75], specifically taking decisions in local government matters [76] through the use of ICTs such as government data transparency websites, shared data platforms [76], or social media [77,78]. Zavattaro and Brainard (2019) explain smart city social media micro-encounters between citizens and their governments as providing city governments with a chance to interact with someone to share information, to crowdsource ideas, to find policy ideas, and to be part of evidence-based decision-making [78]. Zavattaro and Brainard (2019) do not proceed further to explain whether they are talking about decision-making with regards to governance of the city or with respect to the consumption of city-provided services [78].

However, these conceptualizations of the citizen as being involved in decision-making were not supported by the evidence in practice. Questionnaire survey data from 16 individual city manager respondents and city website content gathered from 47 European smart cities revealed that the surveyed cities were not promoting active participation in decision-making by citizens, and the respondents were not aware of the potential of online participation for citizens [76]. Importantly, the surveyed cities were not willing to allow citizens to be involved in city governance [76]. Similarly, the Amsterdam Smart City (ASC) arrangements were found by Sancino and Hudson (2020) to offer no opportunities for citizens to participate in decision-making [79]. Surveying the literature on social media, Falco and Kleinhans (2018) concluded that, notwithstanding the rise of social media, governments were locked in a one-way communication paradigm where citizens are more receivers rather than conscious producers or creators of information, data, ideas, solutions, and decisions in the context of public policies [77].

A latent theme within the literature was that of the citizen as decision-maker using real-time information made available by city governments on the citizens’ smart devices. Frequently occurring examples come from the literature on transportation and urban mobility. Municipal operations centers in Brazil provide the opportunity for citizens to both contribute information through social media and make decisions [36]. Typically, these systems which facilitate decision-making by the citizen in real-time comprise an IoT-based system of sensors and software which enable both the decisions of government traffic service providers, and those of the motorists [67]. Sarrab, Pulparambil, and Awadalla (2020) explain a model where smart devices are not necessary for governments to inform road users but where internet connected roadside signs positioned in anticipation of decision points in the road network provide real-time information so that the road user can make decisions, typically to minimize their travelling time [67]. Bus travelers in Dublin are aided in their decision-making by a system of sensors sending-real-time information as to the location and speed of all services, which is processed in real-time to provide information which the traveler applies in their travel decision-making [37]. In the context of AI-based systems, the decision of the citizen is recorded by way of cameras, swipe cards, or chatbot transactions, and through machine learning comes evidence for future citizen or government decisions, be they transactional, operational, tactical, or policy decisions [38].

5. Discussion

Because sustained, quality decision-making is highly influential to the success of socio-technical systems, our research sought to understand how smart city theory and practice had advanced the understanding of the use of evidenced-based decision-making in smart city policy. We were guided by the question: what does the literature report about the role of evidence-based decision-making in smart city government policy-making?

This focus on decision-making in the smart city context required us to separate the evidence as to policy decisions from the evidence as to operational and tactical decisions [76]. We have reported the evidence as to each of these purposes because the assembled evidence was that evidence-informed decision-making at all three levels of purpose was highly important to sustaining the achievement of smart city objectives. Operational decision-making was the dominant focus of the literature on decision-making, with policy-making being addressed but to a distinctly lesser extent. Importantly, the articles regarding evidence-based policy decisions in smart cities did not carry out examinations of factors [19] impacting performance and capacity of evidence-based decision-making systems. Tactical decision-making was mentioned even less frequently. However, the evidence was that data used for real-time decision-making were in some instances also used later for tactical and policy decision-making. Many of the articles as to real-time decision-making focused on the experience of road traffic or transportation and mobility, in which decision-making was made by humans in control centers or by artificial intelligence agents. We found unequivocal evidence of evidence-based decision-making leading to better achievement of city objectives at the operational level [36], tactical level [40], and policy level [40]. Indeed, more effective, participatory, and collaborative decision-making is a significant element of smart city conceptualizations and practice, to the extent that European smart city practitioners ranked changes to decision-making systems as more important than the use of ICTs per se [1,3]. However, whilst smart city theory [25,36,56] places strong emphasis upon citizen participation in decision-making, we did not find evidence of citizens taking part in governance decisions; rather, they were invited in to contribute to the wider decision-making process.

The theory and practice that contribute to our knowledge of smart city decision-making was dispersed across the literature of many disciplines and was often not ‘signposted’ in the article. In this regard, our learning as to research methods was that data base searches, even full-text searches, were only a starting point, requiring painstaking following of forwards and backwards citations. This report relies upon 37 articles from the data-base searches, and 47 from following citations and subject matter experts. We reported the knowledge in the themes which presented, namely, the purpose of the decision, the characteristics of the evidence, and the decision-makers. This approach revealed persistent areas of concern, namely the themes of challenges associated with increasing the use of evidence-based decision-making in smart cities, and the absence of a fit-for-purpose definition of decision-making in the smart city context. We now discuss those concerns.

5.1. Collaborative Decision-Making Capacity of City Government

We identified two themes in the evidence, one being the data required for decision-making not being shared by organizations separate from the decision-makers such as the city government, and the other being the necessity that such separate organizations truly collaborate with city governments, making shared, not independent, decisions to achieve smart city objectives. The two are inextricably intertwined and we believe that problems of data not being shared are evidence of the larger problem for government where interorganizational collaboration with the same or other levels of government or with private or not-for-profit organizations is very difficult to establish and sustain [59]. For Keast, Brown, and Mandell (2007), shared decision-making is an essential dimension of collaboration [59]. For us, the evidence raises strong concerns that smart city theory has not explained how difficulties with achieving collaboration and shared decision-making between government organizations and other organizations experienced in the wider public sector can or have been overcome.

Collaboration, and the inherent shared decision-making within the city administration, is conceptualized in theoretical articles as beset by a silo approach to decision-making and management by departments individually [36,55]. Authors reported that, in practice, this silo approach continues and leads to dysfunction by departments, frustrating the decision-making of the city government, and the implementation of operational, tactical, and policy decisions [53,56,57,58]. There is limited reporting of innovations to address or obviate this internal collaboration problem, such as the Amsterdam Smart City innovative organizational configurations which are situated outside municipal government [57]. For city governments, strategy preparation and adoption are a major challenge to sustainable implementation of innovations, a problem exacerbated by the governance of smart city initiatives, which are usually working in parallel with the existing administrative structures [80].

This intra-organizational dysfunction was found to frustrate city governments in achieving alignment of data specifications, systems, and sharing across departments which was to bring together data to form evidence to support decision-making as to reducing carbon emissions [53]. Similarly, failings in achieving the sharing of data for decision-making between city governments and other organizations were attributed by Giest (2017) to the number of stakeholders, the lack of standardization of data and their collection by government, and missing links between levels of government, which in some instances was attributable to a national approach of decentralized governance structures [53]. Other literature reported problems as to privacy/legal constraints upon releasing the data for the proposed decision-making purpose [52] and absence of agreement as to the need to provide the data [51].

The evidence of the problems associated with intra-organizational and inter-organizational shared decision-making in the smart city context was not accompanied by any smart city prescriptions about how collaboration and shared decision-making can be achieved. Yet, smart city models rely heavily on collaboration across the city and smart cities require better decision-making to be able to address their quite challenging sustainability objectives in the areas of environment, quality of life, and economic success. Thus, smart city practitioners and theorists must be equipped with a range of tools to design and create governance and service delivery systems that emphasize effective evidence-based decision-making at all levels: operational, tactical and policy. We do not know what those tools will be, however, we believe that their formation can be informed from evidence gathered from two directions. The first being the literature as to theory and practice of decision-making in the wider public sector, and the second being a further development of the evidence as to the experience of smart cities as to collaborative arrangements, such as the municipal control centers [58], which have demonstrated that organizations from different levels of government and other sectors can develop authentic collaboration and make shared decisions.

5.2. Countering the Privileging of Statistical Data

The literature regarding decision-making in the smart city context is dominated by big data as evidence to the extent that, at the least, there is a clear need for knowledge about the contribution of evidence from other qualitative sources and, at the worst, international vendors of big data technology will have driven not-so-smart cities to implement decision support systems which focus on immediate, operational decisions and largely ignore long-term strategic policy decisions.

The literature on smart cities presented an avalanche of articles advocating or reporting the use of statistical data [31,35,38]. Increasingly, that statistical data is gathered through a big data approach involving the interrogation of high-frequency data using analytics, artificial intelligence, and data mining [38,39,43,46,81]. Yet, problems around the provenance of the data [31,38], the appropriateness of the data to the research question [4], and authors conceding their own big data centered methods for transportation research require supplementation by other data gathering methods such as interviews and surveys [35] abound. Indeed, Kandt and Batty (2021) recently judged that the continuing rise of big data will result in long-established, low-frequency knowledge being more, not less, important [31].

This gross imbalance in the types of evidence available for smart city decision-making is portrayed in the literature as attributable to the dominance of smart city thinking by ICT-related matters. However, we pause to acknowledge that beyond the smart city, in the wider public management policy decision-making context, ICT system-sourced statistical data has also dominated, displacing non-statistical evidence and attracting criticisms as to relevance and encouraging impersonal decision-making [8]. The smart city focus on short-term decision-making may be caused by the availability of transactional and high-frequency data and vendors creating products to leverage that data for their profit and possible benefit to the city. Goodspeed’s (2015) analysis of the statements of corporations promoting smart city solutions showed a focus on effectiveness and efficiency through real-time decisions on short-term management issues [82]. Kitchin (2019) believes this ‘realtimeness’ enabled by big data results in a tendency in city operations to privilege real-time response over planning, reflection, and long-term strategizing [37]. The disproportionately small proportion of reported uses of big data in decision-making being about long-term, policy matters may be attributable to understandable practical limits such as sharing and privacy, as well as costs of data transfer, storage, and analysis [31].

Notwithstanding, Kandt and Batty (2021) believe that city governments must confront these barriers because of the need to address long term policy challenges and reposition the effective use of data from being a planning process issue to being a specific item on city political agendas [31]. The road out of privileging of big data and rebalancing of focus towards policy decision-making is beset by obstacles such as city governments not having the human and technical capacities [53] and not being able to sustain the cost required for a big data approach to evidence [31], and, indeed, the analytical capacity to support decision makers to make effective decisions [53]. The risk inherent in smart city governments succumbing to the smart city promise of better decision-making through technological fixes is that better evidence-based decision-making is not delivered but, rather, the capacity of the city government is hollowed out and the hegemony of technology companies is inflated [83].

Going beyond big data to the wider context of utilizing data-driven technologies presents perils and pitfalls as to the actual planning and decision-making where the choice and application of data can potentially remain selective, flawed, biased, normative, and politically inflected [17]. These complexities require a range of other instruments, policies, and practices that are sensitive to the diverse ways in which each city is structured and functions [4]. For these reasons, substantial additional research is required. This smart city research might be informed, not only by the experience of smart cities, but also by the experience of counterpart activities from both the private and public sectors, looking for learnings as to evidence used in decision-making at operational, tactical, and policy levels in, for example, rail services, utilities, or governance. Alternatively, researchers could look internally at smart cities, collecting empirical evidence as to types of evidence used in decision-making and developing knowledge as to what works, and areas where further knowledge must be developed.

5.3. AI Agent as Decision-Maker in City Government

The literature on smart cities as to the theory and practice of decision-making mostly presents the city government, or merely the city, as a simple, single decision-maker, yet our unpacking of the literature revealed a complex reality wherein the city administration was a fragmented decision-maker and the city government was required to cooperate and collaborate with a myriad of other organizations to make shared decisions. Strikingly, a small body of articles [63,64] portrayed artificial intelligence agents as decision-makers. We found that there are small beginnings of examination of the impact of the use of AI agents such as chatbots in the literature on smart cities [38], the literature applying to broader public service delivery [64], and in the cross-sector literature [63,66].

Authors raised loud alarms about the evidence of adverse impacts in decisions about credit, healthcare, human welfare and employment problems [66], and the AI system not being aligned with the values surrounding the service being provided [63,64]. Gabriel (2020) proposed a path for the development of principles that would guide the alignment of technical aspects of AI with norms, and Makasi et al. (2020) developed a taxonomy of public values that could be applied in the design of the technical aspects of AI systems [63,64]. Both these contributions to better AI decision-making capability are conceptualized at a general level and are not specific to a type of service provider, for example a city administration.

The many, many city governments are at different stages of their smart city journey and, in turn, present similar but different services. Governments at all levels differ in their capacity and willingness to embrace new technologies and fund innovative research, just as their populations differ in technological competence, resources, and take-up of technological advances [65]. The local government decision-making eco-system is highly politicized. The negative impacts attributed to AI agents’ decisions will increasingly impact city government and may well amount to being ‘not-smart’.

We believe that smart city theorists and practitioners need knowledge specific to the use of AI by smart cities for decision-making to achieve and sustain objectives. The scope of research into the impacts of AI agents as decision-makers is broad and immense. Because literature on smart cities regarding the non-technical aspects of AI is in its infancy, research as to theory and practice might benefit from scrutinizing the knowledge as to AI in interactions with individuals, focusing in on service delivery in the commercial context and then public administration. At the same time, qualitative research gathering data from informants and archival sources involved in smart-city use of AI decision-making with the principal objective of building knowledge as to the capacities of smart cities to implement the use of AI agents including a comprehensive listing of emerging problems and actions to address these problems is necessary.

5.4. Defining Smart City Decision-Making

We carried out our research using a number of tools, one of which was the most suitable available definition of evidence-based decision-making, provided by Baba and HakemZadeh (2012) [11]. However, our examination of the evidence through a number of lenses, namely the purposes of the decision, the range of types of evidence applied, and the range of actors involved in the decision, lead us to conclude that smart city scholars and practitioners need a fit-for-purpose definition of smart city decision-making. Key to this conclusion is the future-oriented nature of smart city conceptualizations as planning for the future [31], for ‘the long run’ [31], through which we found decision-making to be woven, and the increasing prominence of algorithm-based AI agents to make decisions.

Accordingly, we concluded that a definition of smart city decision-making must make certain the following aspects:

• The actors making the decisions;
• The range of types of evidence utilized, acknowledging the emphasis upon ICTs;
• The range of possible purposes of the decision;
• The forward-looking, positive objective inherent in smart city conceptualizations.

These criteria have been applied in the development of the following prototype definition:

Smart city decision-making is city governments, organizations, individuals, and artificial agents, separately or together, determining actions by applying analysis methods to be taken (or not) in the present or future regarding operational, tactical, strategic, or policy matters having regard to an appropriate balance of ICT-based, and other forms of, evidence.

We have reviewed whether this protype definition is indeed fit for the purposes of smart city theorists and practitioners using two alternate researchers who had not been involved in the construction of this protype definition. Their review included testing against the evidence assembled in this paper and their broader personal knowledge. Their minor adjustments are reflected in the prototype above. Further validation can be made through empirical research as to the experience of cities, most likely in conjunction with research recommended elsewhere in this Discussion section. Such a validation process must be alert for both the inclusion and exclusion of factors that define smart city decision-making.

6. Conclusions

Our research expedition started with an objective of understanding what the literature on smart cities and experiences could add to the existing body of knowledge regarding public sector evidence-based policy decision-making, and whether it was fit-for-purpose for the smart city context. The dispersed and unclear literature regarding smart city decision-making led us to separate out and reassemble the data to present an understanding of the knowledge and its shortfalls.

The assembled literature presents significant knowledge about the use of high-frequency transactional data as evidence in decision-making aimed at achieving and sustaining smart city objectives. Significant benefits from making decisions by way of a big data approach were demonstrated in numerous articles which mostly focused on transportation, traffic, and mobility. A smaller body of articles brought to our attention problems and challenges in the areas of inadequate collaborative decision-making, big data being privileged over other forms of evidence, AI agents becoming decision-makers that do not reflect the values held by the particular city society, and the concept of smart city evidence-based decision-making being confused and incomplete. We wondered why the knowledge in these areas is underdeveloped. Pevcin (2019) observes that research funding has driven the numbers and direction of publications on smart cities. The utilization of the smart city concept can also be attributed to the so-called technology push [84].

Our findings about smart city decision-making led us to recommend a precise definition of smart city decision-making and also recommend separate streams of research drawing on sources from both outside and within smart cities. These streams of enquiry would seek to better understand both the existing theory and practice as to decision-making, not only by the experience of smart cities, but also in the wider public sector, how successful collaborative decision-making arrangements are sustained in any sector, and the types of evidence used in decision-making, in particular the role of AI as a decision-maker. We propose that this knowledge would be interrogated to assist in the development of smart city-specific theory as to decision-making at the operational, tactical, and policy levels. Yet our wider knowledge as to what works in smart cities, and what does not, lead us to reflect that there are bigger issues, beyond the research that we recommend, in the wider smart city ecosystem which may constrain the effectiveness of the reforms possible from that research. The Smart Cities Study (2017) found that 85% of cities have specific projects to promote smart and digital physical infrastructure; however, only 60% of cities have a formalized smart strategy, raising concerns for us as to whether the areas of research and reform regarding decision-making and evidence on which we focus will be effective in the absence of an overall smart strategy in the city [85]. Similarly, the main barrier observed in the Smart Cities Study (2017) was the complexity of the existing bureaucratic processes at the various administrative levels, combined with the lack of coordination and collaboration between actors [85]. This problem was highlighted by Meijer and Bolivar (2016) who advocate institutional transformation to achieve collaboration between city government and other city actors [3]. Collaboration delivers shared decision-making [59]. Consequently, we see distinct constraint upon the effectiveness of the decision-making reforms that we advocate research pursue, until institutional changes within city administrations are affected.

Nonetheless, we recommend that scholars and practitioners take up and improve upon our definition of smart city decision-making, and join us in our agenda of research into how decision-making, and the evidence applied, in smart cities can be improved.