A Method for the Evaluation of Urban Freight Transport Models as a Tool for Improving the Delivery of Sustainable Urban Transport Policy

Abstract

The article presents a method which helps local authorities to evaluate urban freight transport models. Given the complex requirements for input data and the inability to supply them for most cities, a proper quantitative evaluation of model functionality may be quite difficult for local authorities. Freight transport models designed to support sustainable urban freight transport objectives are a particular example. To overcome these difficulties, the structure of the method is based on a qualitative analysis of strategic and operational conditions of urban freight management for modelling purposes. A consistent set of criteria is developed to help with parameterising strategic objectives and the analytical requirements of tools to achieve those objectives. The problems of data availability and capture are also included. The method consists of three tiers that are arranged hierarchically to reflect the interrelations. The proposed method was verified against Gdynia’s (Poland) urban freight management requirements. The city was chosen for its early experience of urban freight studies and improvement measures and because it has already defined its strategic objectives. Two comprehensive freight transport models (Freturb and Wiver) and existing city’s transport model were evaluated. The results have ruled out the existing transport model rendering it ineffective as a tool to support urban freight management to meet the city’s strategic objectives. While Freturb turned out to be much better suited for the needs, dedicated models still face a basic barrier of cities having to redesign their systems for collecting urban transport data.

1. Introduction

If cities are to manage their transport systems sustainably, local authorities must have the skills to choose comprehensive solutions by assessing the planned outcomes. This usually involves understanding the effects on transport infrastructure, emissions and noise, and the safety of all users. It is critical to cover these issues, if the approach to transport system development is to be consistent and follow an accessibility paradigm [1], which marks a departure from the previously used predict and provide approach [2].

Transport models are key to the process. A transport model is a tool providing a quantitative and qualitative output of the likely impacts of alternative solutions formulated at planning level [3]. Requirements emerging from an accessibility paradigm set the new objectives for decision-makers and transport professionals [4]. Instead of undertaking their practice in confined and disciplinary fashion they are requested to embrace a holistic view on mobility [5,6]. Transportation practitioners acknowledge the potentially significant contribution of transport models to understanding the complexities of transport issues and develop more effective strategies to address them. On the other hand, they also stress the need for models to more precisely support the learning process of planners and local decision-makers (authorities) to fully realize this potential [7,8]. According to evidence provided by May [9] importance attached to modelling instruments reflected the policy priorities of respective authorities. Modelling was seen to be most important for road infrastructure, traffic restraints, land use schemes, and public transport improvements. Other issues, such as modelling of walking, cycling, and traffic management received less attention. No solutions related to urban freight transport were mentioned, marking a distinctive gap in the policy approach.

Unlike passenger travel, which is well developed, urban freight transport modelling has stagnated. There are at least two causes of the stagnation. Number one is the internal complexity of urban freight transport with local authorities having to apply separate analytical methods that are not normally used in passenger transport practice. Freight models use different sources of information which makes integration of both systems a complicated task. So far there is no evidence of a comprehensive and ready to use city level modelling system including both passenger and freight vehicle movements [10]. The second problem is that local authorities are not clear about the scope and extent of urban freight improvement measures. Most local authorities only engage in short-term and ad hoc solutions to tackle problems as they arise [11] with no ex ante assessment supported by simulation models [12], making transfer of approaches from one area to another a difficult task [13]. Most urban transport policies do not address these problems, either [14]. Give this, any attempts at freight modelling will have a limited quantitative evaluation due to the different application scopes and model characteristic. However, change is on the way with cities now following the guidelines for developing sustainable urban mobility plans (SUMP) [15] and sustainable urban logistics plans (SULP) [16]. As a result, more and more local authorities in a number of countries are introducing these policies which could increase a demand for analytical tools supporting their implementation [17].

The above mentioned factors clearly show a lack of public authorities’ capability to manage urban goods transport. This leads to a restrictive implementation of urban freight in planning policies and a poor evaluation capacity during the decision process using tools such as transport models [18]. Therefore, the following main research problems can be identified:

• Lack of capacity to properly identify urban freight transport characteristics within a city’s transport system. This influences their ability to adopt effective solutions supporting the implementation of sustainable transport policy, such as urban freight models.
• Local authorities do not follow a systemic approach to urban freight transport. This results in a lack of clearly defined policy objectives or relevant implementation indicators. As a result, it is difficult to precisely define requirements for transport models whose functionality should reflect the purpose for which they are applied.
• There are no comprehensive studies on how urban freight models are applied to improve the implementation of measures. As a result, a reliable connection between sustainable urban policy objectives and policy implementation measures supported by transport models can hardly be established.

Given this, the objective of the work is to develop and verify a method for supporting local authorities with evaluation of urban freight models. The idea of the method is to ensure that there will be a comprehensive check of the requirements and capacity of a city and that all issues involved in modelling are covered. With no previous research looking into the possibility of developing a cross-cutting method to support the evaluation of urban freight models, the method was designed to combine urban freight objectives, tools to deliver the policy, the conditions of the functional structure of urban freight, and the required data. There are some general assumptions which any city should address when deciding which freight models it wants to use [11,18] (Figure 1).

Given the complexity of the problem, the work is divided into six parts. Section 1 (introduction) explains why the topic was addressed. Section 2 (literature review) defines the problem’s position within the structure of available research. Section 3 defines the methodological background for defining the models’ evaluation criteria. The issues include identifying the impact of urban freight structure on the models’ evaluation criteria. The problems of urban transport policy parameterisation are also tackled to make a reference to the modelling requirements. Identification of the analytical requirements of improvement measures is also included. This section also analyses the importance of data availability in the context of evaluating the models. Section 4 presents the structure of the new method and Section 5 outlines how it was implemented and evaluated in a specific city. Conclusions are given in Section 6.

2. Literature Review

The problem of urban freight transport model evaluation corresponding to the requirements of a city’s transport policy is neglected in the literature. Feliu et al. [11] analysed public policy objectives and suggested tools to integrate urban freight in decision-making. Their work provided a general insight into the applicability of several categories of models at different planning levels and against selected objectives, such as prediction of traffic flows or understanding the relations between demand and supply. However, this analysis had a very general character without in-depth analysis of factors determining the application of models under consideration. Moreover, it referred to general model categories (4 step models, tour-based models, etc.), rather than to specific models and their practical considerations. No other work concerning this problem has been identified, mainly due to underutilization of urban freight models by local authorities and uncertainty of their functionality.

Other works estimating the impact of urban freight are concerned with methodologies for scenario analysis in freight modelling [18], requirements for demand modelling for forecasting direct effects [19] or the possibility to assess environmental effects of urban freight [20]. There were also studies on the applicability of selected models with regards to specific goals of local transport policy [21]. These approaches were confined to selected models and their operational effectiveness in a given situation. Other studies provided cross-cutting analyses of modelling methods [22,23], developed a classification of existing techniques [24,25] and looked at the problems of data availability [26,27,28]. Despite their high relevance to understanding principles of urban freight modelling these studies only focused at a general classification of models without any reference to concerns of their end-users, namely local authorities.

Given the poor availability of research on evaluating urban freight models with regards to local authority requirements, the question arose whether existing decision support methods could be adapted for use in transport problems.

A method like this should be able to work with the complexity of the issue and ensure that both strategic planning of sustainable urban transport and model operational requirements are met. An assessment is needed of whether the current decision support methods in transport can now be used by local authorities to evaluate and select a freight transport model. The primary decision support methods in transport include:

• Participative decision-making methods involving the private and public sector;
• Multi-criteria analysis including the input of groups of transport system users (MAMCA—multi actor, multi-criteria analysis); and
• Analysis of economic and social costs and benefits of measures designed to improve urban freight transport (cost benefit analysis/social cost benefit analysis).

The methods [29,30] were analysed and it was established that urban freight issues are represented there sporadically. Neither was there a case involving the use of a method developed specifically for selecting a freight transport model.

The first of the analysed categories includes participative decision-making methods. Where transport is concerned, the focus of the methods is on representing the specific behaviour of participants under specific conditions. Examples may include the effects of urban transport policy scenarios on user behaviour [31]. The analysis also looked at decisions to purchase transport services [32], factors that determine whether an urban consolidation should be built or not [33] and effects of night-time deliveries on logistic operators’ effectiveness [34].

Since these solutions aim to simulate the behaviour of the participants in the decision-making process, they cannot be used for evaluation of freight transport models. Their goal is to provide a quantitative representation of a broad spectrum of decision-making factors. Models, however, should be selected after an in-depth qualitative analysis to understand whether the model can be applied as a tool to support urban transport policy. It is not possible to compare the accuracy of simulating selected models and using this to develop quantitative indicators for the decision-making model. The available comprehensive freight transport models require an individualised operational environment which rules out any direct comparisons for the same applications.

Similar reservations apply to the other two decision-making support categories. Multi-criteria methods use different approaches to building quantitative decision-making criteria depending on the problem in question [35,36,37]. To that end, sets of criteria must be parameterised very precisely based on identified preferences of those involved in decision-making. Analysing costs and benefits (CBA) and social costs and benefits (SCBA) are the least useful methods for evaluation freight transport models. They monetise the effects of transport as a business [38,39,40,41] but the ultimate goal of choosing a freight transport model is to improve the capacity for sustainable urban transport rather than assess the economic effects of the process.

The results of this section’s analysis suggest that local authorities do not have tools to support their choice of urban freight models. As a result, it makes sense to develop a method to support local authorities in evaluation of urban freight transport models which will reflect all the relevant conditions as identified so far.

3. Methodological Background for Identifying Evaluation Criteria for Models

3.1. The Influence of Urban Freight Transport Structure on Model Evaluation Requirements

A method for the evaluation of urban freight transport models should comprise a comprehensive set of criteria. Based on previously identified research gaps, it must be derived from identified freight transport characteristics as part of a city’s transport system. Moreover, the criteria should reflect both urban freight policy objectives and potential measures available for its implementation.

An implication for selection of the evaluation criteria was the sustainable urban mobility plan (SUMP) concept and its planning cycle structure [15]. Urban freight transport should be seamlessly coupled with planning of other transport modes. According to the new approach presented by a SUMP this process should focus on definition of clear objectives supported by short- and medium-term delivery plans, assessment of the current and future performance, and regular monitoring and evaluation of impacts. These factors indicate a demand for reliable analytical tools such as transport models capable of supporting the delivery of the transport policy. Their feasibility relies mostly on the ability to cover indictors describing freight transport with a level of detail required by decision-makers to meet the above-mentioned requirements.

With regards to a planning cycle of a SUMP, the primary area defining criteria for evaluation of urban freight models is related to a phase covering rational and transparent goal setting. It includes setting objectives and measurable targets and developing effective package of measures for their realisation. Clear and measurable objectives must be defined that help to direct implementation measures selection. Measurability should be secured with a set of core indicators for each objective with proper data available to describe them properly. To respond to this challenge adequately, urban freight models must be able to represent these indicators. Hence, the core indicators for realisation of sustainable transport policy represent the main evaluation criteria used in the presented method as described in

Table 1. Urban freight transport sustainable management objectives and related indicators.

Objective	Problem Areas	Method of Description	Core Indicators
Economic efficiency	Transport accessibility	Total travel time	– average vehicle speed
	Transport services supply and demand	Freight and transport demand generation	– freight/delivery demand generation ratio
Operational efficiency	Distance optimisation	Total distance	– distance during delivery process
		Efficient vehicle working time	– total time without driving time – single delivery duration
		Delivery organisation	– Deliveries per round.
	Delivery efficiency	Fleet utilisation level	– Freight vehicles kilometres – Number of deliveries per entity/per 1 employee – Number of vehicles in selected areas. – Load utilisation level. – Daily distribution of deliveries
	Vehicle characteristics	Structure of vehicle types	– Proportion of goods vehicles in total traffic – Number of vehicles according to their classification
Road safety	Risk reduction	Characteristics of incidents involving freight vehicles	– Number of incidents – Freight veh. involvement rate
Environment protection	Reduction of air and noise pollution	Operation characteristics of freight vehicles	– Emission factors – Share of low-emission vehicles in total kilometres travelled
Infrastructure management	Demand for freight transport infrastructure	Characteristics of freight infrastructure utilisation	– number of dedicated delivery places and veh. rotation – number of public parking places – places of freight veh. stop
	Prognosis of freight vehicles traffic	Factors influencing freight vehicles traffic	– O/D matrices – share of freight traffic at main traffic corridors
Urban structure management	Analysis of freight traffic scenarios regarding spatial planning	Impact of functional characteristics of urban space at freight transport characteristics	– Freight/delivery generation according to activity type – Freight demand and supply ratio according to activity type – Total kilometres travelled according to activity concentration

.

A requirement for a development of an effective package of implementation measures is an important factor influencing the models’ evaluation criteria. The most effective measures must be identified and it is essential that they connect directly to the objectives set previously. To ensure internal cohesion between objectives and measures both should be measured with the same set of criteria. To meet this challenge the proposed evaluation method introduces for the first time a concept of an analytical requirements of urban freight measures. They are expressed with the same set of indicators as for strategic objectives. As a result, urban freight models might be evaluated due to capability of inclusion an objective and related measures’ requirements. Adopting a sustainable urban mobility plan concept as a reference point for determining freight model’s evaluation criteria ensures that the developed method provides high utility rate for local authorities.

The complexity of the urban freight movements structure determines the methods to be used for its modelling and, consequently, the framework of model evaluation. Local authorities must use tools to improve the operation of urban freight and those tools must be selected reasonably. The fundamental challenge here is to ensure that the tools work well with the challenges resulting from areas intended for policy intervention. This leads to the conclusion that as local authorities work to evaluate a freight model, it should be able to represent the complexity of the processes behind it and work as a tool for supporting improvement measures. The general characteristics of urban freight which determine the structure of a model’s evaluation method include:

• Diverse types of urban freight transport; and
• The role of local authorities within the structure of those involved in freight transport.

Typically, urban transport is delivered in different ways and involves different entities [42]. Despite this, transport uses the same limited resource of transport infrastructure at the same time making modelling analyses more difficult. To ensure that urban freight is managed sustainably, a transport model used as a tool to support this process should provide a reliable representation of the specificity of the issue under investigation. This applies to selecting a variable to be modelled and data availability. These criteria are crucial for developing an evaluation method. Once identified, model’s operational conditions should be included in the method to ensure that it is consistent with strategy. Figure 2 shows the organisational complexity of urban freight which must be made part of the model [43,44].

A freight model evaluation method should also ensure that an assessment can be made of how well the models under analysis address the engagement of local authorities in managing urban freight. This ties in with the analysis of the role local bodies have among other freight transport operators and an assessment of what may be required as a result.

Local government is one of the four basic participants of urban freight transport processes. They are senders, customers, and carriers (in the logistic sector) [45]. Each participant is related directly to the functional areas nearest to them. In the case of the city this will be the transport infrastructure and the impact on vehicular traffic structure. The city regulates these areas by implementing traffic management rules and market rules for access to infrastructure [46].

By naming local authorities as an indispensable freight transport participant, it is clear that their role in the system must be reflected in the evaluation method. The analysis should include the nature of the links between local authorities and the other entities with demands voiced by the other system users and how it all fits in with the city’s transport policy. To that end, criteria should be defined for urban freight management objectives to match the characteristics of problems areas that emerge between the entities. The freight model evaluation method should take account of these issues and treat them as a basic axis for determining the requirements to be met by analytical tools.

As well as managing the city’s transport and economic system, local authorities are also responsible for representing the interests of the community. There are many reasons why it makes sense to use freight transport models and why developing an evaluation method is in line with cities’ sustainable urban transport policy objectives:

• Statutory responsibility for the local transport system with managerial systems in place already but requiring tools and procedures to match the specificity of urban freight
• The ability to act as a neutral negotiator between participants of transport processes to guarantee information security and ensure a balance between the entities involved
• Ability and need for coordinating regulations across different areas of urban transport policy in line with the principles of sustainable development
• Ability to ensure continuity of actions by including urban freight in strategy papers and shaping the awareness of those responsible for areas of administration which affect freight in the broad sense (infrastructure management, land use planning, economy, and social policy).

3.2. Parametrisation of Urban Transport System Objectives and Analytical Requirements of Implementation Measures

It is clear from the previous chapter that an effective method for evaluating urban freight models should provide a reliable representation of the characteristics of this type of transport by including criteria related to a primary modelled variable and related data availability.

However, reference must also be made to a problem which so far has not had any comprehensive scientific coverage, namely linking parameterised freight management objectives to the analytical characteristics of available tools for delivering those objectives. The end result should be a systemic solution for translating strategy into practice, a must for an efficient urban transport policy, especially given the complexity of urban freight transport. By fulfilling this requirement a coherent set of criteria to help with developing the method would be defined.

Having measurable urban transport management objectives is increasingly treated as a basic requirement of sustainable development [47]. It is also a point of departure for selecting the right methods of delivery [11]. The assumptions of sustainable development of a transport system [48] are derived from the overall definition of sustainable development [49]. The basic challenge is to ensure access to transport systems for all user groups while maximising the effectiveness and minimising the negative effects of transport [50].

The objectives of urban freight management can be parameterised by identifying the basic areas that are impacted by transport and include sustainable development, mobility and living conditions of the population [51]. The objective of the measures taken in these areas is to reduce the negative effects of transport [52]. In the case of urban freight transport, there is a relation between features of urban freight transport operations and their negative impacts [53]. Negative effects of freight can be assessed (noise, air pollution, congestion, etc.) by linking them to kilometres travelled. However, this indicator alone would be too much of a simplification when parameterising urban freight management objectives. A relation must be established between the objectives [54,55], the resulting problem areas, ways to describe them and the relevant indicators [56]. Core indicators used to describe urban freight sustainable management objectives constitute a basis for developing the method for model evaluation. If a given model includes selected indicators it could be used to support the implementation of related policy objectives. Core indicators are shown in Table 1. Indicators that occur more than once are left out to ensure clarity.

The second aspect of the methodological review is analysis of analytical characteristics of measures used to implement a city’s transport policy objectives. Identified in the introduction, analytical requirements of urban freight measures are parameters and data that are necessary for the ex ante and ex post assessment of solutions designed to deliver a city’s strategic goals. They are a result of technical and organisational conditions needed to deliver the goals in practice. If this process is to use freight transport models, strategic objectives and tools for their delivery must follow the same indicators as previously identified. Using a uniform set of quantitative indicators will help to verify the functionality of transport models within the evaluation method As a result, the basic task is to categorise available solutions so that quantitative indicators can be reasonably assigned to them which in turn determine their analytical requirements.

Improvement measures that can be used in urban freight transport come in a number of classifications [14,57,58,59,60]. Since their analytical requirements must be expressed using quantitative indicators, we will start with a classification developed by Munuzuri et al. [14] who proposes a breakdown into simple and complex solutions. Simple solutions pertain to a precisely defined problem or use group. Complex solutions are usually a combination of several simple solutions. While their impact on the transport system is stronger, the planning and requirements are more rigorous (analytical requirements). When defining analytical requirements, the breakdown into core and additional criteria should also be included [61,62]. They are helpful with selecting the key criteria for a comprehensive analysis, once a solution is selected.

Considering the above, a classification of analytical requirements is proposed to be applied to improvement measures as part of the method under development. Improvement measures are divided into those designed for freight movements optimisation, reduction of demand for freight movements and supporting technology solutions [63] and correspond to the concept of independent and complex solutions. While successive categories demand more and more data, they share a basic scope in all three groups with the additional criteria simply making the analysis more detailed. Figure 3 shows an outline of the solution.

The analysis in this chapter provides a solution for overcoming identified research gaps. It integrates parameterised management objectives with the analytical requirements of their implementation measures using a joint set of quantitative criteria. All this forms the basis of the method designed to support the evaluation of urban freight transport models. Models under consideration can be verified for how well they take account of parameters required for a quantitative analysis of improvement measures. Local authorities can also define which tools are more likely to achieve the city’s transport management objectives.

3.3. Significance of Urban Freight Data Availability for Model Evaluation

When evaluating freight models, local authorities must take account of two issues that have to do with data availability for modelling. One of them is day-to-day availability of data to describe the processes in sufficient detail. The second issue is ensuring long-term adaptation of the existing urban data acquisition system to the requirements of effective freight modelling. These factors must be considered as an indispensable element of evaluation as they define the basic requirements for urban freight model implementation by an authority.

Despite a growing number of practical examples of direct local authority involvement in urban freight transport, decision-makers are not quite happy with its quantitative analyses. Dealing with a high number of entities means having no single point of contact with a comprehensive view of the situation or a quantitative description of the system in question [64]. It should be stressed that data resources typically used by local authorities for a city’s transport planning are useless for modelling urban freight transport. Despite two types of flows occurring within the same transport network, there are fundamental differences regarding the structure and granularity of data required for their adequate description. Moreover, standard procedures, such as roadside traffic counts, could not be directly adapted for the purpose of urban freight analysis.

Data availability is often cited as the main problem in urban freight modelling. It inflicts, among others, an ability to evaluate how policies affect distribution patterns [21,65]. Concerns may also be related to identification of facility location and its impact on freight movements [66]. Other research confirm that information barrier is the main problem in adoption of innovation in transport policies, which is especially important in urban freight where examples are scarce [67]. Problems with reliable data provision affect not only the potential to implement any model, but also limit the ability to perform a reliable quantitative evaluation for the purpose of sustainable policy implementation. The available comprehensive urban freight models (see Section 5.1) require a different type of input information, which cannot be provided without complex surveys. Some information may be obtained with ITS/ICT systems, but they do not provide a complex solution to the problem [68]. As a result, local authorities evaluating freight models must rely on qualitative criteria and carefully consider whether they would be able to establish a complex data provision system to feed to the selected model.

There are only a handful of examples of comprehensive approaches to the problem. They include France’s survey of commercial deliveries conducted for the purposes of the Freturb model [69,70,71]. Similar surveys were also carried out in the United Kingdom [27,72,73]. The structure of deliveries and freight characteristics were also studied in Rome to understand the effectiveness of the city’s measures and verify the effects of limited truck access to the city centre [21,74,75]. In Germany every eight years there is a nationwide survey of how commercial freight vehicles are used: the KID (Kraftfahrzeugverkehr in Deutschland) [76]. Results of these surveys are connected to the local context and cannot be directly transferred to other countries without extensive research and verification. Therefore, data availability estimation is another pillar of the method for evaluation of urban freight models.

4. Structure of the Method to Evaluate Urban Freight Transport Models

In the introduction Figure 1 shows the general factors influencing the evaluation of urban freight models by local authorities. For clarity’s sake they are grouped into three categories, as presented in

Table 2. Assumptions for the development of the method supporting the evaluation of an urban freight transport model.

Functional and organisational characteristics of urban freight transport	Availability of urban freight analytical methods and ability to assess the relevant actions	Availability of freight transport models and selection tools
– internally diverse in terms of organisation and entities involved – need for clear separation at analytical level within urban transport system – local authorities key to creating the conditions for transport processes – poor system knowledge of decision-makers about UFT operations – need for practical criteria to describe relations between the entities, a basis for modelling	– need for a dedicated system for data acquisition – highly specialised data and inability to exchange them between modelling methods – need for advance definition of UFT objectives as part of urban transport policy – need for parameterisation of expected policy results – need for inclusion of analytical tool requirements in relation to transport policy objectives	– inability to use methods used in national freight transport models and urban passenger transport models – limited number of comprehensive, dedicated freight transport models and strong methodological variation of the other models – lack of tried-and-tested methods supporting the selection of a model by local authorities

.

Considering the above, the structure of the support method for local authorities was defined. Since the problem must bring together issues of varying levels of detail, a hierarchical layout was used with each step moving further in-depth. Given the poor systemic knowledge decision-makers have about urban freight, the structure prevents users from taking a fragmented approach to the problem.

The method is qualitative in nature and checks the functionality of models for its pertinence to the city’s transport policy and experience of improvement measures. It also verifies the city’s capacity for ensuring a stable operational environment for the models which includes data provision. By doing this, there will be feedback within the method, and strategic objectives and the tools to achieve them can be verified along with a practical assessment of whether transport models can be used. The structure of the method is as follows:

• Strategic level where freight transport models are checked against the strategic objectives of urban freight transport management. The basis for the analysis is to identify quantitative indicators for parameterisation. With a known functional structure of models, they can be assessed for how well they can support the delivery of strategic objectives while ensuring that their key parameters are represented. Strategic objectives can be expressed with a set of indicators which would serve as decision criteria at this level.
• Tactical level where an assessment is made of the analytical requirements of improvement measures (Figure 3) in reference to what the models can do. This problem had not been studied before. It is also one of the most important issues that determine whether models can be used in practice when planning and delivering a sustainable urban transport policy. To ensure that both levels of analysis are consistent, the same quantitative set of criteria is used. As a result, a single point of reference is maintained.
• Operational level where urban freight data are checked for their availability and the capacity to build a comprehensive data collection system is verified. In this case again reference can be made to quantitative criteria applied to parameterise strategic objectives and to define the analytical requirements of improvement measures. As a consequence, a method designed around the qualitative approach uses a quantitative element to combine the problems of strategic freight management with the technical and functional characteristics of transport models. The possibility of feedback with previous levels of analysis is also included. This may involve data acquisition when selecting improvement tools or how they should be verified. The parameterisation of objectives can also be changed to take account of the practical measurement ramifications. Figure 4 shows the structure of the method.

To ensure that the method is applied well, a procedure must be adopted to include:

• Analysis of the urban freight management system. The idea is to identify the relevant experience and competence of local authorities and define strategic goals.
• Evaluation of freight transport models being analysed and a synthetic presentation of their characteristics including the requirements of the method in question. This involves: identifying the basic variable to be modelled and the related demand for input data.
• Conduct a qualitative assessment of the models in question and formulate conclusions at each level of using the method. The conclusions may exclude the models from further analysis due to their inability to meet the basic requirements at any given level.
• Present final recommendations regarding the use of the transport models and identify the conditions for implementing the model to meet local authority requirements as much as possible.

5. Implementation and Validation of the Method

5.1. Analysis of the Urban Freight Transport Management System in Gdynia and Selection of Models to Be Verified

As set out in the procedure, the first step is to evaluate the freight transport management system in the city applying the method. The method was verified in the city of Gdynia. A supra-regional economic centre, the city has a strong development potential partly resulting from its position within the Tri-City conurbation. Gdynia is one of Poland’s first cities to have included urban freight transport in its sustainable urban mobility plan developed under the CIVITAS “Dynamo” project [77]. The goals are fairly general because they are aimed at building an effective and sustainable system of urban distribution and supporting the use of relevant modern technologies. Despite that, the goals allow for a number of potential improvement measures. What is more important Gdynia has already rolled out its plan. In 2018 dedicated delivery places were designated on some of the city centre streets, a move preceded by a thorough study of deliveries to commercial receivers [78]. This makes Gdynia a good example of a city which, in the not too distant future, may need to evaluate a freight transport model to improve its capacity for managing the transport system. As well as developing the SUMP, the city has introduced a three-level passenger-focused transport model as a tool to support planning processes [79]. However, an urban freight transport management system presents the basic stage of development. An operational vision of the measures must be accepted to define requirements for supporting solutions such as transport models. In addition, freight activity data resources are scarce. These issues confirm that a method supporting the evaluation of a freight model should comprise several levels providing for a complex analysis of this decision problem.

What makes the evaluation of models using the method more difficult is that the solutions are highly varied and mostly theoretical [24,25]. A local authority seeking ways to support the implementation of its sustainable urban transport policy may think that comprehensive models are the solution. Comprehensive models include all phases of an analytical approach beginning with demand generation and ending with movements assignment phase. Such models are scarce and in Europe only the French Freturb and German Wiver represent the required level of maturity confirmed by the number of practical applications. They work well as practical tools for transport management because they address all elements of modelling, from the generation of demand for transport to traffic distribution over the road network. Wiver was originally developed in 1985 to simulate commercial goods traffic in West Berlin (hence its abbreviation—WIrtschaftsVERkehrsmodell) [80,81]. Later it was incorporated into the general urban traffic model Viseva (as Viseva-W), which included both private passenger traffic and urban commercial traffic [82]. In this paper the original name of Wiver will be used. Trip matrices obtained from the model’s origin-destination demand phase could be transferred to Vissum for an assignment phase and total private and commercial traffic simulation. The basic unit of simulation in Wiver is a single commercial vehicle and its behaviour described by statistical distribution. The number of deliveries and a delivery round structure are analysed from the perspective of traffic producers (commercial groups), which is linked to traffic receivers’ characteristics (trip purposes and destinations). The Freturb model is a behavioural statistical model based on the nature of the generators of goods and transport choices. It is based on the delivery and freight vehicle round structure [83,84,85,86,87]. Contrary to Wiver, deliveries and pick-up generation are performed based on complex receiver surveys and not freight transport provider surveys. Origin-destination matrices for freight vehicles may be expressed in PCU (passenger car unit equivalent), which helps to include them in general traffic flows observed within the city. Despite a similar approach, they offer different levels of applicability with regards to the potential end user requirements. This should be considered when defining the structure of the proposed method.

There are also examples of decision support systems designed to aid urban freight policymaking, such as the CLASS (City Logistics Analysis and Simulation System) [88,89]. It was designed with an intention to be an answer to a complexity and diversity of methods and models used to investigate urban freight systems. It was conceptualised as a solution supporting verification of logistics measures extending classical regulatory measures, such as land-use development governance and concentration of retail activities with respect to distribution centres. The CLASS system is based on demographic, socio-economic, freight demand and supply, land-use, and road network indicators to simulate various scenarios of urban freight policies. In this aspect it may be considered as a step forward compared to the two previously mentioned models. However, it is not directly comparable with them and was not included in the presented method to preserve reliability of results. Another freight transport model should also be mentioned as it follows a general methodology of the Freturb. It is the CityGoods model developed in Italy an tested in several cities of Emilia Romagna region [88,90]. The model provides a supply chain generation matrices for different cities, based on extensive surveys of freight activity [91]. However, available sources [23], [90] indicate that this model is restricted only to the demand generation phase. Hence, it cannot be directly compared to Freturb and Wiver, which are a complex models.

The point of reference for the dedicated freight transport models is Gdynia’s multi-level urban transport model. It is an advanced transport model [92], but just as the traditional four-step approach it is not fully suitable for modelling urban freight movements. However, it has been used as a point of reference because it helps to identify the limitations of traffic modelling based on schemes developed for passenger movements.

Table 3. Synthesis of functionality of transport models selected for analysis.

Category	Freturb	Wiver	Multilevel Model in Gdynia
Purpose	Urban freight transport	Urban freight transport	Passenger transport; basic functions related to freight
Structure	Complex model	Complex model	Complex model
Objectives	Diagnosis and planning, transport policy support	Diagnosis and planning, transport policy support	Diagnosis and planning, transport policy support
Main variable	Delivery and delivery round	Delivery and delivery round	Passenger trip
Methodology	Generation of demand for deliveries, round generation, round distribution	Generation of demand for deliveries, round generation, round distribution	Four step approach
Primary source of data	Receivers delivery survey and logistics operators surveys	National commercial freight vehicles activity survey (KID)	Traffic counts and public transport passenger surveys

presents a synthesis of the models’ functionalities following the requirements of the method.

5.2. Implementation of the Method

Since the method is quite extensive, only the key elements are presented, especially at the strategic level. The most important strategic goal in Gdynia’s SUMP is operational effectiveness of urban freight transport (

Table 4. Analysis of selected models with respect to operational effectiveness indicators.

Indicator	Freturb	Wiver	Multilevel Model in Gdynia
Total distance travelled during delivery round	Total distance in rounds between zones	Total distance in rounds between zones	Only direct trips as a share of general traffic
Total vehicle working time	Cannot be directly estimated	Cannot be directly estimated	Cannot be directly estimated
Delivery duration	Primary model parameter	Not available	Not available
Deliveries per round	Primary model parameter	Primary model parameter	Not available
Direct deliveries	Primary model parameter	Primary model parameter	Not available
Deliveries per 1 employee	Primary model parameter	Estimation per zone based on total employment	Not available
Freight vehicle kilometres	Based on number and structure of delivery trips	Based on number and structure of delivery trips	Basic calculation (only direct trips as a part of total traffic)
Number and type of freight vehicles in selected area	Three categories of freight vehicles	Three categories of freight vehicles	As a share of each category in total traffic
Load utilisation factor	Not available	Not available	Not available
Daily distribution of deliveries	Available based on detailed delivery survey input	Not available	Available only in relation to total traffic distribution
Share of freight vehicles in total road traffic	Provided in PCU (Passenger Car Unit) equivalent	Possible to include O-D matrices in VISSUM software	Freight vehicles may be included in road traffic for each road section

). The summary of the models’ functionalities will be related to all of the city’s strategic objectives. Operational effectiveness issues tackle the main freight transport parameters comprehensively and there is a direct reference to the characteristics of transport models.

In the case of Gdynia the models offer varied functionalities as regards the delivery of the strategic objectives. The current urban transport model was shown to have little use for modelling freight transport. The two dedicated freight transport models have similar functionality at the strategic level. This suggests that a further comparison should be made at the tactical level (analytical requirements of improvement tools) and operational level involving a system for data provision.

Table 5. Summary of model verification with regard to freight policy objectives in Gdynia.

Strategic Objective	Freturb	Wiver	Multilevel Model in Gdynia
Economic efficiency of urban freight transport	Models offer similar functionality with regard to core indicators such as number of deliveries, round characteristics and structure of freight vehicles used. This is related to the main variable used which is a delivery considered as a movement of a vehicle from point of origin to freight receiver. Deliveries are organised in rounds to better reflect the characteristics of urban distribution patterns. Both models differ in how input data are provided and structured. This implies that further analysis is required concerning their potential to support implementation of improvement measures. This would reveal practical differences between these models.		Model offers no practical possibility to support local authority in implementing this objective. It cannot reflect primary indicators required to properly describe the structure of urban freight transport.
Environmental protection	Inclusion of detailed information on delivery structure with regard to characteristics of receivers and freight vehicles enables in-depth analysis of environmental effects.	As the model utilises a rough classification of receivers based on zonal characteristics, it provides less potential to reflect environmental characteristics of transport activity with regard to receiver type.	Model does not provide any opportunity to reflect origin and destination of freight movements with regard to receiver type and vehicle used. These parameters are crucial for the estimation of environmental characteristics of freight activity and its simulation.
Infrastructure management	The model helps to estimate the level of infrastructure utilisation by freight movements with passenger car unit (PCU). It translates road occupancy by different types of freight vehicles into multiplied passenger car units.	The model estimates road space utilisation level related to freight movements by providing origin-destination matrices which may be used in traffic modelling software VISSUM.	The model cannot be applied to this task because it does not reflect factors determining the structure of freight vehicles movements with an acceptable level of detail. Only basic calculations are possible, but they are based on general structure of total road traffic only.

gives a summary of conclusions regarding the utility of the models for Gdynia’s urban freight transport strategic objectives.

As we can see from the strategic level analysis, the urban transport model is the least useful. Despite its advanced three-level structure, it primarily applies to passenger transport. There is very little it can do to support sustainable urban freight transport. By applying the new method to the other two analytical levels, recommendations can be formulated regarding the use of the two dedicated models.

At the second level of the new method, the models will be verified and checked for the analytical requirements of improvement measures. The point of reference will be Gdynia’s designated delivery places for freight vehicles. As presented in Figure 3, this measure is designed to optimise freight vehicle movement. Using a uniform set of quantitative indicators in

Table 6. Analysis of selected models with respect to the analytical requirements of improvement measures.

Category	Indicator	Freturb	Wiver	Multilevel Model in Gdynia
Freight traffic optimisation	Structure of freight vehicles	★★★	★★★	★
	Share of freight vehicles in total traffic	★★★	★★★	★★
	O-D matrices	★★★	★★★	─
	Freight vehicle kilometres	★★★	★★★	★
	Demand for deliveries	★★★	★★★	─
	Demand for freight	─	─	─
	Daily distribution of deliveries	★★★	★	─
	Type of receivers	★★★	★★	─
	Location of receivers	★★★	★	─
	Direct deliveries	★★★	★★★	─
	Deliveries per round	★★★	★★★	─
	Delivery duration	★★★	─	─
	Vehicle working time	─	─	─
	Vehicle speed	─	─	★★★
Freight traffic demand reduction	Load utilisation factor	─	─	─
	Type of transport service	★★★	─	─
	Deliveries per 1 employee	★★★	★★	─
Technological changes	Share of low emission vehicles	★★★	★	─
	Total kilometres for low emission vehicles	★★★	★	─
	Emission factors	★★★	─	─
Rating scale		(─) application not possible (★)—basic functionality (★★)—limited functionality (★★★)—full functionality

an analysis is given of how the models can be used to implement the categories of improvement measures. The measure used in Gdynia is highlighted to check which model could be used for those measures.

The functionality analysis of the models shows that a city’s transport model cannot be used as a tool to support freight measures. Combined with the previous conclusions, it is clear that it should be excluded from further analysis. The other two models at the tactical level have shown some major functional differences.

Table 7. Summary of model verification with regard to the analytical requirements of improvement measures.

Category	Freturb	Wiver
Freight traffic optimisation	Model provides a comparable level of utility with regard to core indicators such as delivery number and vehicle characteristics. However, Freturb offers a more detailed analysis due to input data based on receiver surveys instead of transport providers. A very detailed description of time and spatial characteristics of deliveries is possible. It helps to model freight traffic at local or even street section level unlike Wiver’s zonal approach.
Freight traffic demand reduction	Model provides complex projection of factors determining the demand for freight movements and transport service organisation. If used by local authority it allows for a complex analysis of comprehensive improvement measures beyond simple traffic optimisation solutions.	Potential of the model is reduced by lack of type of service inclusion (own transport or external supplier). This is one of the most important factors when modelling freight demand scenarios. There is no possibility to reflect differences between these type of services with regard to vehicle used, umber of deliveries and total kilometres travelled.
Technological changes	Model includes a dedicated module which allows for a complex analysis of environmental effects of freight movements. It is based on a detailed characteristics of both the delivery process and spatial distribution of receivers and their demand for deliveries. This allows for a very detailed modelling of changes related to the introduction of low emission vehicles or other technological changes to transport process.	There is a possibility to simulate changes in fleet composition at a general level, including introduction of low emission vehicles. The level of detail is limited by input data characteristics, which offer no information on type of service.

shows a list. Comparing the models at the strategic and operational level shows that Freturb provides greater functionality. This is primarily thanks to the structure of input data which help with a very detailed description of the size and characteristics of demand for deliveries to commercial receivers.

The analysis, however, does not include all of the conditions to be met for cities to use models. It also needs to look at the technical and operational conditions of models, most of which have to do with sourcing the necessary input data. This issue is addressed at the method’s operational level. It ensures that a critical assessment of the city’s actual conditions is conducted and provides a practical verification of the conclusions made previously.

Table 8. Data requirements and how data can be sourced at local level in Polish cities.

Category	Freturb	Wiver
Main variable	Delivery and delivery round	Delivery and delivery round
Method of data acquisition	Surveys of commercial receivers complemented with targeted logistics operators surveys	Nationwide survey of commercial vehicles activity survey (KID, every 8 years since 2000)
Primary data availability in Poland	No comprehensive receiver surveys available	No comprehensive commercial vehicle surveys available
Examples of surveys similar to model’s operational requirements	Receiver surveys in Szczecin (2015) and Gdynia (2017) utilising the adopted methodology, but without modelling implications	Limited attempts to analyse commercial vehicle activity, scattered and not conclusive
Possibility to adapt external data for model development	None, see above. Data from other countries may not reflect local conditions of freight operation in Polish cities.	None, see above. Data from other countries may not reflect local conditions of freight operation in Polish cities.
Potential for integration with standard transport analysis method currently used by cities.	None, dedicated surveys required.	None, dedicated surveys required.

presents a list of the data required for the models and explains whether the information is available in Poland and Gdynia. Included is also the possibility to integrate studies into the existing transport data collection system at the local level.

As we can see, Polish cities do not have the capacity to introduce the freight transport models. With a lack of primary data sources, cities would have to make far-reaching changes in how they analyse urban transport to be able to implement data collection methods. Selected problems (

Table 9. Summary of activities required to provide data for freight transport models in Gdynia.

Model	Required Activities	Comments
Freturb	- Development of classification of commercial activity types and identification of entities location within selected area - Development (or adaptation) of a comprehensive survey method for delivery structure analysis - Development of methodology for transport service organisation analysis (carrier surveys) - Implementation of the survey according to developed methodology	- Data available at local level lack required detail with regard to characteristics and location of commercial activity - Statistical analysis is required to identify the most relevant activity types in terms of their quantity - Receiver surveys and relatively manageable at local level with acceptable level of financial and organisational effort - Receiver survey results may also be used independently, e.g., to indicate location for dedicated delivery places - Main types of transport services may be identified with receiver surveys; it helps to indicate typical services for further analysis
Wiver	- Development of methodology for identification of logistics operators and carriers operating within city limits - Development of a survey method to obtain data from selected operators, including their type and size - Classification of the city into transport zones as transport destination areas, with regard to economic activity type, concentration and employment	- Classification of carriers and operators servicing a city’s area requires a well-developed database on entities with detailed information on vehicle fleet characteristics - Entities database should be analysed at regional level at least to identify all carriers located outside of a city, but operating within its limits - Adopting a regional approach for carrier identification makes a city dependent on national data sources, which have limited quality with regard to urban freight movements analysis - Zoning of a city according to general activity types and employment reduces the possibility for in-depth analysis of transport activity at local level or with regard to specific characteristics determining a demand for deliveries

) suggest that a Freturb-compliant data collection system would be the relatively easier option.

Having applied the method for supporting the choice of an urban freight transport model, we can now offer some recommendations based on the Gdynia analysis. At present, if they were to use the existing local transport data, local authorities could not apply any of the models. This is because existing solutions can only be adapted to the needs of passenger transport modelling. As a consequence, an entirely new system for a comprehensive analysis of urban freight transport would have to be built. Adding to the problem is the fact that both Gdynia and all other Polish cities do not have the practical experience of using external data.

It is safe to say that Freturb offers more flexibility in adapting to the requirements of sustainable urban freight transport. It is better at representing the factors that influence the behaviour of freight transport participants. If used as a tool for supporting urban transport policy, it helps with an in-depth analysis of the environmental impacts of transport. It is also good at representing deliveries and how they are organised (own transport/third party logistics). This important advantage is helpful with analysing complex measures designed to reduce the demand for transport. As regards Wiver, it is easy to integrate its origin-destination matrices with existing programmes for modelling urban transport. However, there is a barrier which is availability of comprehensive national data about commercial vehicle activity. If a city were to conduct a survey for its own purposes, it could face significant organisational and methodological issues, even if the work were to cover the immediate regional catchment area only.

6. Discussion

The purpose of the work was to create and verify a method for supporting local authorities with evaluating freight transport models to improve the delivery of sustainable transport policy. Using the method it is possible to apply a qualitative evaluation of the functionality of selected models in relation to urban freight objectives, characterise the necessary tools and assess data availability. The proposed solution addresses the fact that a quantitative comparison of selected freight transport models cannot be conducted for specific applications. Urban freight models require data that go beyond the standard practice of urban transport planning which focuses on passenger transport. As a result, city authorities struggle with the evaluations.

The method was verified against the case of Gdynia. The city already has experience of using passenger transport models giving it a level of competence which it needs, should it decide to use freight transport models. In addition, its sustainable urban mobility plan gives an outline of urban freight transport and the first actions have been rolled out (designation of dedicated delivery places).

The evaluation looked at two comprehensive and dedicated freight transport models (Freturb and Wiver) and an urban transport model as a point of reference. The analysis confirmed that should the city continue its efforts to improve freight transport, the urban transport model already in place does not ensure the basic functionality even though it has an extended multi-layer structure and a state-of-the-art solution. The urban model does not address properly any of the criteria related to the city’s strategic freight transport goals. It was not designed for that purpose. Conclusions from using the proposed method will help to avoid any wrong assessments of its potential.

As regards strategic objectives, dedicated models have shown similar potential functionality. They were then evaluated for how well they can meet the analytical requirements of improvement measures. The criteria were the same as those applied to strategic objectives. The method was found to be useful because it was very clear in identifying the differences between the models in question. Freturb was much more useful in how it handled the majority of solutions that could support the city’s sustainable transport policy in the area of freight. This goes back to the definition of the basic variable, i.e., delivery of goods. The parameters give a realistic representation of the structure of the freight transport process and user effects. This is key to modelling improvements and their effects on the strategic objectives of urban transport policy. While Wiver is potentially a solid tool for assessing how freight transport affects the use of infrastructure, the data it uses are too general and cannot give an accurate representation of a number of parameters which are specific to the activity of transport process users.

One of the basic issues of the new method was to compare models for their demand for data. This means two things: data availability in Gdynia as the place to be analysed and the steps to be taken to collect the data. The conclusions can also be referred to all cities across Poland because they follow the same patterns for transport data capture. None of the freight transport models in question can operate if they are based on primary data sources. Another problem is exact data to characterise receivers of goods such as industry and employment. Gdynia and Szczecin have been Poland’s only cities to have conducted some limited studies of delivery structures somewhat similar to work on Freturb.

As regards a new system for freight transport data capture, the French model seems much easier for local authorities. All the necessary information can be sourced within the city in question. Wiver requires complex studies of transport service providers and their activity. To that end, the analysis must cover at least the relevant region making it much more expensive and complex.

The method helps to bring together in a comprehensive approach urban freight transport issues and operational requirements of modelling tools. It supports a solid evaluation of model functionality where a quantitative comparison of model capacity is not possible due to a lack of input data. It can offer cities an understanding of the conditions for using freight transport models as tools to support the delivery of sustainable urban transport policy goals.