Next Article in Journal
International Students’ Conceptions of the Sustainable Internationalization of Business Education in Taiwan
Next Article in Special Issue
Impact of Property Tax on Housing-Market Disequilibrium in Different Regions: Evidence from Taiwan for the period 1982–2016
Previous Article in Journal
Enhancement of Material Properties of Lime-Activated Slag Mortar from Intensified Pozzolanic Reaction and Pore Filling Effect
Previous Article in Special Issue
Archaeological Site Conservation and Enhancement: An Economic Evaluation Model for the Selection of Investment Projects
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Impact of Public Management Approaches on Municipal Real Estate Management in Poland and The Netherlands

by
Bartłomiej Marona
1,* and
Annette Van den Beemt-Tjeerdsma
2
1
Department of Real Estate and Investment Economics, Cracow University of Economics, 31-510 Kraków, Poland
2
Department of Planning, Radboud University, 6525AJ Nijmegen, The Netherlands
*
Author to whom correspondence should be addressed.
Sustainability 2018, 10(11), 4291; https://doi.org/10.3390/su10114291
Submission received: 19 September 2018 / Revised: 14 November 2018 / Accepted: 15 November 2018 / Published: 20 November 2018
(This article belongs to the Special Issue Real Estate Economics, Management and Investments)

Abstract

:
Since the economic crisis (2008) municipalities became more aware of their real estate portfolio. Their first reaction to the sense of urgency to pay more attention to this extensive property was to improve their real estate administration. Now, ten years later, municipalities are ready to focus more on the professionalization of the management of their real estate. The purpose of this study is to present the role of individual concepts of public management in Polish and Dutch municipal (public) real estate management. The paper is based on the results on survey research based on public real estate management theory and two public management approaches: new public management and good governance. First, preliminary research was carried out in a Polish metropolitan area after which all Polish metropolitan areas where questioned about their real estate management issues. This questionnaire was also sent to all Dutch municipalities one year later. The Hellwig’s taxonomic method was performed on both separately to assess the level of implementation of good governance and new public management principles in real estate management practices. The research shows that new public management standards are applied at a similar level in municipal real estate management in Poland and The Netherlands. Good governance standards are used a little more broadly in Poland than in The Netherlands. The research shows that in Poland and The Netherlands the concepts of new public management and good governance are not applied as a whole but are deployed as a collection of instruments. Most municipalities choose some of these instruments to apply to their municipal real estate (MREM). Both in Poland and in The Netherlands there are differences noticed in the application of new public management and good governance principles depending on the type of municipality. Besides this originality and scientific relevance, municipalities of both countries could benefit from this comparison by learning from best practices. Practical recommendations and suggestions for public administration concern: (i) the necessity to develop municipal real estate management plans; (ii) increase regularity of asset valuation and (iii) assessment of real estate management performance; (iv) greater transparency in real estate management; and (v) increasing the participation of citizens in the process of managing real estate.

1. Introduction

Due to its size, real estate owned by municipalities is an important segment of the real estate market [1]. Yet it received little attention in society as well as in science until recently. Due to the economic crisis that started in 2008, municipalities felt the sense of urgency to take the management of their real estate portfolio more seriously. First, their focus was mainly economically on how they could cut costs. This question made the municipalities realize there was much work to do on their administration of their real estate and related themes first. Now, ten years later, municipalities are ready for the next step: focus shifts from administrative to more managerial questions. Of course, a lot of research is done about real estate management in profit organizations. Especially in the field of Corporate Real Estate Management (CREM). However, this present research is about public non-profit organizations. This and the political context makes real estate management in a municipal context different from real estate management in the for-profit sector.
CREM literature is about real estate management at both private and public organizations that are not primarily in the real estate business [2]. Despite the fact that CREM is about both public and private, the concept is based mostly on research in the private sector [3,4]. Therefore, it not always fits public sector specifics, public regulations and public values. In this context, very valuable and interesting is Public Real Estate Management (PREM) research, which is based not only on CREM theory but also on the public management approach. Literature review (see Table 1) clearly indicates that there is little research that tries to merge the real estate management perspective with public management concepts, more specifically the New Public Management (NPM) and the Good Governance (GG) approaches as a part of the Public Governance (PG) concept.
Therefore, we attempted to fill the gap in PREM literature and present the role (main purpose of the paper) of NPM and GG in Polish and Dutch MREM. In doing so, this research regards to the persons working in different organizational units of the municipality whose duties and responsibilities are related to real estate management. Additionally, an extra value comes from comparing two countries instead of only describing the status quo at one country. Other researchers recommended comparisons with Polish municipalities and other countries as well [5] and many scholars underline that it is particularly interesting to make MREM comparisons between countries with different institutional and historical public administration backgrounds [6,7]. Therefore, we decided to compare Poland and the Netherlands which have different starting points in public government reforms but both of them began implementing changes in 20th century [8,9].
Therefore, our fist hypothesis is:
1)
The role of NPM and GG concepts in MREM is similar in Polish and Dutch municipalities.
The second hypothesis is based on a previous study [10]:
2)
Both in Poland and in The Netherlands, there are differences within the scope of the application of NPM and GG concepts in MREM depending on the size of the municipality.
Research on real estate management at municipalities based on the principles of NPM and GG comparing two countries and more specifically Poland and The Netherlands, has never been done before. Besides this originality and scientific relevance, municipalities of both countries could benefit from this comparison by learning from best practices. Our practical suggestions for public administration come directly from the survey research which apart from the concepts of GG and NPM, are based on the achievements of Public Real Estate literature [6,7,10,11,12,13,14,15,16,17,18,19,20].
The paper structure is as follows: the next paragraph provides a literature review of PREM literature from an international perspective. Section 3 specifies the dataset and method of the research. Section 4 presents the results and discussion and finally Section 5 provides conclusions and final remarks.

2. International Research on Public Real Estate Management

In 1993, Simons concluded: “corporate real estate assets are generally undermanaged” [11] and in 2000 it was still noticed that PREM lag far behind on the private sector [21]. They also conclude that “the opportunity cost of not managing public real estate strategically is large and has implications for local budgets and services” [21]. Nowadays, both public and private organizations pay more attention to real estate and management [22]. The financial crises (70–80s and 2008) created a sense of urgency and still it is considered to be an asset that deserves more attention than it got before. With this increase of attention, first care did go to real estate itself. Especially public organizations put a lot of effort in making an inventory and managing each building individually. Subsequently, focus shifted to the context of the whole portfolio and now we have arrived at the point that the organization and management are to be developed.
Most public and private organizations do not own real estate to make a profit out of it. In PREM, as in CREM [23], primary focus is not on financial interest but on facilitating primary processes. When it comes to governmental organizations, this means that real estate has to facilitate this organization and also the realization of policy goals [20]. Van der Schaaf defines PREM as “…the management of a government’s real estate portfolio by aligning the portfolio and services to (1) the needs of the users, (2) the financial policy set by the Treasury and (3) the political goals that government wants to achieve” [24]. In this, PREM is not about management of the real estate portfolio itself. It is about aligning real estate with the trinity of function, finance and politics of which the latter is closely related to society. Van den Beemt also found that most actions taken and results achieved are not about the real estate itself [20]. Most interventions that improved the operations of real estate management were taken at organizational level [20]. With the organizational level and the skill to align the interest of the different stakeholders as a starting point, PREM asks more for a management approach than for a real estate focus.
Also, internationally and scientifically the subject gained popularity. It starts to be recognized by international researchers around the world [19,21,25,26,27]. Due to the main objective of the paper the focus of our review was on MREM studies which directly refer to the public management paradigm (see Table 1).
An interesting example of using the GG concept is a guide prepared in 2007 as a part of the Food and Agriculture Organization (FAO) Regular Program work in land tenure and land administration [28]. According to this report [28] there are six features of GG in land tenure and administration [10]:
1)
“The legitimacy of land agencies and land administrators is widely recognized by citizens.
2)
Land agencies serve all citizens, including the weak as well as the strong.
3)
Land agencies provide services that respond to the needs of their customers, for example, in the nature of the services and accessibility to them.
4)
The results of the services are consistent, predictable and impartial.
5)
The services are provided efficiently, effectively and competently.
6)
The services are provided with integrity, transparency and accountability”.
The study by Grosss and Źróbek [18] used a modified Delphi method and employed five basic GG principles established by the Commission of the European Communities (openness, participation, accountability, effectiveness and coherence) [29]. With these principles they evaluate real estate management in different countries (mostly European ones). Also, trends are shown. Another example of using the GG concept is a Marona’s study from 2016 [10]. In this research a survey was sent to a single metropolitan area to research how GG and NPM are implemented in Municipal Real Estate Management (MREM). Marona found that most municipalities choose some of the GG and NPM instruments to apply to their municipal real estate but not the concepts as a whole [10]. In this context, the research of Mardiasmo and Sampford [30] is very interesting. This study looked at the level in which GG principles are applied within state asset management laws. They also examine state asset management regulations in three Indonesian regional governments and concluded that for different GG principles levels of understanding and implementation differ [30]. On the one hand there is a high level of understanding and implementation in (i) transparency, (ii) accountability and (ii) compliance principles. On the other hand, there is only moderate understanding and implementation in (iv) efficiency and (v) stakeholder participation principles [30]. Other Indonesian research on state asset management shows that there is a strong link between the reform, the adoption of GG practice and thoughts and practices of government officials [31].
As was mentioned before, many authors apart from governance concepts use an NPM approach in the PREM research. In some research, despite of the NPM critics [32,33], many researchers find the NPM concept an opportunity for improving state and MREM [34]. In other research [10] there is the assumption that despite the many controversial issues concerning NPM, a lot of standards are visible in the practice of public management and MREM. In recent work by Constantin et al. [7] the MREM is treated as a major component of NPM. As research shows, NPM has affected real estate management [35]. Real estate management joined the NPM movement that is the crucial driving force for efficient and effective public asset management [35].
We can conclude that there is little research that tries to merge the real estate management perspective with public management concepts, more specifically the NPM and GG. We can also conclude that there is no PREM research, which takes into consideration the model that is most discussed nowadays (especially in Europe): the Neo-Weberian State model introduced by Pollitt and Bouckaert in 2004 [40,41]. As also can be confirmed from Table 1, no research of this size and scope as the research presented in this paper was done before. Mostly, research was focused on one country or when including more countries, the scope was narrower. Besides, in most cases the public management framework of MREM research is very general. Therefore, this research adds to all previous research.

3. Materials and Methods

First, a preliminary survey research was conducted in one Polish metropolitan area in 2016 [10] and the questionnaire was based on theories of NPM and GG. The operationalization of the research took place based on 7 NPM standards [42] and 5 GG principles according to the Commission of the European Communities [29] (see Table 2).
In 2017, the survey questionnaire was considerably modified. The list of questions was extended from 18 to 33 detailed issues (see Appendix A) referring to MREM, from which we used 22 in our research (see Table 2).
In comparison with the 2016 research, also the scope of the research was substantially extended. It was carried out in all 304 Polish municipalities belonging to all metropolitan areas in Poland. The research was conducted in May 2017 with the use of traditional post. The questionnaire was completed and returned by 115 municipalities in total (return rate of 38%). In all cases the persons completing the survey questionnaires dealt directly with real estate management and often hold managerial positions (head of a department, director of a department, a section supervisor, possibly a head of municipality). In the case of biggest municipalities, the survey questionnaire was filled in by two persons working in different organizational units of the municipality whose duties and responsibilities are, however, related to real estate management.
This questionnaire was also sent to all Dutch municipalities one year later (return rate of 22%). All respondents have real estate related work but are not all in managerial positions. The questionnaire designed for the research in Poland was translated from Polish to Dutch in order to make it possible for Dutch municipalities to answer the questions. To ensure validity, checks were done by both authors by describing the meaning of the question and providing options for translation. In this way, not only translating from the one language to the other was double checked but also the essence, meaning and nuances of the questions were looked after. Finally, it was checked by a Dutch professor too. The only adjustments made to the order of questions was that the questions about the respondent (name of municipality and function title) were moved to the back of the questionnaire. This intervention was done in order to increase response. Years of survey experience in The Netherlands learned that starting off with these questions makes Dutch municipalities reluctant to continue as they might worry about their anonymity because they do not yet know which questions are to be asked. On 20 March (2018) the questionnaire was sent by e-mail to all 380 municipalities and they were reminded of the questionnaire once. The questionnaire was closed approximately three months later on June 18th. An online questionnaire tool (Enalyzer) was used to collect the data. Once the data were collected, the raw data were put in a dataset that fits the set of the previous research in Poland. In this way, we created one dataset with the results of both countries. This contributes to a reliable, clear and flawless analysis.
All statistical analyses were done in Microsoft Excel and STATA 15. For each country the Hellwig’s taxonomic method was applied using a similar procedure to Pomianek and Chrzanowska [43] and Kasztelan [44]. This method is like the ‘Technique for Order of Preference by Similarity’ [45] which is based on a concept of similarity to an ideal solution. However, originally, Hellwig’s taxonomic method was developed a few decades before (in 1968) as a taxonomic method for international comparisons of economic development of countries [46]. Hellwig’s method allows us to group municipalities into homogenous classes, based on the level of implementation of GG and NPM principles in the practice of MREM in Poland and the Netherlands. The Hellwig’s synthetic measure was calculated in the context of 6 out of 7 NPM standards [42] and 4 out of 5 GG principles according Commission of the European Communities [29]. See Table 2.
The procedure includes:
(i)
normalization of initial diagnostic variables (xij), where i = 1, …, n is the number of objects (municipalities) and j = 1, …, m is the number of variables.
(ii)
calculation of group means of normalized diagnostic variables (zig), where g = 1, …, k is the number of evaluated NPM standards or GG principles (to which the initial diagnostic variables correspond). In the research we evaluated implementation of 6 NPM standards (NPM1, NPM2, NPM3, NPM5, NPM6, NPM7) and 4 evaluated GG principles (GG1, GG2, GG3, GG4), thus 6 or 4 group means were calculated respectively. The reason for calculating group means is an unbalanced number of diagnostic variables corresponding with selected NPM standards or GG principles.
(iii)
construction of the object (pattern) with the highest values of group means of normalized diagnostic variables corresponding to evaluated NPM standards or GG principles (z0g).
(iv)
calculation of the squared Euclidean distance (di) of each object from the constructed pattern:
d i   =   1 k g   =   1 k ( z i g z 0 g ) 2
(v)
calculation of synthetic measures hi that describes the implementation of NPM standards or GG principles by municipalities in Poland and Netherlands:
h i   =   1 d i d i ¯ + 2 S d i
where d i ¯ is arithmetic mean of calculated distances di and S d i is a standard deviation of di. The hi usually have values between 0 and 1 and the closer hi value is to 1 the more similar the object is to the pattern [47].
Hellwig’s taxonomic method was applied in order to fulfil the objective of this research and to test both hypotheses. To reflect the level of implementation of GG or NPM, four classes (see Table 3) were identified based on arithmetic mean of synthetic measures ( h i ¯ ) and its standard deviation ( S h i ) .

4. Results and Discussion

Results should show if the role of NPM and GG concepts in MREM is similar in Polish and Dutch municipalities and whether differences within the scope of the application of these concepts depend on the size of the municipality between these two countries. Therefore, first the NPM results will be discussed in total and for the different sizes. Next, the same was done for the GG concepts.

4.1. New Public Management

As it results from the research both in Poland and in The Netherlands, NPM standards are applied at a similar level in MREM (Hellwig’s synthetic measure is 0.329 for Poland and 0.326 for Netherlands—see Table 4). NPM principles are applied to the fullest in the context of the first of seven Hood’s [42] standards, namely: “hands on professional management in the public sector but Both in Poland and in The Netherlands the majority of employees taking part in MREM receive cyclical training (85% for PL and 91% for NL). In the majority of cases (59% for PL and 78% for NL) the employees of the municipalities working in real estate management have enjoyed real estate education (e.g., real estate management, real estate economics). Moreover, what results show is that most employees of the municipalities engaged in real estate management related tasks have experience in the private sector (53% for PL and 64% for NL).
To a minimum extent both in Poland and in The Netherlands those principles are used which Hood called “explicit standards and measure of performance”, “greater emphasis on output control” and “stress on private sector style of management practice” [42]. Nourse & Roulac [48] think that (corporate) real estate management is about the effectiveness of the real estate function. However, results show that the majority of municipalities in both countries do not evaluate effectiveness of management of their real estate at all. What can be observed is that there are no developed indicators that enable evaluating effectiveness of MREM (19% in PL and 43% in NL). Additionally, in Poland only 49% of the municipalities update the real estate value regularly (67% in NL) what should be a good standard in all municipalities [13] taking into consideration demographic changes [49] and using spatial property information [50]. Besides, very few municipalities (11% in PL and 20% in NL) make a comparative analysis with other municipalities of their MREM results. International research shows that very often the reason is lack of reliable data about public properties [21]. For this reason, making a reliable evaluation of the results with a ratio analysis is very hard due to interpretation difficulties. Moreover, it is tough to conduct effective evaluations of real estate management when as many as 45% of the studied municipalities in Poland and 47% in The Netherlands declared that they do not have MREM plans. At the same time, these problems with MREM are similar to those which are observed in other European countries [7].
In the context of the principle “stress on private sector style of management practice” [42] payment and motivation systems in municipal offices were investigated, which is also one of a key component of effective real estate management [17]. Results show that most municipalities do not differentiate the salaries of employees at equivalent positions in the organizational unit responsible for real estate management (only 26% in PL and 18% in NL). 27% of respondents in Poland and only 5% in The Netherlands indicated that bonuses of employees related to MREM depend on the level of accomplished goals.
The greatest differences between both countries were noticed in the context of the principle which Hood called a “shift to greater competition in public sector” [42]. In this context, a question was asked referring to the extent of the use of outsourcing in real estate management. Since the 1980s, government policy has increasingly focused on outsourcing public services to private organizations [51]. Nevertheless, property management is only outsourced to a limited extend [52]. The research findings show that Dutch municipalities more often decide to use outsourcing solutions than municipalities in Poland (16% and 62%, respectively) but also municipalities in other European countries [53].
The seventh, last principle of NPM described by Hood is pressure on raising discipline in the use of resources (“stress on greater discipline and parsimony in resource use”) [42]. In this context, the studied municipalities generally positively assess the manner of managing their resources. Respondents assessed the use of both (i) land and (ii) buildings and municipal premises, considering the number of undeveloped real properties and vacant properties. Both categories were assessed positively, generating 70% and 88% positive answers in Poland and 50% and 46% in The Netherlands (over 40% municipalities in The Netherlands assessed it in neutral way).
It should be stressed that both in Poland and in The Netherlands, there are differences noticed in the application of NPM principles depending on the type of municipality. In Poland, urban municipalities are characterized by the largest value of the synthetic measure of NPM (0.453). The synthetic measure value in urban-rural municipalities as well as in rural municipalities is similar (0.295 and 0.309, respectively). The situation was similar in The Netherlands, where the synthetic measure value is the lowest in small (0.257) and medium (0.317) municipalities.
What is important is that the analysis of variance shows that the differences between types of municipalities are statistically significant only in Poland (see Table 5).
According to the values of the arithmetic means, municipalities were assigned to one of the four classes (groups) with regard to their level of fulfilment of 7 NPM standards and 5 GG principles. Class 1 consists of municipalities with a very low level of the fulfilment of NPM, class 4 is with the highest level of standard fulfilments (see Table 6).
The analysis of the results included in Table 6 shows that only 13% municipalities in Poland and 18% of municipalities in Netherlands are in Class 4. Therefore, we can claim that in most Polish and Dutch municipalities NPM principles in the real estate management practice are not used much. As far as the scope of the application of NPM principles is concerned, there are differences noticed in both countries depending on the size of municipality. However, as it was emphasized before, only in Poland these differences are statistically significant.

4.2. Good Governance

As for the GG concept, the study focused around four principles, namely effectiveness, openness, participation and accountability. What results from the research is that GG standards in MREM are used a little more broadly in Poland than in The Netherlands (Hellwig’s synthetic measure is 0.4218 for Poland and 0.3265 for The Netherlands—see Table 7), which may be surprising in the context of Gross and Źróbek’s research [18]. At the same time, it should be observed that in the case of Poland there is a higher GG measure than in the case of NPM standards. In case of The Netherlands the measure is almost identical (see Table 4).
The synthetic measure values (Hellwig’s synthetic measure) in Poland and in The Netherlands differ depending on the type of municipality. However, the variance analysis shows again that the differences are only statistically significant for Poland (see Table 8), where the application of the GG concept is substantially higher in urban municipalities than in other municipalities (See Table 7).
The data included in Table 9 show that nearly 18% municipalities in Poland and 12% municipalities in The Netherlands belong to Class 4. Therefore, we can claim that in most cases GG concepts are not used widely in Polish and Dutch MREM. As is the case with NPM standards, differences are observed in both countries in the scope of the application of GG principles depending on the municipality size. These differences are only statistically significant for Poland where there is no universal model of MREM [39].
The higher value of the synthetic measure for Poland (Table 7) to a great extent arises from a greater scope of the application of two GG principles referring to openness and participation in Poland than in The Netherlands. In the case of openness, 60% municipalities in Poland and only 10% of Dutch municipalities declared that there are full details of the current municipality’s properties on their websites. The question about making public the details of the planned investments in municipal properties (for instance modernization, repairs etc.) shows that 62% in Poland and only 14% in The Netherlands announce some or all of this type of information by means of a website. When it comes to making public (annual) accomplishment reports in the context of MREM, results show that only 27% of Polish municipalities publish this type of online elaboration but even fewer (10%) do it in The Netherlands. Another question was about whether citizens of the municipality have been able to express themselves in a structured manner (internet survey, meeting) on selected aspects of MREM (e.g., the way of development of a park, the colour of the elevation of a school, etc.). 48% municipalities in Poland and 30% municipalities in The Netherlands declared that over the last two years there was at least one such an opportunity (exactly the same number of municipalities gave the negative answer). Generally, mainly due to a lower level of fulfilment of the openness and participation principles, the synthetic GG measure is greater for Poland than for The Netherlands.
In the context of accountability, 79% of municipalities in Poland and 58% in The Netherlands declared that MREM departments are evaluated internally. In 67% of municipalities in Poland and 75% in The Netherlands the level of MREM achievements is discussed during staff meetings. In 53% of the municipalities in Poland and 41% of the municipalities in The Netherlands organizational units responsible for MREM prepare annual accomplishment reports.
The last group of questions was related to the operationalization of the principle of effectiveness which was met in both countries to a medium level. In this context it should be emphasized that only 32% of municipalities in Poland and 42% municipalities in The Netherlands evaluate the effectiveness of managing their real estate in general. In the context of Kaganova and Nayyar-Stone’ postulate [21] to ‘periodically evaluate financial performance of each property and the whole portfolios in each class’, this result are not satisfactory. On the other hand, the majority of the studied municipalities (85% in Poland and 52% in The Netherlands) declare punctuality of the implemented real estate management tasks. The majority of municipalities in both countries also declare that all planned tasks are accomplished for a given budget year.
The findings of this study should be considered along with its limitations. Due to difficulties with operationalization of all NPM and GG standards in survey research, not all of them were evaluated. Secondly, the study is quantitative in nature and future research could be based on qualitative research using case study method based on some selected urban, urban-rural and rural municipalities from Poland and large, medium and small municipalities from The Netherlands. Future research could concentrate also on MREM in other countries than Poland and The Netherlands. In further research, it can also be interesting to add other sources about NPM and GG, like Gruening [54], Noto & Bianchi [55] and the Council of Europe [56]. By adding more perspectives on NPM and GG, the questionnaire can be modified and it can contribute to the refinement of the operationalization of the NPM and GG principles. Another interesting direction for future research would be to examine public management principles and concepts (GG, NPM and others) and their impact on MREM efficiency. Moreover, there is a possibility to extend the research taking under consideration not only type of municipalities but also type of municipal real estate assets managed by local government units.

5. Conclusions

In the article, we investigated the problem of MREM in the context of public management theory. Research on real estate management based on the principles of NPM and GG comparing two countries and more specifically Poland and The Netherlands has never been done before.
We can conclude that the first hypothesis was only partially confirmed. The research shows that in Poland and The Netherlands the concepts of NPM and GG are not applied as a whole but are deployed as a collection of instruments. Most municipalities choose some of these instruments to apply to their municipal real estate (MREM). However, results also suggest that GG standards in Polish MREM are used a little more broadly than in The Netherlands. As it comes to NPM principles, both in Poland and in The Netherlands, NPM standards are applied in a similar scope in MREM.
Furthermore, in reference to our second research hypothesis, it should be stressed that both in Poland and in The Netherlands there are differences noticed in the application of NPM and GG principles depending on the type of municipality. In larger municipalities, both concepts are applied more often than in other municipalities but variance analysis tells that the differences are only statistically significant for Poland.
Besides this originality and scientific relevance, municipalities of both countries could benefit from this comparison by learning from best practices. Practical recommendations and suggestions for public administration concern: (i) the necessity to develop municipal real estate management plans, (ii) increase regularity of asset valuation and (iii) assessment of real estate management performance, (iv) greater transparency in real estate management and (v) increasing the participation of citizens in the process of managing real estate.

6. Patents

The contribution share of authors is equal and amounted to 50% each of them. The conceptualization comes from Marona’s previous research [10]. Preliminary results were presented during the European Real Estate Society (ERES) conference in Reading, 2018.

Author Contributions

Conceptualization, B.M.; Methodology, B.M.; Software, B.M. and A.v.d.B.-T.; Validation, A.v.d.B.-T.; Investigation B.M. and A.v.d.B.-T.; Data curation B.M. and A.v.d.B.-T.; Writing—original draft preparation, B.M. and A.v.d.B.-T.; Writing—review and editing, A.v.d.B.-T.; Supervision, B.M. and A.v.d.B.-T.; Project administration, B.M. and A.v.d.B.-T.

Funding

This research was funded by Stichting Hanzehogeschool and Faculty of Economics and International Relations of the Cracow University of Economics.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

This appendix contains the questionnaire sent to Polish and Dutch municipalities. The municipalities received this questionnaire in their own language (not English).
Table A1. Questionnaire.
Table A1. Questionnaire.
Question#VariableQuestionValues
1 *X1Has your municipality a real estate (asset) management plan?0 (No)/1(Yes)
2 *X2Assesses your municipality the efficiency of real estate management?0 (No)/1(Yes)
3. Select the most appropriate answer for each question
3.1 *X3Does your municipality offer training opportunities for the real estate management team/staff?1 (Absolutely not)
2 (Probably not)
4 (Probably yes)
5 (Absolutely yes)
3 (Hard to say)
3.2 *X4Do the employees involved in real estate management have a real estate (related) education/diploma?
3.3 *X5Has your real estate management staff corporate (private) real estate management experiences?
3.4 *X6Do the salaries of real estate management employees differ in comparable positions?
3.5 *X7Do bonuses of real estate management employees depend on the results they achieve (extent to which the objectives are achieved)?
3.6 *X8Is real estate management internally evaluated/assessed at your municipality?
3.7 *X9Are annual reports prepared on the performance of real estate management at your municipality?
3.8 *X10Are the annual goals from the real estate management plan realized at your municipality?
3.9 *X11Will the annual goals from the real estate management plan be realized in time at your municipality?
3.10 *X12Are real estate management performances discussed during (internal) meetings at your municipality?
3.11 *X13Are your municipal real estate management performances/results compared with those of other municipalities?
3.12 *X14Is full information about municipal real estate available on your website?
3.13 *X15Is full information about planned municipal real estate investments (e.g., upgrades, renovations, etc.) available on your website?
3.14 *X16Are the annual reports of real estate management (or other internal reports) published on your municipality’s website?
3.15 *X17Have the citizens of your municipality been given the opportunity to express themselves in an organized way (internet poll, meeting) about certain aspects of municipal real estate management (e.g., the management of the city park, the colour of the school’s, etc.) in the past two years?
3.16 *X18Does your municipality outsource real estate management tasks?
3.17X19To what extent are you familiar with the concept of New Public Management?
3.18X20To what extent are you familiar with the concept of Good Governance?
3.19 *X21Does your municipality have indicators for assessing the effectiveness of municipal real estate management?
3.20 *X22Does your municipality systematically update the value of the municipal real estate (owned)?
4. Assess the following specific categories related to municipal real estate management in your municipality?
4.1X23Collecting rent (housing)1 (Very bad)
2 (Bad)
3 (Average)
4 (Good)
5 (Very good)
0 (Does not apply)
4.2X24Collecting rent (other real estate)
4.3X25Collecting fees for (perpetual) usufruct
4.4X26Collecting emphyteutic lease fees
4.5X27Collecting other real estate claims
4.6X28Legal status (ownership situation) of your real estate
4.7X29Technical condition of the real estate owned by the municipality
4.8 *X30The use of the municipal land, considering undeveloped plots
4.9 *X31The use of municipal real estate, considering occupancy rates/vacancy
4.10X32Sufficiency of real estate owned (size, condition, etc.) in relation to the tasks of the municipality
4.11X33Sufficiency of financial resources for real estate management needed to carry out the tasks (repairs, expert advice, software, etc.)
* Used in this research.

References

  1. Deloitte Real Estate Group. Comparing Public Real Estate Management in European Cities. Mondaq. 8 December 2011. Available online: http://www.mondaq.com/uk/x/156662/Market+Commentaries/Comparing+Public+Real+Estate+Management+In+European+Cities (accessed on 16 October 2018).
  2. Bon, R. Corporate real estate management. Facilities 1992, 10, 13–17. [Google Scholar] [CrossRef]
  3. Manning, C.; Roulack, S.E. Lessons from the past and future direction for corporate real estate research. J. Real Estate Res. 2001, 22, 7–57. [Google Scholar] [CrossRef]
  4. Abatecola, G.; Caputo, A.; Mari, M.; Poggesi, S. Real estate management: Past, present and future research directions. Int. J. Glob. Small Bus. 2013, 5, 98–113. [Google Scholar] [CrossRef]
  5. Rymarzak, M.; Trojanowski, D. Asset management in public sector in Poland. Real Estate Manag. Valuat. 2013, 21, 5–16. [Google Scholar] [CrossRef]
  6. Phelps, A. Municipal property asset management—A comparative study of UK and Russia. Int. J. Strateg. Prop. Manag. 2011, 15, 416–437. [Google Scholar] [CrossRef]
  7. Constantin, D.L.; Mitrut, C.; Grosu, R.M.; Profiroiu, M.; Iosif, A.E. Municipal real properties and challenges of new public management: A spotlight on Romania. Int. Rev. Adm. Sci. 2018, 84, 122–143. [Google Scholar] [CrossRef]
  8. Hendriks, F.; Tops, P. Local public management reforms in The Netherlands: Fads, fashions and winds of change. Public Adm. 2003, 81, 301–321. [Google Scholar] [CrossRef]
  9. Regulski, J. Local Government Reform in Poland: An Insider’s Story; Open Society Institute: Budapest, Hungary, 2003; ISBN 963 9419 68 0. [Google Scholar]
  10. Marona, B. Public management in real estate area—Some empirical evidence from Polish municipalities. Real Estate Manag. Valuat. 2016, 24, 16–22. [Google Scholar] [CrossRef]
  11. Simons, R.A. Public real estate management—Adapting corporate practice to the public sector: The experience in Cleveland, Ohio. J. Real Estate Res. 1993, 8, 639–654. [Google Scholar]
  12. Simons, R.A. Public real estate management and the planner’s role. J. Am. Plan. Assoc. 1994, 60, 333–343. [Google Scholar] [CrossRef]
  13. Dent, P. Managing public sector property assets: The valuation issues. Prop. Manag. 1997, 15, 226–233. [Google Scholar] [CrossRef]
  14. Kaganova, O.; Nayyar-Stone, R.; Peterson, G. Municipal Real Property Asset Management: An Application of Private Sector Practices; Land and Real Estate Initiative, Background Series 12, Urban & Local Government; The World Bank: Washington, DC, USA, 2000. [Google Scholar]
  15. Kaganova, O.; McKellar, J. Managing Government Property Assets, International Experiences; Urban Institute Press: Washington, DC, USA, 2006; ISBN 978-0877667308. [Google Scholar]
  16. Vermiglio, C. Public property management in Italian municipalities: Framework, current issues and viable solutions. Prop. Manag. 2011, 29, 423–442. [Google Scholar] [CrossRef]
  17. Abdullah, S.; Nah, M.N.M.; Razak, A.A. Real Estate Management in Malaysian Public Schools: Determination of Success Factors. Real Estate Manag. Valuat. 2015, 23, 52–67. [Google Scholar] [CrossRef] [Green Version]
  18. Gross, M.; Źróbek, R. Good governance in some public real estate management systems. Land Use Policy 2015, 49, 352–364. [Google Scholar] [CrossRef]
  19. Riratanaphong, C.; Van der Voordt, T.J.M. Public Real Estate Performance Measurement: A case study of a Bangkok Government Complex. J. Appl. Remote Sens. 2015, 12, 135–148. [Google Scholar]
  20. van den Beemt-Tjeerdsma, A.; Veuger, J. Towards a more professionalised municipal real estate management. J. Corp. Real Estate 2016, 18, 132–144. [Google Scholar] [CrossRef]
  21. Kaganova, O.; Nayyar-Stone, R. Municipal Real Property Asset Management: An Overview of World Experience, Trends and Financial Implications. J. Real Estate Portf. Manag. 2000, 6, 307–326. [Google Scholar]
  22. Evers, E.; Van der Schaaf, P.; Dewulf, G. Public Real Estate: Successful Management Strategies; IOS Press: Amsterdam, The Netherlands, 2002; ISBN 978-9040722974. [Google Scholar]
  23. Brown, K.; Arnold, A.L. Managing Corporate Real Estate; John Wiley & Sons: New York, NY, USA, 1993; ISBN 978-0471554974. [Google Scholar]
  24. van der Schaaf, P. Public Real Estate Management Challenges for Government: An International Comparison of Public Real Estate Strategies. Ph.D. Dissertation, Delft University of Technology, Delft, The Netherlands, 2002. [Google Scholar]
  25. Kaganova, O.; Akmatov, A.; Undeland, C. Introducing more transparent and efficient land management in post-socialist cities: Lessons from Kyrgyzstan. Int. J. Strateg. Prop. Manag. 2008, 12, 161–181. [Google Scholar] [CrossRef]
  26. Abdullah, S.; Razak, A.A.; Pakir, A.H.K. The characteristic of real estate assets management practice in the Malaysian Federal Government. J. Corp. Real Estate 2011, 13, 16–35. [Google Scholar] [CrossRef]
  27. Klumbyte, E.; Apanaviciene, R. Real estate strategic management model for Lithuanian municipalities. Int. J. Strateg. Prop. Manag. 2014, 18, 279–291. [Google Scholar] [CrossRef]
  28. Food and Agriculture Organization of the United Nations. Good Governance in Land Tenure and Administration; Land Tenure Studies: Rome, Italy, 2007; ISBN 978-92-5-105753-7. [Google Scholar]
  29. European Commission. European Governance—A White Paper. Commission of the European Communities 2001. Available online: http://europa.eu/rapid/press-release_DOC-01-10_en.htm (accessed on 18 September 2018).
  30. Mardiasmo, D.; Sampford, C. Is Good Governance Conceptualised in Indonesia’s State Asset Management Laws? In Engineering Asset Management—Systems, Professional Practices and Certification, 1st ed.; Tse, P., Mathew, J., Wong, K., Lam, R., Ko, C., Eds.; Springer International Publishing: Cham, Switzerland, 2015; pp. 1157–1171. ISBN 978-3-319-09506-6. [Google Scholar]
  31. Prayoga, N.E. Linkage between Good Governance and State Asset Management Reform in Indonesia. Ph.D. Thesis, University of Canberra, Canberra, Australia, 2017. [Google Scholar]
  32. Margetts, H.; Bastow, S.; Tinkler, J. New Public Management Is Dead—Long Live Digital—Era Governance. J. Public Adm. Theory 2006, 16, 467–494. [Google Scholar] [CrossRef]
  33. Haque, M.S. Revisiting the New Public Management. Public Adm. Rev. 2007, 67, 179–182. [Google Scholar] [CrossRef]
  34. Pozega, Z.; Crnkovic, B.; Zivkovic, A. Effective management of state property as prerequisites for economic development. Econ. East. Croat. Yesterday Today Tomorrow 2014, 3, 418–426. [Google Scholar]
  35. Lu, Y. New Public Management reforms in public asset management as the state governments of the United States. In Proceedings of the International Conference on Public Administration, Sichuan, China, 24–26 September 2011. [Google Scholar]
  36. Schulte, K.W.; Ecke, C. Public real estate management in Germany: An empirical study. In Managing Government Property Assets: International Experiences; Kaganova, O., Mckellar, J., Eds.; The Urban Institute Press: Washington, DC, USA, 2006; ISBN 9780877667308. [Google Scholar]
  37. Gross, M.; Źróbek, R.; Špirková, D. Public real estate management system in the procedural approach—A case study of Poland and Slovakia. Real Estate Manag. Valuat. 2014, 22, 63–72. [Google Scholar] [CrossRef]
  38. Kask, K. Public Sector Real Estate Asset Management Models and Their Evaluation. Ph.D. Dissertation, University of Tartu, Tartu, Estonia, 2014. [Google Scholar]
  39. Lausberg, C.; Wojewnik-Filipkowska, A. Decision-making in the European Municipal Real Estate Management—A Case Study Approach. Word Real Estate J. 2017, 102, 23–30. [Google Scholar] [CrossRef]
  40. Pollitt, C.; Bouckaert, G. Public Management Reform: A Comparative Analysis—New Public Management, Governance, and the Neo-Weberian State, 2nd ed.; Oxford University Press: Oxford, UK, 2004; ISBN 978-0199268498. [Google Scholar]
  41. Byrkjeflot, H.; du Gay, P.; Greve, C. What is the ‘Neo-Weberian State’ as a Regime of Public Administration? In The Palgrave Handbook of Public Administration and Management in Europe, 1st ed.; Ongaro, E., van Thiel, S., Eds.; Palgrave Macmillan: London, UK, 2017; pp. 991–1009. ISBN 978-1-137-55268-6. [Google Scholar]
  42. Hood, C. A public management for all seasons? Public Adm. 1991, 69, 3–19. [Google Scholar] [CrossRef]
  43. Pomianek, I.; Chrzanowska, M. A spatial comparison of semi-urban and rural gminas in Poland in terms of their level of socio-economic development using Hellwig’s method. Bull. Geogr. Socio-Econ. Ser. 2016, 33, 103–117. [Google Scholar] [CrossRef] [Green Version]
  44. Kasztelan, A. The use of the Hellwig’s pattern model for the evaluation of green growth in OECD countries. In Proceedings of the 29th International-Business-Information-Management-Association Conference, Vienna, Austria, 3–4 May 2017. [Google Scholar]
  45. Hwang, C.L.; Yoon, K. Multiple Attribute Decision Making: Methods and Applications a State-of-the-Art Survey; Springer: Berlin, Germany, 1981; ISBN 978-3-540-10558-9. [Google Scholar]
  46. Balcerzak, A.P. Multiple-criteria evaluation of quality of human capital in the European Union countries. Econ. Sociol. 2016, 9, 11–26. [Google Scholar] [CrossRef] [PubMed]
  47. Wawrzyniak, D. Standard of living in the European Union. Comp. Econ. Res. 2016, 19, 141–155. [Google Scholar] [CrossRef]
  48. Nourse, H.O.; Roulac, S. Linking real estate decisions to corporate strategy. J. Real Estate Res. 1993, 8, 475–494. [Google Scholar]
  49. Bencardino, M.; Nesticò, A. Demographic Changes and Real Estate Values. A Quantitative Model for Analyzing the Urban-Rural Linkages. Sustainability 2017, 9, 536. [Google Scholar] [CrossRef]
  50. Wyatt, P.J. The development of GIS-based property information system for real estate valuation. Int. J. Geogr. Inf. Sci. 1997, 11, 435–450. [Google Scholar] [CrossRef]
  51. Aiken, M.; Taylor, M.; Moran, R. Always Look a Gift Horse in the Mouth: Community Organisations Controlling Assets. VOLUNTAS: Int. J. Volunt. Nonprofit Organ. 2016, 27, 1669–1693. [Google Scholar] [CrossRef]
  52. South East Europe—Transnational Cooperation Programme. Synthesis Report on the Efficiencies/Inefficiencies of the Institutional Framework for the Municipal Real Property Management in South-East Europe; SEED Center, University of Thessaly: Volos, Greece, 2012; Available online: http://www.southeast-europe.net/en/projects/approved_projects/?id=98 (accessed on 18 September 2018).
  53. South East Europe—Transnational Cooperation Programme. Municipal Real Property Management in South Eastern Cities. 2012. Available online: https://www.keep.eu/keep/project-ext/1022/Municipal%20PROperty%20Management%20In%20South-Eastern%20Cities (accessed on 18 September 2018).
  54. Gruening, G. Origin and theoretical basis of New Public Management. Int. Public Manag. J. 2001, 4, 1–25. [Google Scholar] [CrossRef]
  55. Noto, G.; Bianchi, C. Dealing with multi-level governance and wicked problems in urban transportation systems: The case of Palermo municipality. Systems 2015, 3, 62–80. [Google Scholar] [CrossRef]
  56. Council of Europe. 12 Principles of Good Democratic Governance; Centre of Expertise for Local Government Reform: Strasbourg, France, 2018; Available online: https://www.coe.int/en/web/good-governance/12-principles-and-eloge (accessed on 18 September 2018).
Table 1. International research using public management paradigms (source: own study).
Table 1. International research using public management paradigms (source: own study).
YearAuthor(s)AreaParadigmDesign
New Public ManagementGood Governance
2006Schulte & Ecke [36]GermanyX Multidimensional approach (two questionnaires). 215 questionnaires total.
2007Food and Agriculture Organization (UN) [28]International XSecondary analyses.
2011Lu [35]USAX Survey. 37 questionnaires.
2014Gross, Źróbek & Špirková [37]Poland, Slovakia XCase Study. Two countries.
2014Kask [38]EstoniaXXLiterature review + best practices. Exploratory study with experimental case study elements.
2014Pozega, Crnkovic & Zivkovic [34]CroatiaX Literature Review
2015Gross & Źróbek [18]Europe & Asia XModified Delphi method (Delphi method + survey). 17 responses from 14 countries.
2016Marona [10]Krakow Metropolitan Area (Poland)XXSurvey. 25 municipalities (49%).
2017Lausberg & Wojewnik-Filipkowska [39]Poland and GermanyXXCase study of two cities: Gdansk in Poland and Jena in Germany
2018Constantin et al. [7]RomaniaX Survey. 8 municipalities.
Table 2. Operationalization of NPM standards [42] and GG principles [29] in MREM.
Table 2. Operationalization of NPM standards [42] and GG principles [29] in MREM.
Public Management ConceptsQuestionnaire ItemsDiagnostic Variables
New Public Management [42]
NPM1: Hands on professional management’ in the public sectorQ3.1, Q3.2, Q3.3X3, X4, X5
NPM2: Explicit standards and measure of performanceQ1, Q2, Q3.19, Q3.20X1, X2, X21, X22
NPM3: Greater emphasis on output controlQ3.6, Q3.7, Q3.11X8, X9, X13
NPM4: Shift to disaggregation of units in the public sector *--
NPM5: Shift to greater competition in public sectorQ3.16X18
NPM6: Stress on private sector styles of management practiceQ3.4, Q3.5X6, X7
NPM7: Stress on greater discipline and parsimony in resource useQ4.8, Q4.9X30, X31
Good Governance [29]
GG1: OpennessQ3.12, Q3.13, Q3.14X14, X15, X16
GG2: ParticipationQ3.15X17
GG3: AccountabilityQ3.6, Q3.7, Q3.10X8, X9, X12
GG4: EffectivenessQ2, Q3.8, Q3.9, Q3.19X2, X10, X11, X21
GG5: Coherence *--
* was not evaluated, due to difficulties with its operationalization in the survey.
Table 3. Four classes of municipalities.
Table 3. Four classes of municipalities.
Type of ClassImplementation Level of NPM Standards or GG PrinciplesMathematical Constraint
Class 1low implementation of NPM or GGhi   h i ¯   +   S h i
Class 2low-mid implementation of NPM or GG h i ¯ + S h i >   h i     h i ¯
Class 3mid-high implementation of NPM or GG h i ¯   >   h i     h i ¯     S h i
Class 4high implementation of NPM or GGhi <     h i ¯ + S h i
Table 4. Synthetic measure for NPM standards in MREM (source: own calculations).
Table 4. Synthetic measure for NPM standards in MREM (source: own calculations).
Polish Municipalities (Types)Synthetic MeasureStandard DeviationObservations
Urban0.4530.23118
Urban-rural0.2950.15530
Rural0.3090.12659
All Polish municipalities0.3290.165107
Dutch Municipalities (Types)Synthetic MeasureStandard DeviationObservations
Large0.3620.190620
Medium0.3170.154142
Small0.2570.098214
No data0.3990.19539
All Dutch municipalities0.3260.163285
Table 5. Analysis of variance for NPM standards (source: own calculations).
Table 5. Analysis of variance for NPM standards (source: own calculations).
Analysis of Variance (NPM) in Poland
SourceSSdfMSFProb > F
Between groups0.33620.1686.890.0015
Within groups2.8741040.024--
Total2.8741060.027--
Analysis of Variance (NPM) in Netherlands
SourceSSdfMSFProb > F
Between groups0.09020.0451.840.1657
Within groups1.789730.024--
Total1.879750.025--
Table 6. Classification of municipalities in Poland and The Netherlands taking under consideration NPM standards in MREM—Hellwig’s synthetic measure (source: own calculations).
Table 6. Classification of municipalities in Poland and The Netherlands taking under consideration NPM standards in MREM—Hellwig’s synthetic measure (source: own calculations).
Hellwig for NPM: PolandClass 1Class 2Class 3Class 4Total
Urban145818
5.56%22.22%27.78%44.44%100%
Urban-rural61012230
20.00%33.33%40.00%6.67%100%
Rural92521459
15.25%42.37%35.59%6.78%100%
Total16393814107
14.95%36.45%35.51%13.08%100%
Hellwig for NPM: The NetherlandsClass 1Class 2Class 3Class 4Total
Large municipalities365620
15.00%30.00%25.00%30.00%100%
Medium municipalities7189842
16.67%42.86%21.43%19.05%100%
Small municipalities1103014
7.14%71.43%21.43%0%100%
Total1134171476
14.47%44.74%22.37%18.42%100%
Table 7. Synthetic measure for GG principles in MREM (source: own calculations).
Table 7. Synthetic measure for GG principles in MREM (source: own calculations).
Polish Municipalities (Types)Synthetic MeasureStandard DeviationObservations
Urban0.5820.24719
Urban-rural0.4100.23531
Rural0.3790.16264
All Polish municipalities0.42180.211114
Dutch Municipalities (Types)Synthetic MeasureStandard DeviationObservations
Large0.3720.162020
Medium0.3310.171642
Small0.3360.179714
No data0.3970.21729
All Dutch municipalities0.32650.163285
Table 8. Analysis of variance for GG principles (source: own calculations).
Table 8. Analysis of variance for GG principles (source: own calculations).
Analysis of Variance (GG) in Poland
SourceSSDfMSFProb > F
Between groups0.60620.3037.620.0008
Within groups4.4181110.0398--
Total5.0251130.044--
Analysis of Variance (GG) in The Netherlands
SourceSSDfMSFProb > F
Between groups0.02420.0120.420.6598
Within groups2.125730.0297--
Total2.1499750.028--
Table 9. Classification of municipalities in Poland and Netherlands taking into consideration GG principles in MREM-Hellwig’s synthetic measure (source: own calculations).
Table 9. Classification of municipalities in Poland and Netherlands taking into consideration GG principles in MREM-Hellwig’s synthetic measure (source: own calculations).
Hellwig for GG: PolandClass 1Class 2Class 3Class 4Total
Urban1441019
5.26%21.05%21.05%52.36%100%
Urban-rural988631
29.03%25.81%25.81%19.35%100%
Rural83220464
12.5%50.0%31.25%6.25%100%
Total18443220114
15.79%38.6%28.07%17.54%100%
Hellwig for GG: The NetherlandsClass 1Class 2Class 3Class 4Total
Large municipalities277420
10.00%35.00%35.00%20.00%100%
Medium municipalities71615442
16.67%38.10%35.71%9.52%100%
Small municipalities346114
21.43%28.57%42.86%7.14%100%
Total122728976
15.79%35.53%36.84%11.84%100%

Share and Cite

MDPI and ACS Style

Marona, B.; Van den Beemt-Tjeerdsma, A. Impact of Public Management Approaches on Municipal Real Estate Management in Poland and The Netherlands. Sustainability 2018, 10, 4291. https://doi.org/10.3390/su10114291

AMA Style

Marona B, Van den Beemt-Tjeerdsma A. Impact of Public Management Approaches on Municipal Real Estate Management in Poland and The Netherlands. Sustainability. 2018; 10(11):4291. https://doi.org/10.3390/su10114291

Chicago/Turabian Style

Marona, Bartłomiej, and Annette Van den Beemt-Tjeerdsma. 2018. "Impact of Public Management Approaches on Municipal Real Estate Management in Poland and The Netherlands" Sustainability 10, no. 11: 4291. https://doi.org/10.3390/su10114291

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop