Next Article in Journal
Residence Layout Examined and Optimized by Sunshine: A Comparative Study between Barcelona and Northern Chinese Cities
Previous Article in Journal
Helically Coiled Tube Flocculators in Water Clarification Systems: Optimal Length Evaluation and Process Efficiency Probabilistic Analysis
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sustainability
Volume 16
Issue 5
10.3390/su16052173
sustainability-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Assessment of Performance Measurement Systems’ Ability to Mitigate or Eliminate Typical Barriers Compromising Organisational Sustainability

by
Flávio Cunha
*,
José Dinis-Carvalho
and
Rui M. Sousa
Algoritmi Centre, Department of Production and Systems, University of Minho, 4800-058 Guimarães, Portugal
*
Author to whom correspondence should be addressed.
Sustainability 2024, 16(5), 2173; https://doi.org/10.3390/su16052173
Submission received: 7 January 2024 / Revised: 22 February 2024 / Accepted: 28 February 2024 / Published: 6 March 2024
(This article belongs to the Section Sustainable Engineering and Science)
Browse Figures
Review Reports Versions Notes

Abstract

:
This paper aims to identify the main performance measurement systems (PMSs) documented in the literature and assess their ability to overcome/mitigate a set of 19 specific barriers (identified in a previous paper) to their effectiveness. It also aims to understand what makes each PMS capable of or not capable of dealing with these barriers (i.e., what traits it has) and to explore their connection to some sustainable development goals (SDG). The PRISMA methodology was used to identify the relevant publications, which were then subjected to a detailed content analysis with statistical treatment, followed by the assessment of the potential of each PMS to deal with the barriers. The results made it possible to identify the PMSs most referred to in the literature (ordered list), quantitatively classify the PMSs according to their ability to overcome/mitigate barriers, and identify the barriers most and least addressed by the PMSs. While no single PMS offers a comprehensive solution, certain common traits contribute significantly to overcoming prevalent barriers. The complex interplay between barriers means that some traits can effectively address multiple barriers either directly or indirectly. Regarding implications, these findings provide important inputs (e.g., key recommendations) for developing or improving PMS frameworks that are able to comprehensively address the barriers, thus contributing to organisational effectiveness and, consequently, to the achievement of the SDGs. This constitutes the innovative contribution of this paper. As for limitations, this work is based on the analysis of 28 PMSs resulting from the systematic literature review in two databases (Scopus and Web of Science).

1. Introduction

In today’s dynamic business environment, maintaining success requires organisations to adopt a flexible approach to evolving conditions, forcing them to adopt strategies to be more competitive and sustainable [1]. The notion of sustainable development is grounded in the principles of development (socio-economic progress within ecological limits), needs (equitable resource distribution to uphold the quality of life for all), and future generations (ensuring resources are utilised responsibly for the sustained well-being of future generations) [2]. All United Nations Member States endorsed the 2030 Agenda for Sustainable Development, a broad initiative centred on 17 Sustainable Development Goals (SDGs), which emphasise the importance of broader societal issues [3]. This endorsement established a developmental framework prioritising the eradication of poverty and deprivations, the enhancement of health and education, and the stimulation of economic growth. Simultaneously, the agenda addresses the critical issue of environmental resource degradation on a global scale [4]. Within this main framework, some goals are dedicated to improving production methods with a particular focus on SDG8 and SDG9. These goals endeavour to champion full and productive employment and sustainable industrialisation, innovation, and infrastructure, playing a pivotal role in advancing technological progress and advocating for sustainable production processes. The ultimate aim is to mitigate the adverse environmental impacts associated with industrialisation. Another pertinent goal within this context is SDG 12, which emphasises responsible consumption and production [5]. SDG 12 aims to reduce waste, enhance resource efficiency, and foster sustainable practices in both production and consumption, contributing to the broader goal of sustainable development.
For an organisation that aims to be competitive and sustainable, it becomes imperative to regularly monitor and evaluate its performance and to make appropriate decisions and actions. So, it is only natural that the performance of an organisation has become the preferred subject of interest of managers since the end of the 20th century [6].
The performance and measurement system (PMS) plays a pivotal role in guiding an organisation’s journey toward continuous improvement and sustainability. It plays a crucial role in revealing the current status of an organisation by providing a foundational starting point for improvement by identifying specific areas where enhancements are needed and should be undertaken.
The main functions of a PMS are to create organisational alignment and to translate strategy into action [7]. Bititci (2015) [8] defines the development and use of a PMS as the process of defining objectives; developing a set of performance metrics; and collecting, analysing, reporting, interpreting, reviewing, and acting on performance data. The extensive and intricate nature of that process makes the development and implementation of a PMS a significant challenge for organisations [7], hindering those organisations competiveness and sustainability. These challenges often give rise to barriers that can impede the overall effectiveness of a PMS.
In a prior study conducted by Cunha et al. (2023) [9], a comprehensive examination was undertaken through a systematic literature review. This investigation enabled the identification and categorisation of the primary obstacles that have the potential to hinder the effectiveness of a PMS. The comprehensive list of these barriers (19) is presented in Table 1. Notably, this systematic literature review followed the PRISMA methodology (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) and was specifically tailored to pinpoint the prevailing barriers frequently cited in the literature as impediments to the effectiveness of a PMS.
Since the barriers have already been identified, it is now crucial to explore the existing PMS frameworks (i.e., not only the PMS software/application itself but also the inherent design and development processes) and their capacity to address these barriers (namely, the tools and resources utilised to either eliminate or mitigate the barriers). Therefore, the primary objectives of this study are to systematically identify the spectrum of PMSs documented in the existing literature and to scrutinise the capabilities of these systems in addressing and mitigating or eliminating the several barriers that may hinder their effectiveness.
There are several studies in the literature that compile and analyse the existing PMSs; however, they have different objectives than the present study. Taticchi et al. (2012) [10] focused on identifying PMSs to offer research guidelines for building a PMS, pinpointing some design challenges. Yadav et al. (2014) [11], on the other hand, explored the existing PMSs mainly to assess the state of the art and establish connections with strategic management theories. In another study, Yadav et al. (2013) [12] centred on compiling and analysing PMSs, specifically aiming to scrutinise the research trends over the last two decades. Folan and Browne (2005) [13] contributed to the literature by outlining the evolution of performance measurement and providing recommendations, frameworks, systems, and insights into inter-organisational performance measurement. Thus, the authors argue that a research gap exists, as the ability of PMSs to deal with barriers to their effectiveness is not adequately addressed by the existing literature. The scientific novelty of this article lies precisely in its contribution to filling this research gap.
This paper is structured according to the flowchart in Figure 1.

2. Methods

As a way of identifying and understanding the PMSs that exist in the literature, a systematic literature review (SLR) was conducted. The SLR was performed following the steps of the PRISMA methodology, which is composed of four phases: identification, screening, eligibility, and included publications.
In the first phase, a search for scientific articles, books, and book chapters was executed on the databases Scopus and Web of Science. The search on the databases was performed with the following restrictions: “performance measurement framework” or “performance measurement model” as keywords and from the subject areas of Engineering, Business, and Sociology. As a result of the search, we identified 2098 publications from the Scopus database and 3784 publications from the Web of Science database.
The steps followed during the application of the PRISMA methodology are described in Figure 2. After identifying the publications according to the restrictions, the first step was to remove the duplicate results found in the two databases. In this step, the 25 identified duplicate results were removed. The remaining 5857 publications were screened by analysing their title and keywords, and 5725 were removed during this step. The remaining 132 publication abstracts were analysed, and 60 publications were excluded. The remaining 72 publications were fully analysed, and in 39 of them, PMS frameworks or models were identified.
The references and/or descriptions of the PMSs found in the 39 publications were recorded in a table (not presented in this paper) containing the name of the PMS and the title of the publication that mentions it. Some publications refer to several PMSs, and some PMSs are referred to in several publications (many-to-many relationship). The frequency with which each PMS is mentioned was determined, and it was decided to analyse only those that are mentioned more than once (to limit the size of the study due to the high number of PMSs found).
To analyse these PMSs, a detailed reading of the 39 publications identified by PRISMA was carried out. This rigorous scrutiny made it possible, for each PMS, to perceive its ability to deal with each of the 19 barriers identified in Table 1. Based on this perception, each PMS was classified in relation to each barrier according to the values shown in Table 2. The total score for each PMS is obtained by adding up the values obtained by the PMS with regard to its ability to deal with each barrier.
The results of that analysis are described in Section 3 and discussed in Section 4, leading to the outline of conclusions in Section 5.

3. Results

This section presents the outcomes of the systematic literature review and the findings related to the classification of the PMSs’ capacity to mitigate or eliminate the most common barriers that hinder their effectiveness.
In the 39 publications included in this study, 217 mentions of PMSs were found, which resulted in the identification of 95 different PMSs. Of these, only 28 were considered for a detailed analysis because, as explained in the Methods section, it was decided to only consider PMSs that are mentioned in more than one publication. These PMSs are listed in Table 3, which also indicates the number of publications and the ones each PMS is referred to.
The in-depth analysis of the 28 PMSs allowed for each of them to be classified using the values indicated in Table 2. That classification can be observed in Table 4.
After the classification process was completed (Figure 3), it became evident that only four out of the 28 PMSs assessed had achieved a score exceeding half of the highest possible classification (38 points). These distinguished PMSs are (i) ECOGRAI, (ii) Performance Pyramid, (iii) Integrated Performance and Measurement System (IPMS), and (iv) European Foundation for Quality Management (EFQM) model with scores of 66%, 61%, 58%, and 53%, respectively. On the other hand, seven PMSs failed to attain even one-third of the highest possible classification, as can be observed in Figure 3.
For the PMSs that were classified as the most complete regarding their ability to mitigate or eliminate barriers to their effectiveness, the following was found:
  • ECOGRAI is capable of mitigating or eliminating eight barriers and has some capacity to mitigate or eliminate another nine. It is classified as not being able to mitigate or eliminate only two barriers.
  • Performance Pyramid is capable of mitigating or eliminating seven barriers and has some capacity to mitigate or eliminate another nine. It is classified as not being able to mitigate or eliminate three barriers.
  • IPMS is capable of mitigating or eliminating five barriers and has some capacity to mitigate or eliminate another twelve. It is classified as not being able to mitigate or eliminate only two barriers.
  • EFQM model is capable of mitigating or eliminating two barriers and has some capacity to mitigate or eliminate another sixteen. It is classified as not being able to mitigate or eliminate only one barrier.
This classification, for each of the 29 PMSs, can be observed in Figure 4.
The barriers (Table 1) with the highest scores in terms of the capacity of the PMSs to address them were the lack of connection to strategy, high complexity, and excess of indicators, registering percentages of 75%, 70%, and 68%, respectively. On the other hand, the barriers with the lowest scores were poor communication system, blame culture, lack of trained resources, and lack of rewards, as can be observed in Figure 5.
Regarding the barriers on which more PMSs were classified as having the capacity to mitigate or eliminate, the following was found:
  • Lack of connection to strategy: A total of 16 PMSs have capacity to eliminate or mitigate this barrier, and 10 have some capacity. Only two PMSs have weak or no capacity.
  • High complexity: A total of 14 PMSs have capacity to eliminate or mitigate this barrier, and 11 have some capacity. Three PMSs have weak or no capacity.
  • Excess of indicators: A total of 11 PMSs have capacity to eliminate or mitigate this barrier, and 16 have some capacity. Only one PMS has weak or no capacity.
The number of PMSs classified according to their capacity to mitigate or eliminate each of the 19 barriers can be observed in Figure 6.
The next section explores a discussion of the most robust PMSs for mitigating and eliminating each specific barrier along with the tools and mechanisms that empower them to excel in this regard.

4. Discussion

This section presents, for each barrier, the PMSs and why they have been classified as having capacity to mitigate or eliminate that barrier. At the end, there is a discussion about the relationship between PMSs and sustainable development, starting with the connection between some barriers to the effectiveness of PMSs and the difficulties in achieving sustainable development.

4.1. PMS Capacity to Mitigate Barriers

  • Blame culture
While no PMS demonstrated a strong capacity to completely eliminate or mitigate this barrier, four were recognised for possessing some potential. These PMSs included EFQM, IPMS, ECOGRAI, and APL. EFQM, in particular, was singled out for its ability to contribute to this context. This is attributed to its focus on fostering a culture of improvement and human resource management, which indirectly serves to mitigate the existence of a blame culture by emphasising the well-being of employees and fostering an improvement culture.
2.
Lack of connection to strategy
Regarding this particular barrier, only two PMSs, namely ABC and the DuPont model, were deemed to have a weak capacity to address and mitigate it.
Of the 16 PMSs classified as having a strong capacity, a common thread emerged—these systems were founded on a fundamental principle: alignment with the organisation’s strategy. Each of these PMSs commenced with the articulation of the organisation’s strategy with subsequent steps contingent on this strategy definition. The strategy definition not only facilitates the establishment of objectives that could be disseminated throughout the organisation but also originates the development of indicators used to monitor and control the progress of these defined objectives.
The definition of the strategy can be achieved by the identification of the organisation’s mission, vision, and values as well as by the key success factors, as recommended by Brown’s framework. Alternatively, it may include the use of strategy maps, as recommended by Balanced Scorecard (BSC), DMDPM framework, and Kanji’s framework.
3.
Issues on target definition
Out of the assessed PMSs, ECOGRAI stood out as the sole system with a strong capacity to effectively eliminate or mitigate this particular barrier. ECOGRAI’s strength lies in its distinctive approach, which emphasises the need for a precise identification of the objectives tied to the indicators as well as a consideration of potential adverse effects and consequences on other indicators. This comprehensive understanding of the factors influencing the indicator is crucial in shaping the target definition for each specific indicator. These targets are expected to be attainable, reasonable, and achievable [40].
4.
Unclear system
Among the PMSs evaluated, five were assessed as having a weak capacity to address and mitigate this particular barrier. Their inadequacy arises from a lack of clarity, as these PMSs were characterised by vagueness, making it challenging to interpret their system and purpose.
Conversely, the 11 PMSs recognised as possessing a strong capacity to address this barrier exhibited a notably clear framework. In some instances, they even provided users with a step-by-step guide, facilitating a clear understanding of the system and its operational aspects.
5.
Lack of top management involvement
PMSs with a strong capacity were distinguished by specific attributes. These included a well-defined organisational strategy set by top management and its efficient deployment throughout the organisation. Top management plays a pivotal role in this process, not only in the formulation of strategy but also in its widespread implementation. Moreover, their continued engagement is essential, manifested through the active utilisation of their own objectives and indicators to enhance performance. Additionally, they are responsible for maintaining the PMS by ensuring the effective adoption of the system throughout the organisation.
6.
Poor communication system
PPVC was the only PMS identified as possessing a strong capacity to effectively eliminate or mitigate issues related to a poor communication system. What PPVC does to ensure the absence of a poor communication system is to have an efficient dissemination of objectives and indicators throughout the organisation. This approach fosters a sense of alignment at every organisational level, where each team not only identifies with their respective objectives and indicators but also remains consistently aware of them.
It is also imperative that everyone at each level of the organisation receives regular updates about indicator status. Equally vital is the prerequisite that they comprehensively grasp how their actions can impact these indicators, either positively or negatively.
7.
High complexity
Among the PMSs assessed, 14 were categorised as having a strong capacity, while 11 demonstrated some capacity to address the issue at hand. Only three PMSs were classified as having a weak capacity. Notably, the distinguishing trait of the PMSs with a strong capacity is their simplicity in terms of both structure and comprehensibility.
It is pertinent to acknowledge that most PMSs primarily offer indicators or serve as guidelines for PMS implementation. Therefore, the complexity of the resulting PMS depends more on the way in which it is executed rather than the inherent complexity of the framework itself. While initiating with a relatively uncomplicated framework is indeed important, maintaining simplicity throughout the implementation process is equally vital.
8.
Lack of use for improvement
Among the PMSs evaluated, two were identified as having a strong capacity to effectively eliminate or mitigate this particular barrier. To eliminate or mitigate this barrier, the PMSs integrate improvement targets at every organisational level and establish improvement actions that are closely linked to the indicators in place. These attributes play a pivotal role in preserving the PMS as an effective tool, proactively preventing its underutilisation and serving as a catalyst for driving continuous improvement across the organisation.
9.
Lack of balance of indicators
Among the PMSs classified as having a strong capacity to eliminate or mitigate this barrier, two significant characteristics emerged. In these PMSs, a limited number of objectives and indicators were shared across all departments. This approach fosters simplicity, thereby reducing the occurrence of conflicting indicators. However, it is worth noting that this simplicity may come at the cost of constraining the PMS range concerning objectives and indicators.
The second noteworthy characteristic involves mapping the influence of each indicator on performance. This practice empowers organisations to comprehend whether an indicator exerts a positive or negative influence on other indicators. Importantly, this feature appears to be more robust in addressing the barrier as it enables the identification of concurrent indicators without imposing limitations on the overall scope of the PMS.
10.
Lack of rewards
In the context of this specific barrier, no PMS was identified as having a strong capacity to completely eliminate or mitigate it. However, two PMSs were categorised as having some capacity due to their inclusion of performance incentives. Notably, in one of these systems, the results are also measured through people satisfaction.
It is important to recognise that these characteristics, while beneficial, may not be sufficient. What is essential is the establishment of a reward system that is directly tied to actual performance results. This should extend to all levels of the organisation with rewards being closely associated with the specific objectives and indicators relevant to each level. Furthermore, it is imperative that this reward system is clear and remains consistent without undergoing changes during the evaluation period.
11.
False expectations
Among the PMSs under evaluation, APL emerged as the sole system with the highest classification. This distinction was achieved due to its unique feature of connecting actions directly to the indicators, all designed to enhance the performance of these indicators.
This linkage represents a critical bridge between performance measurement and actionable improvements, a fundamental step in eliminating the false expectation that merely measuring performance will naturally lead to improvements.
12.
Inappropriate IT tools
The sole PMS that specifically addressed the requirement for IT tools was PPVC, owing to its extensive data demands, necessitating advanced IT resources. In contrast, the remaining PMSs did not specify the type of IT tools required, and this would largely depend on the complexity of each particular PMS. Generally, as the complexity of the PMS increases, the need for more advanced IT tools becomes apparent.
Today, organisations enjoy ready access to common tools like spreadsheets and visual dashboard software. For most PMSs, these readily available and user-friendly tools should typically be sufficient, averting the need for more specialised or advanced IT resources.
13.
Excess of indicators
Eleven PMSs were designated as having a strong capacity to effectively eliminate or mitigate this barrier. They achieved this by adopting two primary strategies. First, they defined or restricted the selection of indicators, or they outlined categories for the types of indicators that should be included. This approach effectively curtailed the number of indicators within the PMS, ensuring a more streamlined and manageable system.
ECOGRAI employed an alternative strategy to overcome this challenge. In ECOGRAI, the indicators were not predetermined; instead, they were derived directly from the objectives. This approach involved the identification of the variables that decision-makers could directly influence to achieve their objectives. The outcome of this strategy results in the generation of pertinent indicators while also keeping their number limited, thus fostering a balanced and efficient PMS.
14.
Lack of trained resources
This is among the barriers for which there are comparatively fewer PMSs equipped with strong or even some capabilities to eliminate or mitigate it. Only three PMSs were identified as having some capacity in this regard. These PMSs acknowledge the importance of training resources to effectively implement the PMS or recognising the value of having well-prepared resources, but they often fall short of specifying the content and nature of this training.
To provide a solid foundation for resource training, it is imperative that the PMS is comprehensively documented. This documentation should encompass information related to objectives, indicators, targets, and the system’s deployment throughout the organisation. In terms of training, resources should receive guidance on several fronts. They should understand their specific objectives and how these align with and contribute to higher hierarchical objectives. Furthermore, they need to be educated about the indicators, including their significance, calculation methods, and how their behaviours and actions can influence these indicators either positively or negatively.
The training program should also encompass the communication strategies utilised for disseminating objectives, indicators, targets, and actions throughout the organisation, thereby ensuring a holistic understanding of the PMS mechanisms.
15.
Lack of employee involvement
Three PMSs stood out as possessing a strong capacity to effectively eliminate or mitigate this barrier. A common characteristic among them was the comprehensive involvement of all organisational levels in the PMS. This is achieved through the efficient deployment of objectives, indicators, targets, and actions across every level of the organisation.
Additional factors play pivotal roles in fostering employee involvement. These include well-structured resource training, the presence of performance-based incentives (rewards), top management involvement, and the promotion of a continuous-improvement culture. These combined efforts contribute significantly to encouraging and nurturing employee involvement.
16.
Inappropriate indicators
Eleven PMSs were identified as having a strong capacity to effectively mitigate or eliminate this barrier. They accomplished this by directly defining the indicators or establishing categories for these indicators. This strategic approach ensures that the indicators are closely aligned with the objectives of the PMS.
In contrast, ECOGRAI adopts an alternative strategy, one that effectively eliminates this barrier without the need for direct indicator or category definitions. Within the ECOGRAI framework, the indicators are derived directly from the objectives by identifying the variables upon which decision-makers can act to achieve their goals. This approach enhances the appropriateness and relevance of the indicators within the system.
17.
Lack of indicator understanding
The only PMS with a strong capacity to effectively eliminate or mitigate this barrier was ECOGRAI. The understanding of indicators is of vital importance, and this entails not only providing training on these indicators but also ensuring access to comprehensive information about them.
In the case of ECOGRAI, each indicator is meticulously documented, featuring the following:
    • Identification details, including the indicator’s name, decision centre, time horizon, and reporting period.
    • Objectives and underlying drivers associated with the indicator.
    • Identification of any potential adverse effects or unintended consequences.
    • Identification of the necessary data for implementing the indicator.
    • Specification of the data processing methods.
    • Utilisation of visual representations, typically through graphics, to enhance clarity.
This comprehensive approach to indicator documentation aligns with the standards outlined in ISO 22400:2014 [45], further emphasising its significance.
18.
Time and resources required
In the context of this barrier, no PMS earned a strong classification. However, a total of 25 PMSs were recognised as having some capacity to address it due to the acknowledgment of the requirements of time and resources to design, implement, and maintain a PMS. The existence of this barrier is directly connected with the complexity of the system. The more complex the system, the greater the demands in terms of the time and resources needed to maintain its functionality.
To tackle this challenge effectively, it becomes imperative to define and allocate the necessary time and resources at each level of the organisation. This allocation should encompass activities such as formulating objectives, ensuring the indicators remain up to date, and designing and implementing performance improvement actions.
19.
Difficulties in collecting, analysing, and presenting data
In the context of this specific barrier, none of the assessed PMSs achieved a strong classification. However, 26 PMSs demonstrated some capacity to address it, as the existence of this barrier is closely intertwined with the complexity of the system.
Conversely, two PMSs were classified as weak. The first is due to its omission of any mention regarding data collection, analysis, or presentation; and the second is because it dealt with substantial volumes of data (as observed in the case of PPVC). These two characteristics rendered the mitigation or elimination of this barrier difficult.
Generally, the majority of PMSs, unless exceptionally complex, should be equipped to manage data effectively through readily available IT tools in contemporary organisations, such as spreadsheets and dashboard software.

4.2. Relationship between PMSs and Sustainable Development

In general terms, it can be speculated that some barriers to the effectiveness of PMSs are related to difficulties in the process of transitioning to sustainable practices. For example, the barrier lack of use for improvement is related to the difficulty in developing better production processes (an aspect directly mentioned in SDG8 and SDG9, see next) and is, therefore, also an obstacle to sustainable development.
In fact, better PMSs contribute to more efficient monitoring of companies’ performance, which can cover various aspects (e.g., economic, environmental, operational, and social), thus allowing for the development of pertinent and focused improvement actions. These actions may address, for example, the rational use of resources, which is directly related to SDG9 and SDG12 [5], more specifically to target 9.4 (“… increased resource-use efficiency…”) and target 12.2 (“… efficient use of natural resources…”), the efficiency of production processes (SDG8, target 8.2 “…technological upgrading and innovation…” and SDG9, target 9.4 “… upgrade infrastructure and retrofit industries to make them sustainable…”), including waste production (SDG12, target 12.5 “… substantially reduce waste generation…”), and employee satisfaction/motivation (SDG8 “…full and productive employment and decent work for all…”), thus contributing to increased productivity and profit. Therefore, the authors argue that PMSs can be designed to be aligned with the principles of sustainable development referred to in the introduction (development, needs, and future generations) and, from this perspective, are instrumental for companies’ sustainability.

5. Conclusions

The analysis showed how PMSs relate to overcoming the studied barriers. While no perfect solution emerged, many share traits vital for tackling common barriers. PMS traits can effectively address multiple barriers due to their interconnectedness.
First and foremost, a PMS must prioritise simplicity and clarity to avoid unclear systems and high complexity. Without these traits, other barriers, e.g., difficulties in collecting, analysing, and presenting data, alongside inappropriate IT tools and time and resources required can be exacerbated. Simplicity and clarity can be attained through a schematic visual representation of the PMS coupled with a step-by-step implementation guide.
Additionally, a successful PMS should foster a culture of continuous improvement, which is crucial for addressing barriers like blame culture, false expectations, lack of employee involvement, and lack of use for improvement. Establishing a direct link between actions and the key performance indicators (KPIs) that they positively impact is essential to sustain this culture effectively.
The key to a strong PMS is its alignment with a well-defined organisational strategy established by top management, guiding the derivation of objectives and selection of indicators, targets, and actions. This helps mitigate issues like lack of connection to the strategy, lack of top management involvement, and inappropriate indicators. The objectives derived from the strategy should cascade throughout the organisation, being customised to each hierarchical level. This helps mitigate issues related to a blame culture, lack of employee and top management involvement, and poor communication systems. Subsequently, indicators must be carefully derived from objectives and tailored to each organisational level to ensure a balanced and relevant set of metrics. This step is pivotal in addressing an excess of indicators and a lack of balance of indicators.
Thorough documentation of the PMS is crucial, ensuring information accessibility to all and facilitating human resource training. This contributes to mitigating issues such as lack of understanding of indicators and lack of trained resources. Part of this documentation should delve into the factors that can influence indicators positively or negatively, aiding in goal definition and achieving a balanced indicator framework.
While certain barriers are addressed by many PMSs, others, such as blame culture, lack of trained resources, lack of rewards, and poor communication systems, remain largely unaddressed. Moreover, issues like inadequate IT tools, false expectations, lack of use for improvement, issues on target definition, and lack of indicator understanding are only lightly addressed or addressed by a limited number of PMSs. Notably, none of the analysed PMSs demonstrated a comprehensive ability to address all these barriers.
Thus, in conclusion, future research is needed to develop or enhance PMS frameworks that can effectively mitigate or eliminate all barriers. This study lays the groundwork for such research by identifying the existing PMSs and their tools to address each barrier as well as highlighting areas where new tools or methodologies are needed. As shown in Section 4.2, better PMSs contribute to organisations’ sustainable development.
As for limitations, this study analysed 28 PMSs (systematic literature review in Scopus and Web of Science databases), focusing solely on the methodologies and tools within these selected systems. We acknowledge the possibility of additional PMSs in the literature that were not included in this study with the capacity to mitigate or eliminate common barriers.

Author Contributions

Conceptualisation, F.C., J.D.-C. and R.M.S.; methodology, F.C., J.D.-C. and R.M.S.; validation, F.C., J.D.-C. and R.M.S.; formal analysis, F.C.; investigation, F.C.; data curation, F.C.; writing—original draft preparation, F.C.; writing—review and editing, F.C., J.D.-C. and R.M.S.; supervision, J.D.-C. and R.M.S. All authors have read and agreed to the published version of the manuscript.

Funding

This work has been supported by FCT—Fundação para a Ciência e Tecnologia within the R&D Units Project Scope: UIDB/00319/2020.

Data Availability Statement

Data are contained within the article.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Dinis-Carvalho, J.; Sousa, R.M.; Moniz, I.; Macedo, H.; Lima, R.M. Improving the Performance of a SME in the Cutlery Sector Using Lean Thinking and Digital Transformation. Sustainability 2023, 15, 8302. [Google Scholar] [CrossRef]
  2. Klarin, T. The Concept of Sustainable Development: From its Beginning to the Contemporary Issues. Zagreb Int. Rev. Econ. Bus. 2018, 21, 67–94. [Google Scholar] [CrossRef]
  3. Manioudis, M.; Meramveliotakis, G. Broad strokes towards a grand theory in the analysis of sustainable development: A return to the classical political economy. New Polit. Econ. 2022, 27, 866–878. [Google Scholar] [CrossRef]
  4. United Nations. Sustainable Development Goals—Teaching Guide and Resources; United Nations: New York, NY, USA, 2018.
  5. United Nations. Transforming Our World: The 2030 Agenda for Sustainable Development; United Nations: New York, NY, USA, 2016; p. 36.
  6. Papulová, Z.; Gažová, A.; Šlenker, M.; Papula, J. Performance measurement system: Implementation process in smes. Sustainability 2021, 13, 4794. [Google Scholar] [CrossRef]
  7. Fantozzi, I.C.; Di Luozzo, S.; Schiraldi, M.M. Industrial Performance Measurement Systems Coherence: A Comparative Analysis of Current Methodologies, Validation and Introduction to Key Activity Indicators. Appl. Sci. 2023, 13, 235. [Google Scholar] [CrossRef]
  8. Bititci, U.S. Managing Business Performance; Wiley: Hoboken, NJ, USA, 2015. [Google Scholar]
  9. Cunha, F.; Dinis-Carvalho, J.; Sousa, R.M. Performance Measurement Systems in Continuous Improvement Environments: Obstacles to Their Effectiveness. Sustainability 2023, 15, 867. [Google Scholar] [CrossRef]
  10. Taticchi, P.; Balachandran, K.; Tonelli, F. Performance measurement and management systems: State of the art, guidelines for design and challenges. Meas. Bus. Excell. 2012, 16, 41–54. [Google Scholar] [CrossRef]
  11. Yadav, N.; Sushil; Sagar, M. Revisiting performance measurement and management: Deriving linkages with strategic management theories. Int. J. Bus. Perform. Manag. 2014, 15, 87–105. [Google Scholar] [CrossRef]
  12. Yadav, N.; Sushil; Sagar, M. Performance measurement and management frameworks: Research trends of the last two decades. Bus. Process Manag. J. 2013, 19, 947–971. [Google Scholar] [CrossRef]
  13. Folan, P.; Browne, J. A review of performance measurement: Towards performance management. Comput. Ind. 2005, 56, 663–680. [Google Scholar] [CrossRef]
  14. Chorfi, Z.; Benabbou, L.; Berrado, A. An integrated performance measurement framework for enhancing public health care supply chains. Supply Chain Forum 2018, 19, 191–203. [Google Scholar] [CrossRef]
  15. Moreira, M.; Tjahjono, B. Applying performance measures to support decision-making in supply chain operations: A case of beverage industry. Int. J. Prod. Res. 2016, 54, 2345–2365. [Google Scholar] [CrossRef]
  16. Franco-Santos, M.; Lucianetti, L.; Bourne, M. Contemporary performance measurement systems: A review of their consequences and a framework for research. Manag. Account. Res. 2012, 23, 79–119. [Google Scholar] [CrossRef]
  17. Atkinson, M. Developing and using a performance management framework: A case study. Meas. Bus. Excell. 2012, 16, 47–56. [Google Scholar] [CrossRef]
  18. Sousa, S.; Aspinwall, E. Development of a performance measurement framework for SMEs. Total Qual. Manag. Bus. Excell. 2010, 21, 475–501. [Google Scholar] [CrossRef]
  19. Ferreira, P.S.; Shamsuzzoha, A.H.M.; Toscano, C.; Cunha, P. Framework for performance measurement and management in a collaborative business environment. Int. J. Product. Perform. Manag. 2012, 61, 672–690. [Google Scholar] [CrossRef]
  20. Gambelli, D.; Solfanelli, F.; Orsini, S.; Zanoli, R. Measuring the economic performance of small ruminant farms using balanced scorecard and importance-performance analysis: A european case study. Sustainability 2021, 13, 3321. [Google Scholar] [CrossRef]
  21. Zanon, L.G.; Ulhoa, T.F.; Esposto, K.F. Performance measurement and lean maturity: Congruence for improvement. Prod. Plan. Control 2021, 32, 760–774. [Google Scholar] [CrossRef]
  22. Frederico, G.F.; Garza-Reyes, J.A.; Kumar, A.; Kumar, V. Performance measurement for supply chains in the Industry 4.0 era: A balanced scorecard approach. Int. J. Product. Perform. Manag. 2021, 70, 789–807. [Google Scholar] [CrossRef]
  23. Pei, Y.L.; Amekudzi, A.A.; Meyer, M.D.; Barrella, E.M.; Ross, C.L. Performance measurement frameworks and development of effective sustainable transport strategies and indicators. Transp. Res. Rec. 2010, 2163, 73–80. [Google Scholar] [CrossRef]
  24. Micheli, P.; Kennerley, M. Performance measurement frameworks in public and non-profit sectors. Prod. Plan. Control 2005, 16, 125–134. [Google Scholar] [CrossRef]
  25. Öz, H.H.; Özyörük, B. Performance measurement in-fourth party reverse logistics. Meas. Bus. Excell. 2021, 27, 549–578. [Google Scholar] [CrossRef]
  26. Gaiardelli, P.; Saccani, N.; Songini, L. Performance measurement systems in after-sales service: An integrated framework. Int. J. Bus. Perform. Manag. 2007, 9, 145–171. [Google Scholar] [CrossRef]
  27. Suwignjo, P.; Bititci, U.S.; Carrie, A.S. Quantitative Models for Performance Measurement System. Int. J. Prod. Econ. 2000, 64, 231–241. [Google Scholar] [CrossRef]
  28. Saleheen, F.; Habib, M.M.; Hanafi, Z. Supply chain performance measurement model: A literature review. Int. J. Supply Chain Manag. 2018, 7, 70–78. [Google Scholar]
  29. Bai, C.; Sarkis, J. Supply-chain performance-measurement system management using neighbourhood rough sets. Int. J. Prod. Res. 2012, 50, 2484–2500. [Google Scholar] [CrossRef]
  30. Gimbert, X.; Bisbe, J.; Mendoza, X. The role of performance measurement systems in strategy formulation processes. Long Range Plann. 2010, 43, 477–497. [Google Scholar] [CrossRef]
  31. Berrah, L.; Clivillé, V. Towards an aggregation performance measurement system model in a supply chain context. Comput. Ind. 2007, 58, 709–719. [Google Scholar] [CrossRef]
  32. Lauras, M.; Lamothe, J.; Pingaud, H. A business process oriented method to design supply chain performance measurement systems. Int. J. Bus. Perform. Manag. 2011, 12, 354–376. [Google Scholar] [CrossRef]
  33. Melnyk, S.A.; Stewart, D.M.; Swink, M. Metrics and performance measurement in operations management: Dealing with the metrics maze. J. Oper. Manag. 2004, 22, 209–218. [Google Scholar] [CrossRef]
  34. Agami, N.; Saleh, M.; Rasmy, M. A hybrid dynamic framework for supply chain performance improvement. IEEE Syst. J. 2012, 6, 469–478. [Google Scholar] [CrossRef]
  35. Garengo, P. A performance measurement system for SMEs taking part in Quality Award Programmes. Total Qual. Manag. Bus. Excell. 2009, 20, 91–105. [Google Scholar] [CrossRef]
  36. Dweekat, A.J.; Hwang, G.; Park, J. A supply chain performance measurement approach using the internet of things: Toward more practical SCPMS. Ind. Manag. Data Syst. 2017, 117, 267–286. [Google Scholar] [CrossRef]
  37. Bhagwat, R.; Sharma, M.K. An application of the integrated AHP-PGP model for performance measurement of supply chain management. Prod. Plan. Control 2009, 20, 678–690. [Google Scholar] [CrossRef]
  38. Tan, W.A.; Shen, W.; Xu, L.; Zhou, B.; Li, L. A business process intelligence system for enterprise process performance management. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 2008, 38, 745–756. [Google Scholar] [CrossRef]
  39. Laitinen, E.K.; Länsiluoto, A.; Rautiainen, I. Extracting appropriate scope for information systems: A case study. Ind. Manag. Data Syst. 2009, 109, 305–321. [Google Scholar] [CrossRef]
  40. Taticchi, P.; Garengo, P.; Nudurupati, S.S.; Tonelli, F.; Pasqualino, R. A review of decision-support tools and performance measurement and sustainable supply chain management. Int. J. Prod. Res. 2015, 53, 6473–6494. [Google Scholar] [CrossRef]
  41. Amir-Heidari, P.; Maknoon, R.; Taheri, B.; Bazyari, M. A new framework for HSE performance measurement and monitoring. Saf. Sci. 2017, 100, 157–167. [Google Scholar] [CrossRef]
  42. Bititci, U.S.; Suwignjo, P.; Carrie, A.S. Strategy management through quantitative modelling of performance measurement systems. Int. J. Prod. Econ. 2001, 69, 15–22. [Google Scholar] [CrossRef]
  43. Lee, H.; Han, I.; Kwak, W. Developing a business performance evaluation system: An analytic hierarchical model. Eng. Econ. 1995, 40, 343–357. [Google Scholar] [CrossRef]
  44. Yurdakul, M. Measuring a manufacturing system’s performance using Saaty’s system with feedback approach. Integr. Manuf. Syst. 2002, 13, 25–34. [Google Scholar] [CrossRef]
  45. ISO 22400; Automation Systems and Integration. International Organization for Standardization: Geneva, Switzerland, 2014.
Sustainability 16 02173 g001
Figure 1. Structure of the paper.
Figure 1. Structure of the paper.
Sustainability 16 02173 g001
Sustainability 16 02173 g002
Figure 2. Results of PRISMA methodology application.
Figure 2. Results of PRISMA methodology application.
Sustainability 16 02173 g002
Sustainability 16 02173 g003
Figure 3. Total score of each PMS.
Figure 3. Total score of each PMS.
Sustainability 16 02173 g003
Sustainability 16 02173 g004
Figure 4. Number of classifications of each type per PMS.
Figure 4. Number of classifications of each type per PMS.
Sustainability 16 02173 g004
Sustainability 16 02173 g005
Figure 5. Total score of each barrier.
Figure 5. Total score of each barrier.
Sustainability 16 02173 g005
Sustainability 16 02173 g006
Figure 6. Number of systems of each category of mitigation/elimination capacity per barrier.
Figure 6. Number of systems of each category of mitigation/elimination capacity per barrier.
Sustainability 16 02173 g006
Table 1. Main barriers to PMS effectiveness.
Table 1. Main barriers to PMS effectiveness.
1. blame culture11. false expectations
2. lack of connection to strategy12. inappropriate IT tools
3. issues on target definition13. excess of indicators
4. unclear system14. lack of trained resources
5. lack of top management involvement15. lack of employee involvement
6. poor communication system16. inappropriate indicators
7. high complexity17. lack of indicator understanding
8. lack of use for improvement18. time and resources required
9. lack of balance of indicators19. difficulties in collecting, analysing, and presenting data
10. lack of rewards
Table 2. Scale of values to classify the capacity of a PMS to mitigate or eliminate a barrier.
Table 2. Scale of values to classify the capacity of a PMS to mitigate or eliminate a barrier.
ValueSymbolMeaning
0●Weak capacity to mitigate or eliminate
1●Some capacity to mitigate or eliminate
2●Strong capacity to mitigate or eliminate
Table 3. PMS to be analysed.
Table 3. PMS to be analysed.
PMSNumber of ReferencesReferences
Balanced Scorecard28[10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37]
Performance prism15[10,11,12,13,16,17,19,21,23,24,25,28,29,30,32]
Performance pyramid (SMART)14[10,11,13,17,21,25,26,27,28,29,30,31,38,39]
SCOR MODEL12[14,18,19,22,25,28,29,31,32,34,36,40]
Activity based costing7[10,27,28,31,32,34,38]
Results and determinants framework7[10,11,12,13,17,26,38]
Integrated Performance and Measurement framework (IPMF)6[10,11,12,13,14,41]
EFQM model6[10,11,12,13,25,26]
Performance Measurement Questionnaire (PMQ)5[10,11,15,27,32]
Economic Value-Added Model (EVA)4[10,11,12,34]
Performance Measurement Matrix4[11,17,21,26]
Integrated Performance Measurement System (IPMS)3[10,11,12]
Performance Planning Value Chain (PPVC)3[10,11,12]
ECOGRAI3[15,31,32]
Theory of constraints (TOC) measurement system3[25,33,34]
Action-Profit Linkage Model (APL)3[10,11,12]
Dynamic Performance Measurement System (DPMS)3[10,11,12]
Quantitative model for performance measurement system (QMPMS)3[12,27,42]
Analytic Hierarchical Performance Model2[43,44]
Dynamic multidimensional performance framework2[11,12]
Flexible strategy game-card2[11,12]
AMBITE2[13,38]
Kanji’s business scorecard2[11,12]
Cambridge Performance Measurement Framework (CPMF)2[10,32]
Brown’s framework2[13,17]
DuPont model2[11,33]
Sustainability performance measurement system2[11,12]
Holistic performance management framework2[11,12]
Table 4. Classification of PMSs according to their capacity to mitigate or eliminate each barrier.
Table 4. Classification of PMSs according to their capacity to mitigate or eliminate each barrier.
PMS12345678910111213141516171819
Balanced Scorecard●●●●●●●●●●●●●●●●●●●
Performance prism●●●●●●●●●●●●●●●●●●●
Performance pyramid (SMART)●●●●●●●●●●●●●●●●●●●
Supply Chain Operations Reference (SCOR) Model●●●●●●●●●●●●●●●●●●●
Activity based costing (ABC)●●●●●●●●●●●●●●●●●●●
Results and determinants framework●●●●●●●●●●●●●●●●●●●
Integrated Performance and Measurement framework (IPMF)●●●●●●●●●●●●●●●●●●●
European Foundation for Quality Management (EFQM) Model●●●●●●●●●●●●●●●●●●●
Performance Measurement Questionnaire (PMQ)●●●●●●●●●●●●●●●●●●●
Economic Value-Added Model (EVA)●●●●●●●●●●●●●●●●●●●
Performance Measurement Matrix●●●●●●●●●●●●●●●●●●●
Integrated Performance Measurement System (IPMS)●●●●●●●●●●●●●●●●●●●
Performance Planning Value Chain (PPVC)●●●●●●●●●●●●●●●●●●●
ECOGRAI●●●●●●●●●●●●●●●●●●●
Theory of constraints (TOC) measurement system●●●●●●●●●●●●●●●●●●●
Action-Profit Linkage Model (APL)●●●●●●●●●●●●●●●●●●●
Dynamic Performance Measurement System (DPMS)●●●●●●●●●●●●●●●●●●●
Quantitative model for performance measurement system (QMPMS)●●●●●●●●●●●●●●●●●●●
Analytic Hierarchical Performance Model●●●●●●●●●●●●●●●●●●●
Dynamic multidimensional performance framework●●●●●●●●●●●●●●●●●●●
Flexible strategy game-card●●●●●●●●●●●●●●●●●●●
AMBITE●●●●●●●●●●●●●●●●●●●
Kanji’s business scorecard●●●●●●●●●●●●●●●●●●●
Cambridge Performance Measurement Framework (CPMF)●●●●●●●●●●●●●●●●●●●
Brown’s framework●●●●●●●●●●●●●●●●●●●
DuPont model●●●●●●●●●●●●●●●●●●●
Sustainability performance measurement system●●●●●●●●●●●●●●●●●●●
Holistic performance management framework●●●●●●●●●●●●●●●●●●●
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Cunha, F.; Dinis-Carvalho, J.; Sousa, R.M. Assessment of Performance Measurement Systems’ Ability to Mitigate or Eliminate Typical Barriers Compromising Organisational Sustainability. Sustainability 2024, 16, 2173. https://doi.org/10.3390/su16052173

AMA Style

Cunha F, Dinis-Carvalho J, Sousa RM. Assessment of Performance Measurement Systems’ Ability to Mitigate or Eliminate Typical Barriers Compromising Organisational Sustainability. Sustainability. 2024; 16(5):2173. https://doi.org/10.3390/su16052173

Chicago/Turabian Style

Cunha, Flávio, José Dinis-Carvalho, and Rui M. Sousa. 2024. "Assessment of Performance Measurement Systems’ Ability to Mitigate or Eliminate Typical Barriers Compromising Organisational Sustainability" Sustainability 16, no. 5: 2173. https://doi.org/10.3390/su16052173

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop