Next Article in Journal
Automobilities after SARS-CoV-2: A Socio-Technical Perspective
Previous Article in Journal
Long-Term-Based Road Blackspot Screening Procedures by Machine Learning Algorithms
Previous Article in Special Issue
Sustainability Outcomes of Green Processes in Relation to Industry 4.0 in Manufacturing: Systematic Review
 
 
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Identification of Key Performance Indicators in Project-Based Organisations through the Lean Approach

Faculty of Engineering, University of the Basque Country, 48940 Bilbao, Spain
*
Author to whom correspondence should be addressed.
Sustainability 2020, 12(15), 5977; https://doi.org/10.3390/su12155977
Received: 30 June 2020 / Revised: 20 July 2020 / Accepted: 22 July 2020 / Published: 24 July 2020
(This article belongs to the Special Issue Green Manufacturing Processes for Leading Industrial Sectors)

Abstract

:
For the time being, companies and organisations are being forced to compete in utterly complex and globalised environments, facing massive natural, economic, and technological challenges on a daily basis. Addressing these challenges would be impossible without a proper approach that helps them identify, measure, understand, and control the performance of their organisations. Lean principles and techniques rise as a solution. This paper justifies and proposes the use of lean principles and techniques to identify key performance indicators (KPIs) in project-based organisations based on their organisational and operational needs. The research focuses mainly on the identification and categorisation of KPIs through a qualitative approach, based on systematic literature review (SLR) of performance indicators, project management, and project success. As a case study, an analysis of relevant information of an R&D and innovation project-based organisation, such as quality manuals, a benchmarking process, internal studies, and surveys regarding what success means for different kinds of stakeholders and for the organisation itself was conducted. As a result, this research is of a high value for project-based organisations, especially those that are not apprised of how to correctly formulate a series of KPIs, or whose path to it is still not clear.

1. Introduction

Nowadays, organisations are forced to compete in utterly complex and globalised environments, facing massive natural, economic, and technological challenges on a daily basis. Addressing these challenges would be impossible without noticeable management of the performance of their business. It is essential to every business, no matter the size, scope, or resources to identify, measure, understand, and control the progress of that performance [1,2]. Measurement systems are needed to set organisational goals and to control the improvements by monitoring the effectiveness and efficiency [3]. A common approach to carry out these measurements is through the use of KPIs metrics. KPIs provide an objective criterion for measuring business activities and project success [4] and are a remarkably important part of corporate strategy for forecasting, measuring and planning business [2].
However, it should be noted that performance metrics vary in their purpose, definition, and content. Therefore, different methodologies are used to define and select the business KPI’s, to make sure that they match the competitive environment and strategy. According to Iuga et al. [2], three important criteria need to be taken into account for the optimal selection: Validity, helpfulness, and relevance; and to ensure that the measurement system is under control a disciplined methodology is needed.
The lean approach has been explored for defining, measuring, and monitoring performance. The methodology focuses on continuous process performance improvement and enables decision making based on real facts and data analysis, information and objective evidence gathered through quantification and estimation methods [2,5]. The lean concept is characterised by managing the efficiency and effectiveness of the organisation, by putting the emphasis on customer value and waste reduction.
So, lean consists of a set of principles and tools that have been implemented in a broader range of industries [6,7]. Although its origin was within manufacturing, it has also been applied in non-industrial organisations such as services [8], banking, or healthcare [9]. Also, lean focus has expanded in a broad range of disciplines like sales, product development [8], project management (PM), and Research and Development (R&D) [6,10]. Although R&D is a new concept, some studies on lean R&D have been published, especially in the field of healthcare [6,7,11].
Despite the fact that the mentioned concepts have been studied, a lack of homogeneous process for identification and measure of KPIs was detected. Although the adoption of KPIs in order to achieve objectives within the production environment is widely spread [8], there is scarce research published on developing a model to identify the KPIs in project-based companies.
Literature concerning KPI’s in a project-based organization context is generally related to project success [4,12,13]. The measures on which the success or failure of a project is judged are the success criteria and the KPIs are the factors that constitute those success criteria [4]. Project management KPIs are crucial as they enable the progress of projects to be monitored. It must be ensured that KPIs are aligned with the organisation’s strategies, that the perspectives of all stakeholders are considered and that short- and long-term benefits are covered [12]. Regarding the relationship between project management performance and project success in project-based organisations the KPI’s of the project management are the most significant variables for the success of a project [12].
Measuring R&D and innovation (R&D&I) project performance and if a project is successful or not in project-based companies has become a fundamental concern for managers and executives in the last decades. As a result, the issue has been extensively debated in the literature. However, determining whether an R&D&I project is successful is a subtle matter [14] and a challenging task. If someone is able to reach the top performance on your project, it does not guarantee that particular project will be successful.
R&D&I projects are complex per se, with several dependent phases that makes it even harder to determine project success factors criteria. It is clear that projects usually have multi-dimensions, and that different people involved in the project have different priorities. Therefore, not only should several dimensions for assessing the project success be considered, but also the fact that project success may vary over time based on different people’s interest [15]. Davis [16] stated that different project success dimensions (PSDs), such as time, mission and objective, project manager competencies, strategic benefits, and top management support, have different importance for different stakeholders.
This paper justifies and proposes the use of lean principles and techniques to identify KPIs in project-based organisations. A qualitative approach, based on SRL, was adopted, which was used to analyse and compare research about project success and the use of lean for the identification of performance metrics. Using the SLR output, a lean-based KPI model is proposed with the aim of being a guide for the identification of performance indicators in project-based organisations. For the elaboration of this model, Lean Six Sigma process so-called DMAIC (define–measure–analyse–improve–control) as a cycle-based approach [17] was followed and adapted. Later, the model was validated through a case study in a R&D project-based organisation. Since the focus of this paper is on the identification and categorisation of KPIs, the validation, adequacy, and control of KPIs are recommended as future research.
This research will answer practical questions about how lean tools and principles lead to KPIs identification and which are the benefits from it. Additionally, it will be shown how a model for the identification of KPIs developed with lean will benefit project-based organisations.
In Section 2, we will expand the research methodology and how it leads us to the Literature Review (Section 3). Next, in Section 4, the definition and description of the model will be discussed, followed by the description of the Case study in Section 5. The results of the utilisation of the model will be shown in Section 6, and general conclusions and future research will be given in Section 7.

2. Research Methodology

To identify and examine the current state of the art of the research topic an SLR was followed. The process starts with the description of the research questions stated in the Introduction. Then, databases to be used to search the publications were chosen. To fill the keywords in the databases, Boolean “AND” operator was used to combine the keywords and to focus the results of the search. The range of years was not specified in order not to limit the number of publications. Nevertheless, it was observed that most of the research done on the subjects in question was published during the previous 20 years.
In the third step of the SLR, the documents identified were selected according to the exclusion and inclusion criteria. The central focus of the study was the relationship between project success, lean, and performance indicators in project-based context. In the scope, it was included subjects such as the use of lean for the identification of performance indicators; the use of lean tools and principles for KPI development; project success criteria; and KPIs in project-based companies. This last subject was selected to establish a context for the case study. Different types of documents were sought for distinct parts of the investigation. The SLR process continued with the fourth step of analysing the publications through thematic analysis and synthesising the information gathered. The last step consisted of reporting and using the findings, which will be discussed in the Literature Review section.

3. Literature Review Discussion

Lean is defined in many ways, some authors call it methodology, others philosophy [18], yet there is a common consensus on its approach. In the literature, lean is explored from two broad ways: Strategic and operational. The first is associated with lean principles and goals, while the second refers to practical aspects related to its implementation, tools and techniques [5].
Lean is based on five fundamental principles: Defining value from the customer perspective, identifying the value streams, making the value flow, implementing pull-based production, and striving for perfection continuously [5]. It also focuses on waste elimination by improving process performances and value creation [5,8,18]. The term waste in lean context is defined as an event or process that does not reflect customers value or generate any added value to the final output [2]. Seven types of waste are usually described in lean literature: Transport, inventory, motion, waiting, over-processing, overproduction, and defects [19]. Other researchers include an eighth kind of waste, the unused people’s creativity [2]. Cherrafi et al. [19] highlighted another concept to be considered, which is sustainability (economic, social, and environmental concepts) due to an increase in the environmental and social awareness.
The lean approach to customer value and the elimination of waste leads to the identification of an organisation performance indicators. Waste has a direct impact on performance [19]; hence, some authors have researched on the definition of KPIs based on these types of waste. Iuga et al. [2] explored KPIs selection criteria based on the same waste categories. The link between KPIs and the lean waste concept enables a wider perspective on the performance assessment analysis [18].

3.1. Lean in Project-Based Organisations/R&D

Even though lean in R&D is a rather new approach [6], there are studies from years ago that have explored the use of lean for performance improvement within this discipline. Marti [11] explains how Lean Six Sigma leads to a better understanding of crucial customer requirements providing more value to services and to focus on improving critical areas of the R&D process. On the other hand, recent studies have also shown a positive impact in the implementation of lean within this area. Foruhi et al. [10] demonstrate how lean principles and tools in R&D organisations can improve their key skills by increasing efficiency and reducing waste and therefore costs. Al et al. [6] developed a model using lean to map and improve the functions of R&D project activities. Foruhi et al. [10] identified and determined the customer value as the main focus of lean concepts and how can be applied to all disciplines including R&D. Hence, Panat et al. [20], through a case study, demonstrated the benefits of using lean combined with Six Sigma methodology in the infrastructure and operations of the R&D organisation.
Lean can be used in conjunction with other improvement methodologies such as Six Sigma. Lean Six Sigma has been studied as a business strategy and methodology to measure and improve operational performance [11]. By integrating the tools, techniques, and principles of both methodologies, it achieves to eliminate defects in processes and improves process performance focusing on customer value [11,19]. Lean Six Sigma is also used on projects with the aim of improving the process through workflow creation and elimination of variation [21].

3.2. The DMAIC (Define, Measure, Analyse, Improve, Control) Methodology

The DMAIC is a Lean Six Sigma method consisted of five process phases: Define, measure, analyse, improve, control. In this study, the central idea behind DMAIC process is used for the creation of a lean KPI model by remaining focused on customer value.

3.2.1. Define Phase

In this phase, the scope of the project is aligned with the organisation strategy to detect in which aspects the performance meets or not the customer needs [11]. One of the lean tools for identifying the needs and requirements is the voice of the customer (VoC) [22]. The information captured with the VoC can be used to identify performance indicators.
Through VoC, critical requirements of the client and what he considers as value can be identified. By collecting the customer’s needs, the information can be structured in a hierarchical way prioritized in terms of relative importance and customer perceptions of performance [22,23]. In addition to the VoC, there is the voice of the business (VoB), which concerns what the organisation aims to achieve [17].

3.2.2. Measure and Analyse Phase

During the measure phase, the VoC and VoB specifications are translated into measurable and controllable factors (quantitative data) through the critical to quality (CTQ) tool. The CTQs are specific quantifiable metrics that are linked to the organisation objectives [24]. This tool displays the customer’s expectations towards the quality of a product. The CTQs are represented with a flow-down tree and when applied for performance indicators, some authors refer to them as CTQ [25]. Figure 1 shows the generic representation methodology or a project where there is an input (e.g., requirements, statements) provided by a stakeholder (e.g., supplier, customer), where there is a process to deliver an output [13]. In Lean Six Sigma, there is a similar mapping of a flow process known as SIPOC (supplier, input, process, output, customer) [20]. This diagram also works as a guide to identify metrics as KPIs. In CTQ context, the outputs are represented by Ys and the factors that impact them, and the inputs by Xs [21].
The CTQs are considered as performance indicators [21]. KPIs, within the VoC, correspond to the CTQ characteristics, which are a set of indicators with clear targets and specified limits [17]. Yang et al. [25] specified the CTQ-Y as the KPIs of the CTQ. The specifications of the CTQs are the measures of the dependent variable (Y) and the Xs are the key variables or drivers. The factors affecting the CTQ can be represented by the Equation (1).
Y = f(X)
The current state of the CTQ is specified and the performance measures or key variables (Xs) are searched. In this phase, the potential Xs can be identified through a value stream map (VSM) [21]. VSM is a process flow chart that identifies the added-value and non-value-added activities in a stream product transformation process [5]. The VSM reveals hidden issues in the process, brings options to the surface, and enables the potential to maximise performance by eliminating the waste.
After defining and understanding the process, and having specified and documented the performance measurements, we proceed to the analysis phase. In this phase, the critical factors directly related to the Ys are established. The analyses of the data and process activities allow the detection of the main factors that have an impact on quality from the customer’s perspective (value-added) [11].

3.2.3. Improve and Control Phase

The improve phase is for implementing the proposed methods and improvements [18]. To control and monitor the progress, Cortes et al. [18] recommended the use of a web application (e.g., customised dashboard) in order to access the required standards and project management tools. Since the focus of this paper is on the identification and categorisation of KPIs, the validation, adequacy, and control of KPIs are recommended as future research in the Conclusions section.

3.3. Key Performance Indicators (KPIs)

KPIs are metrics used by some organisations to track the success and guide their progress towards specific strategic objectives. In addition to business strategy, it is important to consider the corporate culture since there is a direct connection between the organizational culture and performance. A solid corporate culture drives the performance and each specific feature impacts the strategy’s implementation [26]. Popa [26] highlights that in order to develop KPIs, four main factors must be taken into account. First, ensuring that activities are in line with the objectives; second, collecting the required information to improve the activities; third, controlling and monitoring the activities and the people involved; supporting the reports for the stakeholders.
In a project context, the KPIs are linked to the project success or failure [13]. These indicators have to be measurable and controllable therefore must be quantitative and qualitative [4,13]. All organisations are different and have their own needs and objectives, so KPIs must be tailored to each one; nevertheless, a general framework can be useful as a guide [4]. Kerzner [13] stated that a KPI’s most crucial attribute is that it is actionable, meaning that actions can be taken to correct any unfavourable trends.
Within the literature review, several descriptions of KPIs categories were detected. Cortes et al. [18] cite five strategic KPIs categories: Cost, quality, flexibility, stock, and lead time. With these categories, they aimed to capture the company strategic goals and enable the alignment of strategic, tactical, and operational performance. Ogunlana [4] identified several authors that included other performance measurements in addition to the classic iron triangle (time, cost, and quality): Customer satisfaction and overall satisfaction of stakeholders.
Within projects, Kerzner [13] identifies time, cost, resources, scope, quality, and actions as core metrics for project management KPIs. Additionally, Ogunlana [4] delves into performance measurements adding the capacity of the project team to manage project risks and solve problems found in projects in order to evaluate the success of the project. The authors also mentioned other research that suggested measuring project success by the technical performance efficiency of execution, managerial and organisational implications, personal growth, manufacturers’ ability, and business performance.
Furthermore, España et al. [1] argues that conventional metrics such as cost, schedule, quality, and security should be used to support the system improvement and not as isolated parameters that request an individual response. The authors explain that evaluating together cost and schedule indicates whether the system is stable or corrective actions are needed, while the cost in conjunction with safety parameters suggest if the work is planned and performed correctly [1].
Ogunlana [4] highlight as future research to focus on the integration of the organisations KPIs with the following aspects: “operational (time, cost and quality), life cycle (maintenance capacity, energy consumption and user satisfaction), strategic (inter-organisational co-operation, organisational learning) and socio-economic (social and human development)”. Moreover, the authors assured that the criteria for measuring the success of the projects should be based on strategy, sustainability, and safety. Yang et al. [25] describe two categories of KPIs: Financial (e.g., increased sales and decreased material, inventory, and transport costs) and operational (e.g., cycle time, utilisation rate, delivery time, forecast accuracy). All the categorisation mentioned by the authors is summarised on Table 1. These categories are an example of the categories that may be considered within an organisation.
Cortes et al. [18] proposed a Lean Six-Sigma framework based on lean indicators for management support during lean implementation intending to lead tactical and operational decisions for performance improvement and maintenance. Lean tools such as the “five whys” and the root cause and effect analysis can be used for establishing performance indicators and for identifying improvement metrics.
The work of Dombrowski et al. [8] proposed specific criteria of a measurement system for performance indicators in product development context. The criteria to be taken into account consists in: Relevance for the enterprise targets, quality data (based on the validity and timeliness of data), compatibility with the hierarchy, variability (react quickly to changes), periodicity, visualisation, and effort. Furthermore, Kerzner [13] describes six fundamental characteristics for project-oriented KPIs: Predictive (future), measurable (quantitative), actionable (changes to correct), relevant (relationship to project success/failure), automated (reports minimise human error), and few in number (those needed). Table 2 shows the characteristics that the KPIs should accomplish according to the literature analysed.

3.4. Project Success

As has been said, project success as the heart of project management, and the factors that affect it, is a commonly discussed topic in research in project management that has been studied by practitioners and academics since 1960s (it started with the definition of success in terms of the iron triangle, time, cost and quality [27]; and remains relevant to the present day); however, there is not a unified definition [15]. Moreover, we can distinguish two different components: Project success factors: Elements of a project that if influenced, increase the like hood of success (e.g., stakeholder, risk and quality management, etc.) and project success criteria: Measures used to judge on the success or failure of a project (e.g., stakeholder satisfaction, cost, scope, time, etc.) [28].
The first impression of project success is a project implemented among the constraints of time, cost, and quality; however, project success is more than implementing the project within this iron triangle. Constrains and exceeds from those boundaries do not mean that the project was or not successful. Public opinion considers the Sydney Opera House as a successful project, even if it was 14 times over budget and time [29]. Some researchers carried on upon that project and other similar ones led to highlighting two main issues. First of all, the differences in the perception of project success among different stakeholders, and secondly, the fact that project success is beyond controlling the iron triangle constraints and that more dimensions should be considered.
Based on questions like “What factors lead to project management success?” or “What factor lead to a successful project?”, Cooke-Davies [30] defined the success criteria (SC) as indicators for measuring the success or failure of the projects, and identified 12 factors that are in one way or another, critical to project success. As well, Lim and Mohamed [28] helped to define the success factors as the set of principles or standards for judgement about the success of a project. Among the top frequently cited on the literature we can identify support from senior management [31], clear and realistic objectives [27,32], strong/detailed plan kept up to date [33], good communication/feedback [30], among others. Top management support was introduced as SF by Müller and Turner [31], and by Pinto and Slevin [27], project ownership [34]. Also, Chan et al. [35] stated that project team commitment, contractor competencies, risk and liability assessment, client competencies, users’ needs, and constraints imposed by users are project success factors (PSFs).
There are plenty of studies about the PSFs, each of which represented a wide range of success factors; however, these factors are usually listed or in a very general way or, with such specificity that can only be applied to a particular variety of projects. Nonetheless, in their research, Belassi and Tukel [33] stressed grouping success factors and explain the interaction between them, putting aside the focus from the identification or specificity of such individual factors.
Although varied PSFs are introduced through numerous studies, Cooke-Davies [30] stated that finding the projects’ real success factors is important. However, some of this PSFs are extremely important and must exist to ensure the project success; these are called critical success factors (CSFs). In their work, Fortune [36] carried out a review of a series of publications that focused on CSFs, allowing them to identify which were the most frequent success factors mentioned on theoretical or empirical studies and the different stages of the project where these factors were evaluated, letting them conclude that the evaluation of some success factors on different phases of a particular project can help to determine if it is going to be successful or not. Pinto and Slevin [27,32] represented a list of CSFs, including: Technical tasks, client acceptance, power and politics, communication, client consultation, top management support, urgency, environmental events, and characteristics of the project manager, troubleshooting, and personnel recruitment.
As illustrated in Table 3, various factors contribute positively or negatively to project success. Nevertheless, analyses of all success factors are extremely hard. Therefore, some authors grouped the CSFs under “success dimensions”, which make the evaluation and interpretation of project success more understandable. Different authors offered different dimensions to be used as criteria to judge project success.

4. Defining a Lean-Based KPIs Identification Model

In this study, a model for the identification of KPIs in a project context was developed. For the design of the model, the theoretical concepts previously analysed were used as a basis. Additionally, concepts from success factors and lean models related to performance measurement identified in other studies were considered in the design of the model.
Leading KPIs are established in a way that impacts the most relevant results of the organisation. Although the main indicators vary from one organisation to another, a process based on lean thinking can determine the KPIs of a company. The first step for defining the KPIs is to identify the organisation strategic objectives and the different impacted levels [18]. Cortes et al. [18] proposed a KPI classification, based on the work of Pakdil and Leonard [43], and Gopinath and Freiheit [44], in line with the main lean fundamentals. Roberts and Latorre [45] in their research made a critical analysis of the KPI measurement system. The model proposed is based on selecting the categories from SLR, so the needs identified from the customers (VoC) and the organisational strategies and objectives (VoB) can be set in an organised form. Once that process is completed, the identification of the parameters to be measured as a driver to accomplish the future KPI. Next, the identification of the CTQs gives the information required to raise the performance indicators. Following this, we propose a tie in a measurement to that indicator (e.g., number of, percentage of, amount of, etc.), and finally, the organisation goal must be set. The proposed model to defining and establishing KPIs for projects is resumed in Table 4.

5. Case study—CFAA (Advanced Manufacturing Centre for Aeronautics)

Looking for strategies to increase the innovative capacity of universities and thus boost the local economy, the collaboration between companies and universities has been an institutional strategy used to guide ideas, inventions, and innovations generated in universities and transmit them to the industrial and social network [46]. The machine-tool and advanced aeronautical manufacturing sectors have been some of the strategic areas for the economy of the Basque Country in Spain over the last few decades, generating a turnover of up to 3.83 billion Euros [47]. In this context, the creation of a Research, Development, and Innovation (R&D&I) Centre with focus on advanced aeronautical manufacturing technologies that could integrate these two sectors with the University and allow the easy and fast transmission to the industrial production ecosystem associated with the value chain was a fairly straightforward decision [48]. The result was the Centre for Advanced Manufacturing in Aeronautics (CFAA, in Spanish), an open and shared space for researchers, students, and professionals of the sector, companies, and research centres at national and international level, where applied knowledge, technologies and new methodologies for the previously mentioned sectors are being developed.
CFAA was created to operate within the structure of the University of the Basque Country /Euskal Herriko Unibersitatea (UPV/EHU), and emerges from the agreement signed between the UPV/EHU and a group of aeronautical and machine tool companies, and is supported in its origin, constitution, and in the acquisition of facilities and machinery by the Regional Government of Bizkaia and by the Society for Competitive Transformation (SPRI)
The R&D&I projects carried on at CFAA are located between Technology Readiness Level (TRL) 5–7 [49], which guarantee a quick knowledge transfer, and promote mutual benefits to companies and universities from a micro up to a macro-economic point of view, so that universities obtain funding to conduct their research and train staff to enable them responding positively to the demands of the labour market [50]. In addition, it advocates for fostering the relations between companies from different sectors, and in the university–business axis [51,52].

Current Situation

The role of the CFAA within the projects is, on the one hand, to filter out projects that are in line with their strategy, and from those that result, test their economic, scientific, and technical feasibility in an industrial environment designed and equipped to simulate a real factory. On the other hand, CFAA on its own, and thanks to the interaction of different scientific groups, proposes, develops, and tests advanced manufacturing technologies, techniques, and applied knowledge, born from state-of-the-art research.
CFAA is a project-based organisation, where the success of their projects takes great relevance and a determinant role in the present and future opportunities for the Centre, in the form of being able to participate, or present itself to European calls, taking part in international and specialised clusters, or attracting young and professional talent. However, some studies carried on CFAA had demonstrated that there are still significant improvement opportunities to measure and improve the rate of success in this Centre [53].
To date, almost 300 projects have been carried out with a success rate close to 70%, meaning that the results obtained are being used by the company (or group of companies) that leads the project. This result is thanks to close co-operation between researchers, technicians, University experts, project managers and company staff, collaborating on the prevention of wrong pathways taken at early stages of the project.
Since its inception, CFAA has been committed to boosting scientific activity and contributing to various scientific publications e.g., journals with different impact rates, conferences, book chapters, etc. From 2017, the impact of the CFAA and its manufacturing groups is reflected on more than 140 scientific publications (March 2020) which have generated more than 840 citations. Also, 11 doctoral theses have been developed. Currently, CFAA is working on the implementation of a project management methodology developed ad hoc for the Centre, whose objective is to manage projects, programmes, and portfolios, and push the organisation towards more agility and efficiency [54].
Despite the good state of the CFAA regarding its scientific production, use of resources, collaboration with institutions, and participation in co-operation projects at European level, several of the general objectives set for the Centre [55] and those described in the Centre’s Quality Manual, are not being adequately studied, described, and measured.
To date, a few quantifications of consumed hours in projects, number of projects developed, and hours spend using the resources of the CFAA, are being measured (Figure 2). Leaving room for a new set of KPIs oriented to measure the performance of organisational and production needs.

6. Applying the Lean-Based KPIs Identification to the Case Study

Burimova et al. [56] carried on a study in order to obtain a clear picture of what “Success” means for CFAA and their most important stakeholders. The study was constructed following one of the most critical remarks behind project success, which is that the success of the project depends on which area (or dimension) is inspected, therefore these areas need to be addressed individually, including not just the business side, but also people management. Consequently, it is crucial to define and evaluate the success criteria at different dimensions. For that study, the success dimensions chosen were:
  • Project Management Success Dimensions (PMSD): Focus on the aspects that are necessary for the right management of the project, e.g., control of time, cost and scope, compliance of quality standards, resources and stakeholder’s management, etc.
  • Delivery Activities Success Dimensions (DASD): Focus on the processes that are used to create the deliverables.
  • Deliverable Success Dimensions (DSD): Describing the output of the project, including the success criteria needed to verify that the final result of the project matches the scope.
  • Operations Success Dimensions (OSD): Includes the success criteria needed to verify that the operation ensures that the ongoing process of the project is carried on in an appropriate way.
Once the study was conducted, the resume of the results of the most important success factors can be seen at Table 5.
The results have shown that the most important success factors are related to the safety of the workplace, also if the project goal was achieved and how the customer feels about the quality of the different deliverables. These results are quite valuable as an input of the strategic needs that CFAA must necessarily focus on.
The analysis of the organisation strategies and needs were found seeking for CFAA documents with relevant information that may guide the identification of performance indicators, e.g., data from a survey about project success for stakeholders and personnel from the organisation, and a benchmarking with a similar Centre like CFAA. Later, a stakeholder validation was carried out to find out their perception of the performance indicators.

6.1. Ruhrvalley Innovation Cluster

As has been mentioned before, the performance parameters used by centres alike need to be analysed in order to compare and understand the rationalisation of their measurement parameters from a more mature point of view. For this research, “ruhrvalley” was chosen among different innovation centres due to different aspects:
Region: The companies that conform the ruhrvalley innovation cluster are SMEs focused on small but very advanced niches that some of them, regardless of their size, stand as world markets leaders in their domain.
Cluster: ruhrvalley was formed to provide wider solutions in areas like eMobility, renewable energy systems, and digital transformation.
Impacts: The expected impact on solutions for urban mobility and energy systems, and the innovation community in the region, develop a strong innovation profile for universities and development of technology-driven start-ups and SMEs on the region.
Collaboration: An effective interaction model between leading universities and research centres, scientific and industrial associations, and important players from society and politics [65].
ruhrvalley seeks a combination of applied research, academic education with a strong application focus (high TRL), industry co-operation, and a strong rivet on innovation and entrepreneurship [66], as does CFAA. Sastoque et al. [51] mentioned that CFAA and ruhrvalley have had a strategic matching to increase an impact of R&D results in the scientific community. They also have some contextual factors in common, like their location, due to both of them being placed in industrial urban areas under processes of digital transformation through technology- and innovation-driven approaches. Regarding the TRL, ruhrvalley goes one step further than CFAA, with TRL 6–7 for CFAA and TRL 6–8 for ruhrvalley. Also, they both focus on Industry 4.0 technologies.

6.2. KPIs Construction

In order to assemble the KPIs for CFAA, the classification was made based on the study carried by Wolff and Nuseibah [65] and divided on four different scopes: Innovation, region, research and development, and university. Next, the strategic needs for each one of those scopes were inferred from the information sources previously mentioned Table 6.
At the end of the analysis of the information gathered, besides these 12 strategic needs, 51 operational needs and 56 internal factors (drivers) were also identified. However, for this paper, the result of the study of the strategic and operational needs for the scope of innovation will be shown.
The CTQ started by identifying the critical Xs for the first strategic need (Y): Creating and marketing new services, product lines, and technological capabilities. Next, the specific measurements required to fulfil the quality requirements were previously identified. A target value for each of the measurements must be set according to the organisation expectations (Table 7; Table 8).

7. Conclusions and Future Research

Lean thinking can lead to knowing an organisation in a deeper way, asking questions that were overlooked, and reaching conclusions of highly strategic value for the company. In this research, through an SLR, the literature on the process of KPI definition was analysed through a lean approach in project-based organisations.
Through the concepts gathered in the SLR, a classification of KPIs was identified, and a lean-based model was developed to define the KPIs. This model was applied to a case study conducted within a R&D project-based organisation (CFAA). Following a series of steps based on lean tools and principles, the model worked as a guide for the identification of KPIs. The DMAIC methodology phases were used in order to keep an organised process flow for the model implementation.
The first step was to establish the current status of the company as well as explore the stakeholders needs and requirements with the VoC. To complement the initial information and to determine the strategic needs (VoB), the company quality manual was reviewed. The literature highlights the importance of KPI formulation based on strategic business objectives as well as corporate culture. Although this is the starting point towards the identification of performance indicators, the scope of this research does not include the analysis of corporate strategy and culture. Furthermore, it was considered useful to benchmark with a similar leading organisation (ruhrvalley) in order to get baseline performance indicators. After obtaining the data to define the CTQs, we proceeded to measure and analyse the internal factors, measurements, targets, and performance indicators that constitute the KPIs.
The result was a series of qualitative and quantitative KPIs that evaluates the strategic and operational needs of a project-based organisation and helps to understand and improve their performance criteria. However, those defined KPIs cannot be unmovable. The continuous changes in the market or the research methodologies require KPIs to be constantly redefined and updated, in order to ensure that KPIs are suitable for the current environment of the organisation [18].
The scope of this research was limited to proposing a model for the identification of KPIs using the lean approach, therefore for future research, it is suggested to continue with the following actions of validating, communicating, reporting, and controlling the adequacy of the KPIs. “Visual management” is a suitable technique to manage the quality of the KPIs [2]. For managers and project managers, the use of KPIs “dashboards” are recommended during to report, monitor, and control the KPIs [11].
As a next step, a project plan must be developed to ensure the right applicability of the KPIs at CFAA and a system to measure their effectiveness in the organisation. Additionally, the integration of the information coming from different areas of the organisation (IoTs, edge computing devices, project reports, etc.) to ensure the correct functioning of the KPIs should be assured. It is a process as important as the formulation of the KPIs itself, for the use of dashboards, scorecards, and reports that show information almost in real time is highly recommended.
The result of this research is of high value for project-based organisations, especially those new ones that are not apprised of how to correctly formulate a series of KPIs, or whose path is still not clear.

Author Contributions

C.C.V., L.S.P., and J.R.O.O. proposed the methodology; N.T.G. and N.L.d.L. gave ideas and analysis of CFAA KPIs; C.C.V., L.S.P., and N.L.d.L. participated in writing the article. All authors have read and agreed to the published version of the manuscript.

Funding

Authors are grateful to Basque government group IT IT1337-19 and the Spanish Ministry of Science Mineco REF DPI2016-74845-R and project PID2019-109340RB-I00.

Acknowledgments

The authors wish to express their gratitude to all those who have contributed to the study described in this article, with special thanks to the School of Industrial Engineers of Bilbao (University of the Basque Country UPV/EHU), and at the Centre of Advanced Aeronautical Manufacturing “CFAA” of Bizkaia. Special thanks to all people working hard against COVID19, medical people, nurses and all the people staying at home making all of us safer.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. España, F.; Tsao, C.C.; Hauser, M. Driving continuous improvement by developing and leveraging lean key performance indicators. In Proceedings of the Annual Conference of the International Group for Lean Construction, San Diego, CA, USA, 18–20 July 2012. [Google Scholar]
  2. Iuga, M.V.; Kifor, C.V.; Rosca, L.-I. Lean information management: Criteria for selecting key performance indicators at shop floor. ACTA Univ. Cibiniensis 2015, 66, 67–72. [Google Scholar] [CrossRef][Green Version]
  3. Zakaria, Z.; Yaacob, Z.; Noordin, N.; Sawal, M.Z.H.M.; Zakaria, Z. Key performance indicators (KPIs) in the public sector: A study in Malaysia. Asian Soc. Sci. 2011, 7, 102. [Google Scholar] [CrossRef]
  4. Ogunlana, S.O. Beyond the ‘iron triangle’: Stakeholder perception of key performance indicators (KPIs) for large-scale public sector development projects. Int. J. Proj. Manag. 2010, 28, 228–236. [Google Scholar]
  5. Thangarajoo, Y.; Smith, A. Lean thinking: An overview. Ind. Eng. Manag. 2015, 4, 1000159. [Google Scholar] [CrossRef][Green Version]
  6. Al, E.; Ali, Z.; Türkyılmaz, A.; Zaim, S. Lean principles in R&D projects. In Proceedings of the Global Conference on Engineering and Technology Management 2014, Istanbul, Turkey, 23–26 June 2014. [Google Scholar]
  7. Madsen, D.Ø.; Berg, T.; Stenheim, T.; Moum, J.V.; Bordewich, I.O.; Storsveen, M. The long-term sustainability of lean as a management practice: Survey evidence on diffusion and use of the concept in Norway in the period 2015–2017. Sustainability 2019, 11, 3120. [Google Scholar] [CrossRef][Green Version]
  8. Dombrowski, U.; Schmidtchen, K.; Ebentreich, D. Balanced key performance indicators in product development. Int. J. Mater. Mech. Manuf. 2013, 1, 27–31. [Google Scholar] [CrossRef][Green Version]
  9. Majerus, N. Leveraging lean principles in R&D: The experience of The Goodyear Tire & Rubber Company shows how applying to Lean principles to R&D can improve the efficiency and effectiveness of new product development. Res. Technol. Manag. 2017, 60, 17–25. [Google Scholar]
  10. Foruhi, T.; Behzad, M.; Amiri, Z.A.; Felekari, M.; Havangi, H. Employing lean concepts and tools in innovative and R&D based organizations. Int. J. Appl. Optim. Stud. 2018, 1, 71–85. [Google Scholar]
  11. Marti, F. Lean Six Sigma method in phase 1 clinical trials: A practical example. Qual. Assur. J. Qual. Assur. J. Pharm. Health Environ. Prof. 2005, 9, 35–39. [Google Scholar] [CrossRef]
  12. Mir, F.A.; Pinnington, A.H. Exploring the value of project management: Linking project management performance and project success. Int. J. Proj. Manag. 2014, 32, 202–217. [Google Scholar] [CrossRef]
  13. Kerzner, H. Project Management Metrics, KPIs, and Dashboards: A Guide to Measuring and Monitoring Project Performance; John Wiley & Sons: Hoboken, NJ, USA, 2017; ISBN 1119427282. [Google Scholar]
  14. Bizan, O. The determinants of success of R&D projects: Evidence from American-Israeli research alliances. Res. Policy 2003, 32, 1619–1640. [Google Scholar]
  15. Shenhar, A.J.; Tishler, A.; Dvir, D.; Lipovetsky, S.; Lechler, T. Refining the search for project success factors: A multivariate, typological approach. R&D Manag. 2002, 32, 111–126. [Google Scholar]
  16. Davis, K. Different stakeholder groups and their perceptions of project success. Int. J. Proj. Manag. 2014, 32, 189–201. [Google Scholar] [CrossRef]
  17. Panagopoulos, I.; Atkin, C.J.; Sikora, I. Developing a performance indicators lean-sigma framework for measuring aviation system’s safety performance. Transp. Res. Proc. 2017, 22, 35–44. [Google Scholar] [CrossRef]
  18. Cortes, H.; Daaboul, J.; Le Duigou, J.; Eynard, B. Strategic Lean Management: Integration of operational performance indicators for strategic lean management. IFAC-PapersOnLine 2016, 49, 65–70. [Google Scholar] [CrossRef]
  19. Cherrafi, A.; Elfezazi, S.; Chiarini, A.; Mokhlis, A.; Benhida, K. The integration of lean manufacturing, Six Sigma and sustainability: A literature review and future research directions for developing a specific model. J. Clean. Prod. 2016, 139, 828–846. [Google Scholar] [CrossRef]
  20. Panat, R.; Dimitrova, V.; Selvamuniandy, T.S.; Ishiko, K.; Sun, D. The application of Lean Six Sigma to the configuration control in Intel’s manufacturing R&D environment. Int. J. Lean Six Sigma 2014, 5, 444–459. [Google Scholar]
  21. Kim, Y.W.; Han, S.H. Implementing Lean Six Sigma: A case study in concrete panel production. In Proceedings of the 20th Annual Conference of the International Group for Lean Construction, San Diego, CA, USA, 18–20 July 2012. [Google Scholar]
  22. Gaskin, S.P.; Griffin, A.; Hauser, J.R.; Katz, G.M.; Klein, R.L. V oice of the c ustomer. In Wiley International Encyclopedia of Marketing; John Wiley & Sons: Hoboken, NJ, USA, 2010. [Google Scholar]
  23. Ferrús, R.M.; Somonte, M.D. Design in robotics based in the voice of the customer of household robots. Robot. Auton. Syst. 2016, 79, 99–107. [Google Scholar] [CrossRef]
  24. Six Sigma Institute. Six Sigma DMAIC Process—Define Phase—Capturing Voice of Customer (VOC). Available online: https://www.sixsigma-institute.org/Six_Sigma_DMAIC_Process_Define_Phase_Capturing_Voice_Of_Customer_VOC.php (accessed on 15 June 2020).
  25. Yang, H.M.; Choi, B.S.; Park, H.J.; Suh, M.S.; Chae, B.K. Supply chain management six sigma: A management innovation methodology at the Samsung Group. Supply Chain Manag. Int. J. 2007, 12, 88–95. [Google Scholar] [CrossRef]
  26. Brînduşa Maria Popa. Challenges when developing performance indicators. J. Def. Res. Manag. 2015, 6, 111–114. [Google Scholar]
  27. Pinto, J.K.; Slevin, D.P. Critical success factors in effective project implementation. Proj. Manag. Handb. 1988, 479, 167–190. [Google Scholar]
  28. Lim, C.S.; Mohamed, M.Z. Criteria of project success: An exploratory re-examination. Int. J. Proj. Manag. 1999, 17, 243–248. [Google Scholar] [CrossRef]
  29. Wikipedia. Sydney Opera House. Available online: https://en.wikipedia.org/wiki/Sydney_Opera_House (accessed on 24 January 2020).
  30. Cooke-Davies, T. The “real” success factors on projects. Int. J. Proj. Manag. 2002, 20, 185–190. [Google Scholar] [CrossRef]
  31. Müller, R.; Turner, J.R. Matching the project manager’s leadership style to project type. Int. J. Proj. Manag. 2007, 25, 21–32. [Google Scholar] [CrossRef]
  32. Pinto, J.K.; Slevin, D.P. Critical factors in successful project implementation. IEEE Trans. Eng. Manag. 1987, EM-34, 22–27. [Google Scholar] [CrossRef]
  33. Belassi, W.; Tukel, O.I. A new framework for determining critical success/failure factors in projects. Int. J. Proj. Manag. 1996, 14, 141–151. [Google Scholar] [CrossRef]
  34. Baccarini, D. The logical framework method for defining project success. Proj. Manag. J. 1999, 30, 25–32. [Google Scholar] [CrossRef]
  35. Chan, A.P.C.; Ho, D.C.K.; Tam, C.M. Design and build project success factors: Multivariate analysis. J. Constr. Eng. Manag. 2001, 127, 93–100. [Google Scholar] [CrossRef]
  36. Fortune, J.; White, D. Framing of project critical success factors by a systems model. Int. J. Proj. Manag. 2006, 24, 53–65. [Google Scholar] [CrossRef]
  37. Angus, G.Y.; Flett, P.D.; Bowers, J.A. Developing a value-centred proposal for assessing project success. Int. J. Proj. Manag. 2005, 23, 428–436. [Google Scholar]
  38. Abdullah, W.M.W.; Ramly, A. Critical Factors in Project Success. In Proceedings of the Pacific Association of Quantity Surveyors (PAQS) Congress, Edmonton, AL, CA, 17–18 August 2009. [Google Scholar]
  39. Jugdev, K.; Müller, R. A retrospective look at our evolving understanding of project success. Proj. Manag. J. 2005, 36, 19–31. [Google Scholar] [CrossRef]
  40. Cserháti, G.; Szabó, L. The relationship between success criteria and success factors in organisational event projects. Int. J. Proj. Manag. 2014, 32, 613–624. [Google Scholar] [CrossRef]
  41. Project Management Institute. A Guide to the Project Management Body of Knowledge Project Management Institute, 6th ed.; Project Management Institute: Newtown Square, PA, USA, 2017; ISBN 9781628251845. [Google Scholar]
  42. Shrnhur, A.J.; Levy, O.; Dvir, D. Mapping the dimensions of project success. Proj. Manag. J. 1997, 28, 5–13. [Google Scholar]
  43. Pakdil, F.; Leonard, K.M. Criteria for a lean organisation: Development of a lean assessment tool. Int. J. Prod. Res. 2014, 52, 4587–4607. [Google Scholar] [CrossRef]
  44. Gopinath, S.; Freiheit, T.I. A waste relationship model and center point tracking metric for lean manufacturing systems. IIE Trans. 2012, 44, 136–154. [Google Scholar] [CrossRef]
  45. Roberts, M.; Latorre, V. KPIs in the UK’s construction industry: Using system dynamics to understand underachievement. Rev. Constr. 2009, 8, 69–82. [Google Scholar]
  46. De Falco, S. Monitoring the performance of university technology transfer offices: The bias control. Arch. Bus. Res. 2015, 3. [Google Scholar] [CrossRef][Green Version]
  47. AFM. 40 Young Talents Specialised in Machine-Tool and Aeronautics Received Their Diplomas at the CFAA. Available online: https://www.afm.es/en/news/40-young-talents-specialised-in-aeronautics-and-machine-tool-received-their-diplomas-at-the-cfaa (accessed on 15 June 2020).
  48. Sastoque Pinilla, L.; Llorente Rodríguez, R.; Toledo Gandarias, N.; López de Lacalle, L.N.; Ramezani Farokhad, M. TRLs 5–7 Advanced manufacturing centres, practical model to boost technology transfer in manufacturing. Sustainability 2019, 11, 4890. [Google Scholar] [CrossRef][Green Version]
  49. NASA. Technology Readiness Level. Available online: https://www.nasa.gov/directorates/heo/scan/engineering/technology/txt_accordion1.html (accessed on 20 April 2020).
  50. Poyago-Theotoky, J.; Beath, J.; Siegel, D.S. Universities and fundamental research: Reflections on the growth of university–industry partnerships. Oxf. Rev. Econ. Policy 2002, 18, 10–21. [Google Scholar] [CrossRef]
  51. Pinilla, L.S.; Bengfort, S.; Mikhridinova, N.; de Lacalle, N.L.; Wolff, C.; Gandarias, N.T. Patterns for international cooperation between innovation clusters: Cases of CFAA and ruhrvalley. In Proceedings of the 2020 IEEE European Technology and Engineering Management Summit (E-TEMS), Dortmund, Germany, 5–7 March 2020; pp. 1–7. [Google Scholar]
  52. Syreyshchikova, N.V.; Pimenov, D.Y.; Mikolajczyk, T.; Moldovan, L. Development of a risk management technique in strategic planning of universities: Case study of a Polytechnical Institute. Proced. Manuf. 2020, 46, 256–262. [Google Scholar] [CrossRef]
  53. Artelt, S. Mitigate the Risk of Project Schedule Overruns and Project Delay Chains in a TRL 5–7 Research Centre: Improving the Quality of Project Estimating and Scheduling in a Multi-Project Environment. Master’s Thesis, University of the Basque Country/Euskal Herriko Universitatea, Basque Country, Spain, 2020. [Google Scholar]
  54. Farokhad, M.R.; Otegi-Olaso, J.R.; Pinilla, L.S.; Gandarias, N.T.; de Lacalle, L.N.L. Assessing the success of R&D projects and innovation projects through project management life cycle. In Proceedings of the 2019 10th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS), Metz, France, 18–21 September 2019. [Google Scholar]
  55. CFAA. General Objectives—Aeronautics Advanced Manufacturing Center–UPV/EHU. Available online: https://www.ehu.eus/en/web/cfaa/helburu-nagusiak (accessed on 15 June 2020).
  56. Burimova, A.; Artelt, S.; Sastoque Pinilla, L.; Toledo Gandarias, N.; Lopez de Lacalle, N. Project success evaluation of a TRL 5–7 public-private research centre. Manuscript in preparation. 2020. [Google Scholar]
  57. Project Management Institute. Delivering Value: Benefits Focus in Project Execution. Available online: https://www.pmi.org/learning/thought-leadership/pulse/focus-on-benefits-during-project-execution (accessed on 20 April 2020).
  58. McLeod, L.; Doolin, B.; MacDonell, S.G. A perspective-based understanding of project success. Proj. Manag. J. 2012, 43, 68–86. [Google Scholar] [CrossRef][Green Version]
  59. Khang, D.B.; Moe, T.L. Success criteria and factors for international development projects: A life-cycle-based framework. Proj. Manag. J. 2008, 39, 72–84. [Google Scholar] [CrossRef]
  60. Dvir, D.O.V.; Sadeh, A.; Malach-Pines, A. Projects and project managers: The relationship between project managers’ personality, project types, and project success. Proj. Manag. J. 2006, 37, 36–48. [Google Scholar] [CrossRef]
  61. Müller, R.; Jugdev, K. Critical success factors in projects: Pinto, Slevin, and Prescott-the elucidation of project success. Int. J. Manag. Proj. Bus. 2012, 5, 757–775. [Google Scholar] [CrossRef]
  62. Martens, M.L.; Carvalho, M.M. Sustainability and success variables in the project management context: An expert panel. Proj. Manag. J. 2016, 47, 24–43. [Google Scholar] [CrossRef]
  63. Agutter, C. ITIL® Foundation Essentials–ITIL 4 Edition; IT Governance Ltd: Ely, UK, 2019; ISBN 1787781186. [Google Scholar]
  64. Project Management Institute. Success in Disruptive Times. Pulse of the Profession 2018. Available online: https://www.pmi.org/learning/thought-leadership/pulse/pulse-of-the-profession-2018 (accessed on 20 April 2020).
  65. Wolff, C.; Nuseibah, A. A projectized path towards an effective industry-university-cluster: Ruhrvalley. In Proceedings of the 12th International Scientific and Technical Conference on Computer Sciences and Information Technologies (CSIT), Lviv, Ukraine, 5–8 September 2017. [Google Scholar]
  66. Ruhrvalley. Mobility and Energy for Metropolitan Change. Available online: https://www.ruhrvalley.de/en (accessed on 22 June 2020).
Figure 1. Supplier, input, process, output, customer (SIPOC)—process representation (based on [21]).
Figure 1. Supplier, input, process, output, customer (SIPOC)—process representation (based on [21]).
Sustainability 12 05977 g001
Figure 2. Performance information Centre for Advanced Manufacturing in Aeronautics (CFAA).
Figure 2. Performance information Centre for Advanced Manufacturing in Aeronautics (CFAA).
Sustainability 12 05977 g002
Table 1. Key performance indicators (KPIs) categories list.
Table 1. Key performance indicators (KPIs) categories list.
KPIs CategoriesIndicatorsSources
FinancialIncreases in sales[25]
Decreases in material[25]
Inventory[25]
Transportation expenses[25]
StrategicCost[18]
Quality[18]
Flexibility[18]
Stock [18]
Inter-organisational cooperation[4]
Organisational learning[4]
Tactic [18]
Cycle time[25]
Utilisation rate[25]
Lead time[25]
Forecast accuracy[25]
Time[4]
Cost[4]
Quality [4]
Customer satisfaction[4]
Overall satisfaction of stakeholders[4]
Project (operational)Project team’s ability to manage project risks [4]
Ability to resolve project problems [4]
Efficiency of execution[4]
Managerial and organisational implications[4]
Personal growth[4]
Manufacturer’s ability [4]
Business performance[4]
Life cycleMaintenance capacity[4]
Energy consumption[4]
User satisfaction[4]
Safety [4]
Sustainability (socio-economic aspect)Social and human development in the area[4]
Table 2. KPI characteristics.
Table 2. KPI characteristics.
Dombrowski et al. [8]Iuga et al. [2]Kerzner [13]Toor & Ogunlana [4]
Actionable X
Automated X
Compatibility (hierarchy)X X
EffortX
Few in number X
Helpful X
Measurable XX
Objective X
PeriodicityX
Predictive X
RelevantXXX
TimelinessX
ValidXX
Variability (react quickly to changes)X
VisualisationX
Table 3. Project success dimension.
Table 3. Project success dimension.
Project Success DimensionReferences
Mission, top management support, schedule, client consultation, personnel, technical, client acceptance, communication, feedback, and trouble-shooting[27,32]
Communication, time, mission, project management competences, the project delivering the strategic benefits and top management.[16]
Policy and strategy, mission[16,30,37]
Project-related, human-related factors, process-related factors, input-related factors, output-related factors[37]
The future potential of the projects in terms of innovations, generating a new product line or new technological capability[15,16,38,39,40]
Project management factors: planning, scheduling, monitoring and control, quality management, and risk management[27,30,32,38]
Meet stakeholders’ expectations, benefit to the stakeholder group, client/customer specific, Client acceptance and consultant.[16,27,32,38,40,41]
Project efficiency, impact on customers, business and direct success and strategic potential (preparing for the future). Time and costs were considered as resources and quality as customers. Satisfaction in contrast to using them as separate entities[42]
Goals and objectives, performance monitoring, decision-maker(s), transformations, communication, environment, boundaries, resources, continuity[36]
Table 4. Lean-based model for identifying KPIs.
Table 4. Lean-based model for identifying KPIs.
CategoriesVoCDriversCTQMeasurementsTarget
Proposed KPIs categories detected in the SLRCustomer’s needParameters to be measuredThe performance indicatorsData at a single point in time, specific, measurableOrganisation’s goal
Organisation’s strategies
Table 5. Most important success factors for CFAA and stakeholders.
Table 5. Most important success factors for CFAA and stakeholders.
No. Success FactorsSuccess DimensionReferences
1Workplace SafetyOSD[57]
2Project goal was achievedPMSD[27,32]
3Customer satisfaction regarding the quality of delivery activities of the specific projectPMSD[58]
4Reputation of the organisation has increasedPMSD[59]
5Knowledge generation regarding project activities (e.g., tools, techniques, approaches, processes)DASD[60]
6Customer satisfaction regarding the management of the specific project.PMSD[61]
7Customer satisfaction regarding the deliverable.DSD[62]
8Degree to which the deliverable meets its intended purpose.DSD[63]
9Return on Investment of the projectPMSD[57]
10Workplace securityOSD[57]
11Completed within defined and agreed scopePMSD[31,64]
12Completed within defined and agreed timePMSD[31,64]
13Completed within defined and agreed costsPMSD[31,64]
Table 6. Strategic needs—CFAA.
Table 6. Strategic needs—CFAA.
ScopeStrategic needs
InnovationCreate and marketing new services, product lines and technological capabilities
Institutional support of SMEs with innovation impulses in the development of new business models
RegionCFAA as a place of interest for local and international partners to develop projects.
Increase the attractiveness of the Basque Country for research, innovation, employment and start-ups
Intensify knowledge transfer
Interact with society, strengthen actors and civil society
ResearchGather and generate experience, new knowledge and understanding from activities and management of the project (e.g., tools, techniques, approaches or processes)
Increase the impact of R&D in the scientific community
UniversityProfessionalizing project management
Promote compliance with the goals and objectives for which CFAA was created.
Securing the freedom, financial and personnel basis for research and transfer activities in the long term
Strengthen collaboration with University in the implementation and development of projects
Table 7. Strategic need: Creating and marketing new services, product lines, and technological capabilities (Y).
Table 7. Strategic need: Creating and marketing new services, product lines, and technological capabilities (Y).
Operational NeedCategoryDriversCTQMeasurementsTargets
Contribute to sustainability in the creation, design and result of deliverables from the projectsSustainabilityStrengthen strategic human resources development as a research priorityWatch over the rights and interests of the workers.Number of complaints received vs. attended.100% complaints received vs. attended.
Number of wellbeing activities programmed vs. realised100% wellbeing activities programmed vs. realised
At least one well-being activity programmed vs. realised
Number of Safety programme activities programmed vs. realised100% of Safety programme activities programmed vs. realised
85% of participation from personnel at Safety programme activities programmed vs. realised
SustainabilitySustainability through creation, design and result of deliverables from the projectsAvoidance of wasteNumber of good practices implemented in the use of waste100% compliance with good practices implemented in the use of waste
Hazard materials treatmentsKg per type of hazard materials treated100% of hazard materials treated
Notices received by the authority for mismanagement of hazardsCero notices received by the authority for mismanagement of hazards
Safety and security initiatives to avoid accidents or incidentsNumber of safety programme activities programmed vs. realised100% of safety programme activities programmed vs. realised
Number of security programme activities programmed vs. realised100% of security programme activities programmed vs. realised
Number of safety eventsAt least 5 safety events programmed per year
Number of security eventsAt least 5 security events programmed per year
Develop and implement marketing of the R&D service portfolioTechnicalMarketing of R&D and innovation service portfolioMarketing campaigns of R&D and innovation service portfolioMarketing campaigns of R&D and innovation per innovation At least one marketing campaigns of R&D and innovation per innovation
Generate new usable knowledge and engineering solutionsOperationalUsable knowledge and engineering solutions generatedNew methods vs. solutions published in scientific journalNumber of new methods vs. solutions published in refereed journals.At least 3 publication by each new methods vs. solutions produced at CFAA.
Number of new methods vs. solutions presented at refereed conferencesAt least 3 attendances by each new methods vs. solutions produced at CFAA
New patentsNumber of new patents registered100% of new patents registered
Number of articles published in refereed journals regarding uses of patents.At least 3 articles published in refereed journals.
Number of presentations at refereed conferences regarding use of patents. At least 3 presentations at refereed conferences.
Number of marketing campaigns per patentAt least one marketing campaigns per patent.
Increase demand orientation in transferStrategicTransfer-oriented demandR&D strategy workshop participations vs. OrganizedNumber of participations at R&D or innovation workshops85% of participations at R&D or innovation workshops
Innovative activities developedNumber of R&D or innovation workshop organisedAt least one R&D or innovation workshop organised by CFAA
Introduce continuous innovation managementTechnicalContinuous innovation management introductionInnovation projects developedNumber of projects developed characterised as “Innovative” from partnersSuccess rate > 80% of projects developed characterised as “Innovative” from partners
Number of projects developed characterised as “innovative” from CFAASuccess rate > 80% of projects developed characterised as “innovative” from CFAA
Table 8. Strategic need: Institutional support of SMEs with innovation impulses in the development of new business models (Y).
Table 8. Strategic need: Institutional support of SMEs with innovation impulses in the development of new business models (Y).
Operational NeedsCategoryDriverCTQMeasurementsTargets
Enable and encourage talent early onTechnicalFormation to encourage innovation skillsFormation to develop innovation skillsNumber of courses related//needed to encourage innovation skillsAt least 2 courses per year
Students at CFAA doing their TFM or TFGStudents participation at CFAANumber of new students doing TFM or TFGAt least 30 students per year
Students at CFAA doing On-the-job formationStudents at On-the-job formation at CFAANumber of new students doing On-the-job formationAt least 10 students per year
Expand existing cooperation into strategic innovation partnershipsStrategicCooperation in strategic innovation partnershipsPartner participation in projects% of Partners participation in projects70% of partners participating in at least one project per year
Effective cooperation development planPlan’s implementation30% of partners involved in the plan
Framework agreement with all partnersPartners involved in the Framework agreement30% of partners involved in the plan
Promote the foundation, establishment and accompaniment of spin-offsTechnicalFoundation, establishment and accompaniment of spin-offsSpin-offs from CFAANumber of new spin-offs from CFAAAt least 1 initiative per year
Foundation, establishment and accompaniment of spin-offsInitiatives enabled and encouragedInitiatives enabled and encouragedAt least 1 initiative per year
Business start-ups provokedStart-ups projects activatedNumber of recruitments of innovative start-ups and joint venturesAt least 1 initiative per year
Coordination and cooperation with start-up support entities (Universities, governments, private investors)Number of new cooperation agreements with associations and business development agenciesAt least 3 agreements signed per year

Share and Cite

MDPI and ACS Style

Cruz Villazón, C.; Sastoque Pinilla, L.; Otegi Olaso, J.R.; Toledo Gandarias, N.; López de Lacalle, N. Identification of Key Performance Indicators in Project-Based Organisations through the Lean Approach. Sustainability 2020, 12, 5977. https://doi.org/10.3390/su12155977

AMA Style

Cruz Villazón C, Sastoque Pinilla L, Otegi Olaso JR, Toledo Gandarias N, López de Lacalle N. Identification of Key Performance Indicators in Project-Based Organisations through the Lean Approach. Sustainability. 2020; 12(15):5977. https://doi.org/10.3390/su12155977

Chicago/Turabian Style

Cruz Villazón, Carolina, Leonardo Sastoque Pinilla, José Ramón Otegi Olaso, Nerea Toledo Gandarias, and Norberto López de Lacalle. 2020. "Identification of Key Performance Indicators in Project-Based Organisations through the Lean Approach" Sustainability 12, no. 15: 5977. https://doi.org/10.3390/su12155977

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop