Lean Maturity Assessment in ETO Scenario

: The obligatory path towards a lean manufacturing organization requires assessment and monitoring. However, a lean assessment framework is not yet available for the engineer to order (ETO) scenario. This work explored ten lean ETO maturity principles—identiﬁed from the literature— that take insight from three formally deﬁned sets (Toyota Way, lean construction, and lean product development principles). A practical assessment model was proposed based on the evaluation of ten lean ETO objective criteria (four with mathematical formulation) and was validated on a real industrial case. A problem-solving tool, including a new lean tool, called the Problem Focus Matrix (PFM), was also presented; this tool was aimed toward development of an integrated framework that would include the organization mission, management, and continuous improvement.


Introduction
According to Womack's definition from the 1990s, lean production (or lean manufacturing) is a set of methods and tools to identify and eliminate inefficiencies and waste at the enterprise and manufacturing level without requiring additional resources [1].
This work proposes a theoretical lean maturity assessment model, along with its application to a real case study, in the specific context of engineer to order (ETO).
A graphical representation of the paper structure is provided in Figure 1. Background information on lean manufacturing and ETO is presented in Sections 1.1 and 1.2, respectively. The relevant literature on both is reported in Section 1.3, focusing on the ETO lean maturity assessment. Section 2 describes the methods used in the formal definition of the ten lean principles retrieved from the literature and the generation of the ETO lean assessment model using ten relative criteria. In Section 3, the assessment model is further detailed and validated on a real case study. A problem-solving approach based on the evidence, which emerged in the application of the model, is also presented. The main results and research perspectives are exposed in Sections 4 and 5, respectively. Conclusions can be found in Section 6. Figure 2. CODP, also known as freeze point or order penetration point. This refers to the point in the value chain at which a customer triggers a company's activities [28]. All activities before the CODP share a forecast-driven, push-based approach (dotted lines/arrows). Activities after the CODP are characterized by a customer-driven, pull based approach (continuous lines/arrows).
From mass production to mass customization, the growing degree of personalization in production has focused efforts on selling margins, rather than on volume. This results in efficiency loss, which is unacceptable in highly competitive global markets [29]. This is a contributing factor to the growing interest in lean methods and tools in ETO organizations [29][30][31]. However, some barriers in the implementation of lean practices in an ETO scenario remain [32].
The purpose of this work was to define an assessment framework for the ETO scenario and provide an objective, absolute evaluation of the current lean maturity level. Additionally, the work purposed to monitor the evolution of lean methodologies within individual or multiple organizations.

ETO Lean Assessment
The culture of companywide assessment has grown more and more popular, leading to the diffusion of quality, health, safety, and environmental management protocols. One primary goal of this growing business culture is to measure system status with respect to given criteria, and enhance the system through feedback actions [33,34]. This paper benefitted from and contributes to the development of such culture.
A comprehensive review on the assessment of the implementation of lean practices within an organization can be found in [35]. Remarkable work has been carried out using quantitative, qualitative, and hybrid approaches to assess lean implementation and transformation (see [36][37][38]). The Leanness Assessment Tool (LAT) in [36] provided theoretical implications that helped us define the idea behind our presented model. In this work, the LAT was developed based on lean research, using both quantitative and qualitative dimensions to assess lean implementation. Among other contributions, one of the most rewarding was the Lean Enterprise Transformation Maturity Model (LESAT). This model is focused on assessing the degree of maturity of an enterprise by using lean principles and practices to achieve the best value for the enterprise and its stakeholders [39,40].
In [41], a combination of lean and cloud manufacturing, used to provide a lean monitoring system in a i4.0 contemporary framework, was provided.
However, despite the numerous general assessment tools and theories available in the literature on lean assessment, more research for their application to ETO scenarios is required. In particular, it seems that a practical lean assessment method suited for ETO systems is not yet available.
The aim of this work was to propose a practical model specifically for ETO organizations. The proposed model offers various target uses (e.g., self-assessment, consulting, and third-party auditing) and can be applied at different improvement stages (e.g., identifying and prioritizing areas for improvement, benchmarking with other organizations, progress monitoring over time, and sharing organization-wide learning). The two primary ingredients on which the proposed assessment tool was built up were: the ten ETO principles stated in [30], and the ISO 9004:2018 standard for achieving sustained success self-evaluation tool [42].
As stated in [30], the new set of principles (listed in Table 1) aimed at enabling ETO manufacturers to pursue the lean ideal, gathered insight into lean production from the 14 principles of the Toyota Way [43], the 11 lean construction principles in [44], and the 13 lean product development principles in [45].
In Figure 3, these principles are clustered over Management, Processes, Products, and Stakeholders. It details how those principles concur on delivering value to the customer. In an ETO scenario, the customer demand triggers the system. In this framework, continuous improvement is the engine that feeds the organization, from a dynamic perspective.
The ISO 9004 self-auditing tool is based on five levels of assessment. This tool can clearly outline a high-level overview on management systems' effectiveness in accordance with regulatory standards. It allows the review of the organization's performance against specified criteria. It can also identify current maturity levels and determine strengths and weaknesses-with respect to management-in order to facilitate sustained organizational success. A similar approach was adopted within the ETO lean principles previously defined. Table 1. The ten ETO lean principles from [30] with their Px. Abbreviation in bold font.

P2.
Leadership, People and Learning

P5.
Continuous flow process

P10.
Continuous Improvement Figure 3. Map of ten ETO lean principles.

Methodology
This section describes the methodology used in the generation of the proposed ETO lean assessment tool, which is synthetically represented in Section 2.2. Section 2.1 formally defines the ten ETO lean principles, identified in Section 1.3. Section 2.2. presents objective evaluation criteria arranged in the assessment tool. Section 2.3, defines mathematical criteria for four of the ten principles. In Figure 4, the logical flow of the methodology is shown, and the main steps are represented by light blue boxes.

ETO Lean Principles Definition
Ten principles have been stated and defined. For each of the principles, a synthetic review to extract relevant information for a formal definition has been offered. The purpose of this formal definition lies in the search for objective principles on which it would be possible to base assessments. The Stakeholders Value Analysis (SVA), developed by [46], is a methodology that provides a mechanism to define, link, and integrate the concerns of all stakeholders into a decision. The main steps characterizing the SVA methodology can be associated to the following five steps:

1.
Identification of relevant stakeholders, including external (e.g., customers) and internal (e.g., employees and company owners) stakeholders. The identification process may result in a long list.

2.
Identification and assignment of (key) performance attributes, called values, to the stakeholder group. In this step, the main values in which the stakeholders' groups are interested are defined.

3.
Definition of value weights through the pairwise comparison model, stakeholders cluster method, or others. Weights for an attribute are determined simply, by comparing pairs of attributes one at a time.

4.
Computation of a total value index for each option being assessed by the analysis. A cumulative score for each option is calculated.

5.
Research an option that satisfies most of the main concerns of the stakeholders. Create of a new option, as well, possibly based on those originally considered.
The presented methodology is a representative example of a structured SVA process and can be considered a guideline in the evaluation of the maturity level of this specific principle.

P2. Leadership, People and Learning
Leadership is evaluated by the LESAT methodology [39,40], which is based on the following three macro topics, particularly the dominant one (i): i.
Transformation and change management, ii.
Product development, and iii.
Procedures for infrastructures.
People and learning is based on [47], in which an evaluation of the maturity level of Malaysian SMEs in industry 4.0 was carried out, strongly considering people's competences and the acquisition of lean concepts empowered throughout the organization.

P3. Flexibility
In a perfectly lean and flexible production system, the lead time between customer demand and delivery would be zero. To quantitatively evaluate the maturity level of the flexibility principle, see Formula (1) from [48] for a mathematical definition in which the ratio of the deviation and the total demand were weighted by the capacity demand (more details in Section 2.3.1).

P4. Modularization
In [49], the authors presented a mathematical model for the evaluation of the modularization degree for a given product architecture. That definition has been adopted to quantitatively assess the lean maturity of the modularization principle. The amount of modularization is a function of new components, substitutability factor, and interface constraints (more details in Section 2.3.2).

P5. Continuous Process Flow
Numerous parameters for monitoring process flow continuity are available. In order to quantitatively and practically evaluate the lean maturity of the continuous process flow principle, for a mathematical definition, see Work in Progress (WIP) [50] (more details in Section 2.3.3).

P6. Demand Pull
As a matter of fact, ETO systems are pull-based. Moreover, demand pull is a basic lean manufacturing principle. In order to quantitatively evaluate maturity level-with regard to the organization phasing of customer requests-the relation between forward scheduling (push) and backward request (pull) in\product development and production was considered by [51] (more details in Section 2.3.4).

P7. Stakeholders and Systems Integration
Stakeholder integration can be related to either stakeholder management or management of stakeholders. The different forms of interaction between stakeholders and companies is a delicate topic, analyzed in depth in order to identify key factors that provide companies with competitive advantages. There is significant, relevant need for variables that allow the definition of a measurement scale for stakeholder integration [52].
A set of stakeholders' interaction indicators from [52] was considered. These were: knowledge (of the stakeholders and their demands), interaction (between stakeholders and company), and the adaptational behavior of the company toward their stakeholders.

P8. Transparency
At the organizational level, effective transparency is enacted through a company's willingness to consistently relay information to internal or external interested parties [53,54]. Thus, transparency assessments (in an organizational context) should consider the nature of the information provided by the organization. Clarity, for example, is identified as the amount of understandability perceived by the information receiver. That fact deserves significant attention [53,55].

P9. Technology
In addition to the previously mentioned ETO lean principles (which serve a strict role in the transition of an organization from a practical hardware/software side), technology is the core of many industrial systems. It is thus, understandably, an important component of the engine of change. It is for this reason that the smart factory paradigm received so much attention [56]. For example, several technology enablers in manufacturing scenarios have penetrated almost every production system in place today, even ETO. These include CAE systems for design and simulation, advanced subtractive/additive manufacturing, sensors, programmable robots for material handling, automated quality control systems, ERP management systems, MES digital twins, and more. [57,58].

P10. Continuous Improvement
One of the quality pillar-continuous improvement-plays a key role in driving an organization toward better performance. It also impacts the creation of new opportunities. Numerous international quality management system standards (e.g., ISO 9001 [59]) have highlighted this concept as the propulsor of the Deming wheel [21,60].
Among continuous improvement assessing models, the capability model proposed by [61] is considered a useful guide in efforts to define lean maturity. This model described continuous improvement in terms of a set of key behaviors or behavioral routines deemed essential for long-term success. In this framework, awareness of the major lean tools (highlighted in Section 1.1) was applied in a problem-solving approach. Table 2 summarizes the assessment criteria (over five maturity levels) for each of the ten ETO lean principles defined in Section 2.1. Levels or grades serve as alternatives to a continuous score (percentage), where mathematical expressions are available (Section 2.3).  The maturity level/grade can be inferred by identifying the actual company state for each principle, using Table 2. A fine tuning of the maturity percentage can be defined within each relative 20% discretization range.

Alternative Mathematical Definition
For four principles, (P3. to P6.) the following mathematical definitions are available in the literature.

P3. Flexibility
To evaluate volume and mix flexibility for each variant in the production system, a definite time frame t must be fixed. During time t for a specific product type i, it is possible to define the market demand d t,i and the quantity produced p t,i . If the production system is not flexible enough to satisfy the customer demand for the product type i during the time t, there will be a difference between d tiI and p t,i . For a portfolio of n different product types, the total deviation in units can be formally described by the Formula (1). Different product types typically require different specific resources. For this reason, a correction-weighting the deviations in volume according to their impact on capacity-is needed. c i is defined as the capacity demand for one unit of product type i [48].

P4. Modularization
The modularization function (2) decreases nonlinearly from a perfectly modular architecture (e.g., only standard, no new components) to a perfectly integral architecture (e.g., no standard components, only new ones).
where M(u) is a modularization function of u, number of new components, N is the total number of components, s, the substitutability factor, δ, the interface constraints [49].

P5. Continuous Process Flow
The continuous process flow (WIP) is calculated in (3) as defined by [50]. The flow time, F T , is the actual amount of time spent, and the cycle time, C T is the ideal time needed. The first term varies, the second is fixed.

P6. Demand Pull
The pull degree, quantified in (4), was defined in [51]. This criterion is closely related to the CODP defined in Figure 1 and quantifies the ratio between the activities after the decoupling point, in contrast with the total number of phases in the product life cycle (Engineering, Purchasing, Assembly, Packaging and Delivery).

Case Study
Further guidelines for the model and method of application are presented here. The assessment tool was developed and tested on an Italian manufacturer of customized woodworking machines and lines in an ETO/MTO scenario. Section 3.1 offers a validation of the tool, along with a more detailed guide to the practical evaluation of the principles whose mathematical formulas have been defined in Section 2.3.
Based on the evidence in Table 3, P10. *, continuous improvement was the weakest principle. For this reason, a set of steps followed in the continuous improvement path is presented in Section 3.2.

Tool Validation
The outcome of the application of the proposed tool is presented in Table 3. It shows the assessment evidence and evaluation. With respect to the four principles defined in Section 2.3, the mathematical evaluation and accompanying detailed guide in the use of its formulas can be found in Sections 3.1.1-3.1.4.

P3. Flexibility
As shown in Table 4, the difference between market demands d i and product quantity p i is equal to 0 for each product type. Therefore, the Formula (1) results equal to 1. Table 3. The ten criteria, evaluated on a manufacturer of woodworking machines and lines in an ETO/MTO scenario. Maturity grades and scores are shown. In addition, all evidence for each principle has been summarized. (*) denotes principles where a mathematical formula is not defined.

Lean Maturity
Grade Score P1. * Little clarity in the stakeholders' roles in the company emerged. Absence of an organizational chart. 2 30%

P2. *
The management is responsibly committed to the achievement of goals and promotes active training on the job.
The company adapts to customer requests easily. Result from (1)

P4.
The reuse of standard solutions or parts to produce special machines is common practice. Result from (2) M(u) = 0.8

P6.
For the machines analyzed, the assembly process starts after receiving the customer orders. Result from (4) Pull Degree = 60% 4 60%

P7. *
The company consults the interested parties before making decisions. Meetings are by department, not collective.

40%
P8. * Average level of disclosure. There is no standard, tested system for information disclosure. 4 60%

P9. *
The company uses a specific management software (anonymized). The management software includes the customer orders, insertion of the bills of materials and the purchase of the materials necessary for production.
Commercial CAD software is implemented.

P10. *
Senior leaders and management are aware of the continuous improvement philosophy, but lean tools and methods are not actually implemented.

20%
OVERALL Ten criteria average value 4 63% The degree of modularization expressed in (2) was computed by approximating the product (Model A) as a reference for all the product types. The number of new and total components for Model A are, respectively, u = 12 and N = 62. With respect to substitutability, the value can be assumed as s = 1 because of the possibility of replacing any new component with an old one. Finally, the interface constraint factor δ can be divided into two levels. At level 1: where δ component in (5) has been expressed as: where I in (6) is the number of subcircuits and the interface constraint value is δ i = ∑ K c n c , with K c number of interfaces and n c is the number of components in a subcircuit. Regarding the subcircuits δ sub_circuit : where K sub_circuit in (7) is the number of subcircuit interfaces. Using the data in Table 5: I = 5; δ sub−circuit = 1.4; δ component = 5; δ 1 = 6.4. At level 2: (8) is the number of total mechanical components and where K j in (9) is the number of interfaces for the subcircuits analyzed and N j is the number of mechanical components at the interface with the subcircuit. Using the data in Table 6, δ 2 = 6.75. Using the above results, δ tot = δ 1 +δ 2 I = 2.63, where δ tot represents the interface constraint factor for one base of the product Model A. Therefore, the final δ tot is multiplied by 2. The final degree of modularization of the 12 new components, determined by applying (2), is M(12) = 0.8.

P5. Continuous Flow Process
The Work in Progress (WIP) can be calculated, as in (3). In the present case study, the flow time F T and the cycle time C T are calculated referring to the product Model A.
In Table 7, daily data related to the product Model A (including the actual amount of production time (flow time)) are presented. Based on the data presented in Table 7 Table 8. Table 8. Conversion table of 1.12 in 88%. The maturity is assumed as a linear with F T , maximal when the F T = C T and minimal when F T = 2 C T .

Percentage [%]
WIP The lean maturity score for the WIP in the present case study for the continuous flow process criterion ∼ = 88%.

P6. Demand Pull
The pull degree was calculated according to (4), where P N = 3 (assembly, packaging and delivery), the number of pull phases (after the decoupling point) and P Tot = 5, the total number of phases. Therefore, pull degree = 60%. This parameter also shows that the examined company fell between pure ETO and make to order (MTO).

Problem Solving
As shown in Table 3, the organization had a critical lean maturity level, mainly on continuous improvement. As anticipated in [62], many small and medium-sized enterprises, like the present case study, often prefer to invest in quick high potential improvements (due to their limited resources) in their implementation of lean principles. For these companies, the focus should lie on low hanging fruits, including methods like continuous improvement, that can be introduced with minimal effort and generate maximal benefits. Sections 3.2.1-3.2.6 describe the main steps followed on the continuous improvement implementation path, based on a systematic problem-solving approach which included some common lean tools. In particular, VSM, the 5 whys, and the new PFM (described in Section 3.2.5) were applied in a DMAIC (define, measure, analyze, improve, control) cycle [63], revised according to the framework in Figure 4 and detailed here.

Problem Identification
To identify all the problems, briefing activities were carried out. These included: (i) procedure definition, (ii) data collection sheet definition, (iii) audit, and (iv) noncompliance (NC) data collection. The most effective tool used in this phase was a noncompliance checklist designed for data collection.

Problem Prioritization and Selection
After listing and prioritizing the identified problems according to a Pareto chart cumulating the recurrence, the most frequent one was selected: incomplete thread of the support housing screw.

Problem Definition
In this phase, regarding the problem (incomplete thread of the support housing screw), the process focusing on the interested station was analyzed. The value stream mapping (VSM) tool was applied to the process flow diagram to detect possible inefficiencies or wastes. It highlighted how the retapping of the thread occurred, after an initial tapping by the supplier. This repeated tapping was identified as a possible source of inefficiency because it added no value.

Problem Analysis
In this phase, the causes of the selected NC were investigated by applying the classical lean 5 whys tool, as shown in Figure 5. Starting from the circumscribed problem identified in Section 3.2.3, five "Why?" questions were asked, in an effort to determine the root causes. The first question in the chain was: why re-tapping the thread? The two identified root causes: (a) External: generated by the supplier (because the supplier does not protect/clean properly the nut-screw during manufacturing/painting/delivery). (b) Internal: generated by the technical office (because the section of the M10 holes is not present in the technical drawing).

Focus Definition
To face the two identified root causes, a graphic new lean tool for strategy implementation is proposed in Figure 6: the problem focusing matrix (PFM). In the case of a specific problem, like that in the present case study, the functional area that suggests lighter strategies was considered. If the problem affected the whole organization, systemic action would need to be considered.
For example, in the case of external/cross-functional systemic problems, the theory introduced by Kraljic [64] as well as Olsen and Ellram [65] required more attention, in contrast with smaller, easy-to-solve problems.
Focusing on the two root causes identified in Section 3.2.4, In cause (a), the team is basically functional and focused on the manufacturing area. The problem is caused by an external issue: the supplier. In this scenario, the suggested strategy is supplier notification.
In cause (b), the team is basically functional and focused on the manufacturing area. The problem is caused by an internal issue. The head of the department decides how to solve and eventually redesign the process/product.

Solution Implementation
To remove the root causes, the PFM defined in Section 3.2.5 can be used as follows: (a) "The supplier does not properly protect/clean the nut-screw during manufacturing/painting/delivery". Solution: in the case of an external and circumscribed problem like this, the resolution is a simple notification to the supplier to request a preventive action for the regenerating of that NC. Improved communication could help speed up this process. (b) "The section of the M10 holes is not present in the technical drawing": Solution: Focusing on the internal cause, two actions were adopted: the correction of the technical drawing, and the removal of the root cause-the absence of defined roles/responsibilities. The responsible-accountable-consultant-informed (RACI) tool was a simple and effective way to avoid the repetition of similar issues caused by the absence of formal separation of duties inside the company.

Results
The application of the proposed assessment criteria in Table 2 to the examined case provided the evidence and the grades/scores shown in Table 3, and visually represented by a radar graph in Figure 7.  Table 4 (blue); After data resulted from the application of the solutions in Section 3.2.6 (orange). (*) denotes a principle whose mathematical definition is lacking.
The management of the real ETO industrial company involved in this project identified the priority of investing in the weakest principle, as determined by the analysis in Table 3: continuous improvement.
It was clear that continuous improvement tools and techniques positively affected the following principles: P10. *: direct effect on continuous improvement, increasing from grade 2 to grade 3 (from 20% to 40%); P7. *: indirect positive effect on the stakeholders and system integration principles (from 40% to 50%).
Among the numerous lean tools used in the problem-solving process, the VSM and the 5 whys enabled the identification of a source of waste in the production process. By removing the no-value-added activity of retapping, 25 min were deducted from the total station time of~45 min (or~55% of the station time).
After the adoption of these improvements, the new overall score on the ten principles increased from 63% to 66%. The objective improvement after repeating the assessment was clear, and can be considered as a validation of the proposed method.

Discussion
The presented model is a systematic and easily-implemented tool for ETO companies wishing to move toward lean manufacturing. It offers an objective look at their lean manufacturing maturity level and a way to define improvement paths.
From Figure 7, it can be observed that the ten principles are not completely independent. In particular, continuous improvement indirectly affects stakeholders and system integration (and, to a lesser extent, other principles). Surveys on clusters of ETO manufacturers (e.g., in geographic areas or within specific sectors) may infer statistical evidence to refine the proposed assessment model.
Interferences among other principles were observed, but their low significance did not allow a generalization to be drawn from the examined case. Extensive application in surveys on ETO industrial compartments, company sizes, geographic areas, etc., may infer interferences among lean principles (e.g., leadership and others).
Regarding the stakeholders and system integration principle: the application of the strategies shown in Figure 5 to the two root causes of the proposed matrix enriched a classic problem-solving approach. Namely: - In (a), it facilitated collaboration within the company throughout the value chain. - The use of the responsible-accountable-consultant-informed (RACI) tool in (b) defined roles and responsibilities within the organization matrix by focusing on the interested parties within the organization.
Regarding lean maturity and the seven kinds of waste [21,22]: the 55% reduction of one production station's time falls under the waiting time waste class. A formal relationship between the proposed model and the waste to be eliminated is a possible future development.
The current model includes the definition of objective parameters for all the ETO lean principles. Additional mathematical formulas can be included and further refined for a more detailed assessment, e.g., a synthetic method for people management [66], smart sensor outputs [67], and supply chain integration [68].
In Figure 8, a hierarchical representation of the developed model is provided. The principles and criteria have been defined in cascade. Once the assessment process has been carried out, the resulting evidence can be used as a starting point for improvement activities. The problem-solving approach in Section 3.2 is an example of activity. Figure 8 represents a systematic framework to integrate lean production and the mission of the organization, by exploiting the building blocks represented by the lean tools (exploded in the middle). This area connects the assessment framework and the strategic management.
Further works would help in defining a comprehensive assessment tool focused onto ETO lean practices with direct connections to the elements of the pyramid above. Similar connections were found in the European Foundation for Quality Management (EFQM) model [34,69], the Baldrige award, and the Deming price, which provide a systematic perspective for a future lean ETO award. Such standardized framework may adopt specific lean tools and techniques, e.g., poke-yoke, JIT, SMED, 5S, TPM, and others. An example was provided within the proposed problem-solving approach, including two lean tools (VSM, 5 whys and PFM).
One weak point in the present work is model generalization. The application of the proposed method to a statistically significant number of applications is required.

Conclusions
The assessment model presented in this work offered an overall view of lean maturity (from a comprehensive and systematic approach) in an ETO scenario.
The aim of this work was to offer a practical tool that could be easily implemented by ETO manufacturers for as-is mapping and weakness identification. Additionally, the tool would enable monitoring of the manufacturers' achieved maturity of lean principles.
This work identified ten dominating lean principles from the restricted literature on lean ETO. These principles were further defined through further analysis of the literature that was not specific to ETO. An assessment model for lean ETO maturity was not available and was thus defined by quantitative criteria on five levels, grades, and scores. Four criteria were associated to mathematical formulas.
The real case study presented showed, in detail, the practical application of the lean maturity assessment model, serving as a step toward validation of the proposed model. The problem-solving approach (based on the evidence uncovered during the assessment step) represented a step toward a structured framework to connect the organization mission and management via existing lean tools (VSM and 5 whys) and a newly developed one (PFM).
The proposed method led to more research questions than answers. These have been extensively discussed above.
Among the dominating lean principles in ETO, continuous improvement deserved particular attention, and showed a positive impact on other principles.
A double nested virtual improvement cycle was identified for the proposed model and the lean improvement path.