An Additional Model to Control Risk in Mastering Defense Technology in Indonesia

Abstract

Reverse engineering is widely used to acquire defence technology relatively quickly. However, this process carries the risk of creating uncertainty, leading to significant investments if the process is not controlled. A technology readiness level (TRL) calculator has been used to control the process of technology mastery during forward engineering. This study aims to build an additional model so that TRL can also be used to control risk in the process of mastering technology. It does this by using reverse engineering to help organisations reduce costs. This additional model is presented through a reverse engineering concept based on theory and is tested through a case study of a defence organization in Indonesia. The results of the case study show that the TRL calculator can be used as a reference in mastering technology through reverse engineering.

1. Introduction

Mastery of technology has an essential meaning in national defence systems. Excessive dependence on the technology of outsiders in the defence sector can weaken the national defence system [1]. Mastery of technology in defence and aircraft requires a significant investment [2] because it can be a stimulant for innovation [3] and can affect the country’s economy [4]. All countries of the world are competing to invest their resources to master defence technology. On average, developed countries spend 2–3% of GDP per year to implement national defence. This implementation is carried out continuously, and it must not be interrupted [5]. However, because of competition, mastering technology in the defence sector has many areas of uncertainty. Uncertainty in the mastery of defence technology affects the amount of investment that the government must issue in implementing the defence system [1]. Therefore, mastery of defence technology requires a strategy to take place efficiently [6].

There are two strategies for mastering a technology: building and buying [7]. Developed countries master technology by building technological capabilities in stages. Technological development begins with a concept, which is then developed as a prototype to be tested before it is mass-produced [8]. Technology development requires investments, such as human resources (HR), research institutions, complex equipment, facilities, raw materials, electrical energy, water, and finance. In some cases, the development of complex technologies takes a long time because of uncertainties, such as the availability of raw materials, manufacturing equipment, and proper test equipment. Therefore, developed countries try to minimise investment using the technology readiness level (TRL) [6]. TRL contains references to technological maturity, which is arranged in stages to achieve technological mastery (which yields minimum uncertainty) [9]. TRL has been proven to control mastery of technology using a forward engineering approach. TRL was first applied in the National Aeronautics and Space Administration (NASA) program in the late 1970s [10]. As an alternative, mastery of technology can occur by buying it.

A purchased license is one way to meet the needs of a defence system by using the domestic defence industry [5]. Unfortunately, most developing countries use this strategy because of limited resources. In this process, the government purchases licenses and then assigns the national defence industry to study and master the production capabilities of foreign licensors. Thus, purchases are made to shorten the technology mastery process and minimise investment [7]. However, purchasing licenses also has weaknesses, especially during the process, because (i) not everything related to the purchased technology is transferred [11], (ii) the transferred technology may not be absorbed by all because of differences in culture, language, and knowledge [12], and (iii) the transferred technology requires maintenance to be developed [13]. Furthermore, the transferred capabilities are limited to production capabilities.

The technology transferred through a purchased license is focused only on HR, production tools and facilities, and test equipment to ensure quality. An industry cannot compete for long if it controls only production capabilities because it will be eroded with obsolescence [14]. Therefore, these production capabilities must be maintained and developed. After mastering production, the industry must develop capabilities in engineering so that it can continue to compete with other industries. Reverse engineering is an effective way to develop complete engineering skills. This approach can save time and investment if it is carried out appropriately [15,16]. However, besides its advantages, reverse engineering has a weakness. The process leads to high uncertainty (compared to forward engineering).

TRL has been used worldwide to measure technological maturity [17] because it is quite efficient and effective. It is easily understood by people with a variety of backgrounds [6]. TRL has been developed according to needs, including manufacturing [18], integration [19], innovation [20], systems [21], regulation [22], and marketing [23]. This development shows that TRL is flexible in its ability to be developed. As explained above, TRL is suitable for mastering technology using an advanced approach [24]. The question that arises is this: ‘Can TRL be used as a reference in mastering technology using reverse engineering?’ This question is used as a reference for this study. The steps used in the research follow a framework that starts with a literature study to review the reverse engineering concept. That concept will be verified with a correlation theory and validated using a case study of one of the defence industries in Indonesia (Appendix A). Once the concept has been validated, additional models can be used as a reference for it, and these are controlled using a TRL calculator. Due to the fact that reverse engineering has been used in a wide variety of fields, the additional TRL model from this research can be used to control other reverse engineering programs.

2. Conceptual

2.1. Sustainability

The United Nations (UN) defines sustainability as an effort to meet current needs without jeopardising the potential for meeting future needs [25]. Reverse engineering is an effort to reduce the use of resources, such as tools and facilities, that require electrical power and raw material experiments [26] to develop a performance that is close to existing technology so it can be developed sustainably into more efficient products.

In its implementation, reverse engineering is used to recover critical software that has been tested elsewhere in the context of security, such as flight control and nuclear energy [16]. Reverse engineering is also used to discover the original shape of a worn object [15]. In the defence sector, reverse engineering is often used to recycle technologies that are already seen as obsolete, such as components, sub-components, and critical software. Reverse engineering is used to ensure safety and provide a high level of certainty [16]. Furthermore, reverse engineering is also carried out to reuse and develop products for investment efficiency.

This contrasts with forward engineering, which requires sorting uncertainty into certainty [24]. The sorting process requires testing and proving, which requires a lot of resources. Moreover, more than a few alternative technology concepts must be repeated to meet the target. Therefore, in some cases, mastering technology using reverse engineering requires a lower investment than forward engineering.

2.2. Purchased License

The government or industry often uses purchased licenses to obtain production capabilities, which include detailed technology drawings, information about the operation and manufacturing equipment, and quality assurance of production results. Therefore, the important factor in the licensing process is the maturity of the licensor technology [11]. The greater the maturity of the technology owned by the licensor, the less uncertainty that buyers have [27]. Uncertainty has a direct correlation to the investment that must be made. Following the TRL concept, the production capabilities are at low-rate initial production (LRIP or TRL8) or full-rate production (TRL9) [28].

At level eight, the product has been demonstrated and certified [17]. The technical changes have been so minor that the detailed drawings are almost complete and ready to carry out full-rate production [29]. At level nine, the industry capability has been tested, and the industry has a complete picture in order to sustain the operation and support stages: competitors are known, investment forecasts have been made, and there are no more technological changes [29]. However, when purchasing a license, the knowledge to perform forecasting is minimal because it is not included in the production capability. Forecasting requires conceptualising technology until it is ready to be tested in a laboratory and operating environment (TRL6 and TRL7) [10]. This ability is known as the engineering ability.

The licensor is obliged only to provide and transfer production capabilities, such as the completeness of drawings and standard operating procedures (SOPs), along with production process troubleshooting, maintenance, the repair of production machines (if the production machines are the same), and production quality test procedures (shaded in grey in Figure 1) [11]. There is little or no information about the method of drafting the design concept and prototyping. This is protection for the licensor from the buyer’s industry becoming a competitor (not shaded) [11].

The engineering ability differs from the production capability. Engineering skills focus on conceptualising technology and implementing, testing, and developing it. Meanwhile, the production capability is the ability to produce products based on technical drawings produced by the design engineer. Reverse engineering is needed to achieve complete technological mastery. Mastery of technology through reverse engineering is essential because the effectiveness and efficiency of the product in one lifecycle is determined during the development of the technology concept (concept phase) [24]. The sub-components/components that will be maintained and developed are determined at this stage. Therefore, reverse engineering activities must be well planned, especially those related to internal and external risks [30].

According to Mankins (2009), the highest risk in building a technology concept with a forward engineering approach occurs at a low TRL because of high uncertainty (see Figure 2). Red indicates high uncertainty (high risk), yellow indicates moderate uncertainty (medium risk), and green indicates controlled low uncertainty (low risk). The investment required to build technology maturity is 8% for TRL1, 10% for TRL2, and 36% for TRL3 36%; then, in the yellow, 24% for TRL4, 19% for TRL5, and 6.3% for TRL6. In the green, it is 0.1% for TRL7 [17]. This shows that risk is directly proportional to investment and uncertainty.

According to Ben Hicks et al. (2009), product development that is an improvement in nature—group 1 in Figure 3—has a trim to moderate level of risk (improvement TRL9). Invention—group 3 in Figure 3 starts from TRL1, and it has a high level of risk. The mastery of technology using forward and reverse engineering has similarities with Figure 2. However, the research conducted by Ben Hick et al. (2009) is more inclined to master technology using reverse engineering because the aim is the development of post-production technology. If viewed from the risk of investing, mastery of technology using reverse engineering faces challenges that are low (1), moderate (2), and high (3). This contrasts with forward engineering, which starts from high uncertainty to being difficult to control. In addition, the TRL is an effort to determine investment needs because many alternatives and concepts must be tested and validated [6,8,10].

2.3. The Technology Readiness Level (TRL) Calculator

The TRL is arranged in stages from seven levels to nine levels of technology readiness [24]. Each level has definitions and indicators to assess technology maturity [17], which can reduce uncertainty in technology mastery [24]. Once mature, the technology is ready to be perfected by developing variants or being replaced with new technology following research rules. In 2016, the Ministry of Research and High Education in Indonesia adopted the TRL calculator produced by Nolte to simplify its assessment of technology maturity. The TRL calculator is listed in the Minister of Research and Higher Education Regulation of RI number 42 Year 2016 [29] (see Appendix B).

This reference starts the assessment by observing the basic principles to the criteria that the technology has tested. Then, each level has simple assessment indicators to provide a more concrete picture to its users. The concept of the TRL calculator is developed enough to be used as a reference because it combines several assessment concepts other than technology, such as manufacturing requirements and technology integration [29]. Currently, indicators of the TRL calculator continue to be developed to suit the development of science and technology. Here is the order of the TRL calculator definitions from lowest to highest:

• The basic principles of technology have been observed.
• The concept of technology and its application has been formulated.
• Important concepts and characteristics of technology have been proven analytically and experimentally.
• The code, component, and breadboard have been validated in the laboratory environment.
• The model or prototype has been validated in a laboratory or simulation environment.
• The prototype has been tested in a natural environment.
• System prototypes have been demonstrated in the actual environment or application.
• The system has been demonstrated and tested in the actual environment or application.
• The technology has genuinely been tested or proven through successful operation.

2.4. Post-Production Defence Product Development

Post-production defence technology continues to be observed and evaluated, especially when it enters its half-life and full life. The primary consideration in observation and evaluation is the cost of maintenance and repair [1,5]. In addition, the evaluation results become an object for the government to decide on the next activity [22]. Observation and evaluation activities are as follows:

• Research on critical subcomponents and outdated software

Observations of critical subcomponents are carried out during maintenance and repair. Critical subcomponents are evaluated to reduce dependence. This activity is carried out routinely, and it involves research and development (R&D) institutions, users, and the defence industry using reverse engineering. The reverse-engineered subcomponents are partially tested, integrated, and certified to ensure feasibility. This activity does not change the product drawings. The same activity is carried out when replacing obsolete sub-component level S/W. If it is associated with TRL, this activity is included in the low-risk category because it repeats the demonstration of system prototypes in the actual environment or application until it is proven through successful operation (a3, group 1);

2.

Modifying or upgrading the half-lifecycle product performance

Product performance is reviewed every half lifecycle by considering changes in operational costs because of the increasing number of critical spare parts and changing operating requirements [5]. The activities carried out are component-level modifications or upgrades. Before carrying out modifications or upgrades, preliminary research is conducted. The research on critical components may involve a broader range of R&D institutions. The reverse-engineered subcomponents are partially tested, integrated, and certified to ensure feasibility. This activity changes the production drawings so verification is required by testing in the laboratory to ensure successful operation. The same activity is carried out when replacing the component-level S/W. If it is associated with TRL, this activity is included in the moderate risk category because a series of verifications must be carried out as a concept study: several partial tests, integrating the technology with old components, system testing at the laboratory and operating scale, and quality testing (Figure 3, group 2);

3.

Upgrading or disposing the product performance in one lifecycle

The product performance is reassessed as it enters an operation lifecycle. The study focuses on operational costs and changes to operational needs [5]. If technical and operational costs are still feasible, the products life cycle is extended; otherwise, it is destroyed. System research at this point might involve a variety of R&D institutions;

4.

Replacing the system using reverse engineering

The system, because of reverse engineering, is integrated to upgrade capabilities. The activities performed are the same as in Figure 3, group 2. However, if the decision is to dispose the item, the defence industry must repeat the process from the concept stage or buy a replacement from foreign defence industries (Figure 3, group 3). This activity has a high risk because it must start by observing the basic principles of technology;

2.5. The Concept of Reverse Engineering

Reverse engineering helps developing countries such as Indonesia master technology with a fast process. The aim of mastering technology is to allow the defence industry to meet the needs of the defence system immediately and to compete in terms of improving the country’s economy. In addition, reverse engineering can save investment in resources if it is appropriately controlled.

Several developed countries have used the TRL calculator to master technology through a forward engineering approach. This approach starts with mastery of technology from high-risk to low-risk areas of uncertainty. Reverse engineering achieves the opposite. The time for mastering reverse engineering is a technology lifecycle, so it can take advantage of the post-production product development pattern to develop engineering capabilities and refine the indicators in the TRL calculator (Appendix B) by considering the risks in Figure 3.

3. Methodology

Case study research is similar to historical research and experiment-based research because it asks ‘how’ and ‘why’ [31]. However, case studies differ in that they have limited opportunities to control the behaviour of objects or events. Therefore, a case study is used in this paper to examine a particular situation intrinsically to present a perspective on improving a theory through interviews with resource persons.

From a theoretical perspective, reverse engineering has a duration of only one technology lifecycle, and its control can take advantage of post-production activities and indicators in the TRL calculator. Therefore, to answer the question, ‘Can TRL be used as a reference in mastering technology using reverse engineering?’ it is necessary to prepare additional questions for interviews with resource persons, including:

• What is the history of mastery of assault rifle technology by PT. Pindad?
• How does PT. Pindad conduct the SS-2 production process?
• What are the differences between SS-1 V1 and SS-2 V1?
• What are the efforts of PT. Pindad to improve the performance of assault rifles in the future?

(See the steps of research in Appendix A)

4. Case Study

4.1. A Defence Industry Case in Indonesia

PT. Pindad is a national defence industry that has been producing weapons since the 1960s. The rifles it has produced include the SP-1, SP-2, SP-3, SS-77, SS-79, and SS-1. The ability to produce long weapons (the SP category) is based on purchasing a license from Berreta. Berreta products were the primary weapons used by the Italians in various operations. However, the SP rifle has several weaknesses in military operations and training in Indonesia, including frequent jams because of the bullet cladding, a trigger compensator that often comes off, and a wooden buttstock that rots quickly. Several analyses have concluded that the causative factor is the difference in weather between Italy and Indonesia.

Because of changes in user needs, PT. Pindad is trying to develop assault weapons (SS-77 and SS-79). The weaknesses of the SPs became the focus for improvement in this development. However, the SS-79 development process took a long time. Therefore, the government of Indonesia decided to meet its need for weapons by purchasing a license from the Belgian Fabrique Nationale Carabine (FNC). The main reason for this choice was the proven toughness of FNC, the ease of obtaining the technology and basic materials, and the 5.56 mm × 45 mm bullet used by the Indonesian infantry.

Fulfilling needs is carried out in two stages. The first is to bring FNC products directly from the factory. The engineers of PT. Pindad obtained the technology transfer at the Belgian FNC plant. That activity took place in parallel with preparing the production line at PT. Pindad Bandung with assistance from FNC engineers. When the engineer and production line of PT. Pindad is ready, the second stage begins. PT. Pindad produces weapons independently in Bandung. The product is named SS-1.

The most visible physical difference between the SP type and the SS-1 is that the buttstock and handguard are not made of wood, whereas the SS-1 and SP are similar in using gas-operated technology and a rotating bolt. Until now, PT. Pindad has produced over 165,000 SS-1 shoots. The SS-1 has been used in various operations, such as routine border patrolling, and special operations, such as hostage releases and training. PT. Pindad has also developed several variants of the SS-1 because of demands in the market. PT. Pindad successfully developed variants of the SS-1. The SS-1 V2 has a short buttstock, the SS-1 V3 has a fixed buttstock, the SS-1 V4 includes a telescope, the SS-1 V5 has a special design for tank crews (it is smaller than SS-1 V1), and the SS-1 series M is specifically for marines who must have paint that is resistant to seawater.

The maintenance and repair of the SS-1 is straightforward, including intermediate inspection after firing three and 10,000 rounds. Therefore, the SS-1 has always been used as the primary weapon by infantry and some TNI operations. However, when the product enters its half-life (about 15 years), spare parts become scarce, and the maintenance of the production equipment becomes absolute. In 2001, PT. Pindad developed the SS-2 using the same technology as the SS-1. However, there were some changes. The weight of the new weapon is 3.4 kg, the material of the barrel and components are stronger and more resistant to the moisture of the tropical climate, and the model is more ergonomic. Development was based on input from customers and support from universities and R&D institutions. However, the production equipment and facilities used are the same as for the SS-1. Therefore, the decision to develop the SS-2 was used to rejuvenate some old production equipment and automate other production equipment to reduce production costs.

The renovations included replacing the lapping machine with a honing machine, CNC turning with CNC Turning Multitask, and making the CNC Swiss machine more precise and energy-efficient. PT. Pindad can produce 200–300 SS-2s per day. The obstacle that is still being addressed is procuring raw materials, since most of the raw materials for SS-2 still come from abroad. During the COVID-19 pandemic, several suppliers of raw materials increased prices by around 30–40%. This creates a vulnerability to rising production costs. Therefore, PT. Pindad continues to seek domestic industrial partners who can consistently provide SS-2 raw materials at competitive prices. PT. Pindad also continues to optimise production in terms of process and product quality in collaboration with various institutions.

4.2. Reverse Engineering Activities

The SS-1 assault gun technology was obtained by purchasing a license that included technology transfer activities from the Belgian FNC. The maturity of FNC licensor technology is greater than TRL7. To strengthen the statement, first is the technology transfer activity (assistance or supervision) of FNC engineers to the engineers of PT. Pindad. The second parameter is assistance in constructing a production line for PT. Pindad in Bandung (FNC has mastered the ability to at least have LRIP). Technology transfer also includes preparing human resources for production by supporting the completion of the drawings and noting the similarity of tools, facilities, production processes, and quality assurance as part of the technology transfer. Before developing the new product (SS-2), PT. Pindad carried out the following activities.

1.

Observing SS-1 technology components and subcomponents during routine maintenance and repairs to identify the superior and critical ones.

Replacing components and subcomponents is used as data for sorting. In particular, for replacing components or subcomponents, a test is carried out according to SOPs. If it is associated with the TRL theory, the test repeats the TRL7 test process. Retesting helps train engineers to update processes constantly; improve procedures, methods and equipment; complete drawings and test equipment.

2.

Developing the variants of the SS-1 was carried out to meet user demands.

Altering the buttstock and barrel components involves a low-risk development [30]. Therefore, the changes are minor, changing the shape of the buttstock without changing other significant components. The activity required designing the buttstock and integrating it with the central part (see 081 Appendix C). Activities start with TRL7, TRL8, and TRL 9 (see Appendix B).

3.

Developing the SS-1 V4 by adding a telescope.

This is a moderate risk because it combines two different technologies. This development is an attempt to explore the possibility of combining the SS-1 technology with other technologies. This development changes product drawings (see 201 Appendix C) and testing SOPs. This development should repeat TRL5 to TRL9 (see Appendix B).

4.

Creating the unique design of the SS-1 V5 for tank crews and SS-1 M for the marines.

The SS-1 V5 was a unique design for tank crews, making it smaller than the SS-1 V1 and the SS-1 M series involved with paint resistance to seawater. These activities must start with TRL 2. This development step helps to improve the engineers’ abilities by changing the design to be smaller. Mastery of technology is needed before adding variants with different designs because this activity has a high risk [30]. The increase in the level of risk is caused by changes in design, the setting of production equipment, and testing SOPs. Activities must be carried out from the beginning by using existing tools and facilities. This activity focuses on design and production capabilities. Failure to add variants would have a significant impact on efforts to meet the needs of special market segments. The role and support of R&D agencies are needed to conduct research and assessments to support these processes [20]. The success of PT. Pindad in building variants shows that PT. Pindad has mastered the SS-1 assault gun technology.

5.

Designing the SS-2

After developing several variants of the SS-1, PT. Pindad started designing the SS-2 in 2001. The technology concept was developed in stages, from the initial TRL to marketing the product in 2005. Prior to the redesign, PT. Pindad already knew user needs, competitor technologies, and the essential elements to be developed. The parts that are lacking in the SS-1 are well developed, including the strength of the primary component materials, a larger gas cross-section, and an ergonomic shape. These are performance improvement parameters that can reduce the use of resources.

5. Discussion

The rapid development of science and technology directly increases the complexity of mastering technology [32]. Furthermore, linking technology with other fields increases uncertainty in areas such as the availability of equipment and manufacturing facilities [33], integration [21], law, and politics [11,12]. Therefore, efforts toward technology mastery must be well controlled. Reverse engineering to master technology requires a reference to minimise uncertainty [34]. This is because reverse engineering rediscovers the technology preparation process with a high level of uncertainty (from green to red in Figure 3). The result of reverse engineering is an arrangement of materials and software close to the original materials and software. The higher the uncertainty, the greater the gap in performance between the existing technology and the new technology [35]. Moreover, the size of the gap is directly correlated with the investment that must be made. Furthermore, the time for mastering technology is limited to one lifecycle so that it is not eroded by obsolescence.

The case study shows that two skill areas must be mastered to develop technology from SS-1 to SS-2: engineering design capabilities and production capabilities. Engineering and production capabilities are mutually supportive [8]. Engineering starts developing the technology concept to produce detailed designs in the form of drawings and ways to implement and test the product [29]. The production department develops production procedures based on existing manufacturing equipment and facilities [36]. The technology transfer gives the license buyer detailed drawings, procedures, and available production equipment. However, the existing production capabilities cannot prepare forecasting and technology investment (TRL9 Appendix B). Forecasting and development investment can be carried out only if information and data support regarding the existing technology is available. Therefore, to conduct observations and collect more structured information, an organisation is needed [20]. Furthermore, there must be a part of the organisation that monitors and maintains observational data and continuous quality improvement.

The focus of technology transfer by purchasing a license is restricted to assistance with production capabilities. As a result, the engineering process yields only a very minimal design [37]. Therefore, it is necessary to reverse engineer to develop technological capabilities. These methods can be carried out independently by observing (i) product maintenance and repair processes, (ii) critical spare parts, (iii) testing procedures, and (iv) user input. One example is minimising dependence on spare parts to anticipate obsolescence.

Parts that are replaced often can be investigated, especially the shape and constituent materials (such as firing pins) using reverse engineering [15]. A firing pin (see 430 Appendix C) is a subcomponent that wears out quickly because of collision with the bullet. Therefore, firing pins must be replaced periodically after some filling. Data support and studies related to firing pin materials are needed. This information makes it possible to redesign in order to achieve better quality.

Prior to design, the shape and material of the firing pin were known so that the reverse engineering focused directly on building and developing the firing pin. Both activities can reduce resource requirements. Moreover, the constituent materials used in the firing pin can be selected considering the availability of the materials in Indonesia. Firing pins developed will be integrated and tested on a laboratory scale and demonstrated to ensure feasibility (TRL7 Appendix B). According to Mankin (2009), the level of risk from replacing spare parts is 0.1% [17]. The risk of replacing spare parts is low because it only repeats SOP’s demonstration to prove that the firing pin can work adequately (TRL 7). The focus of the activity is finding suitable materials for firing pins while the shapes and sizes are the same. The impact of these activities is the ease of obtaining firing pins at low prices. Furthermore, the reverse engineering of the firing pin in terms of its shape, the building material, and the production process will add to the engineering data bank. This kind of activity also applies to technology other than in the defence sector, such as replacing automotive parts.

From the firing pin discussion above, it can be concluded that the initial observations made were intended to investigate the reliability of the sub-component technology. The identification results will be used as a reference in making plans and processes to obtain materials and produce firing pins. The second stage is the development of variants such as the buttstock (see 081 in Appendix C) and the barrel (see 011 in Appendix C) improvement. Those developments required complex activities involving other parties from outside [20]. Several activities should be carried out, such as materials analysis, new design creation, integration planning, and testing procedures preparation. The results of the variant development will be integrated, demonstrated, and tested (TRL5–7, see Appendix B). The variant development also requires changing the existing drawing (TRL8 see Appendix B). However, the potential risk is not the same as in the case of forward engineering (22.1%) because development activities are only partial and more focused.

After reviewing and repeating activities or procedures starting from integration, testing, and changing images, the third stage begins. It focuses on engineering mastery from the early stages. The variants were developed by changing the weapon’s design to be smaller and rust-resistant. This development is like building a new design using existing technology (gas operation and rotating bolt). The activities include making technological formulations, proving analytically and experimentally, integrating, and testing (TRL2–8 Appendix B). These activities are the last stage before the decision to build a new technology [30]. The increase in the level of risk is caused by changes in design, the setting of production equipment, and testing SOP. In the third stage, an assessment of the risks associated with the production equipment, materials preparation, integration process, and testing procedures must be carried out again.

According to risk theory, this third stage of development has a failure risk of about 56% [17]. The reverse approach will be helpful in this phase, especially assessment related to equipment, facilities, and HR requirements, the results of studies on raw materials, and the design process in the first and second deepening, along with test procedures. However, developing the third stage requires a more complex study than the previous stages. Activities using the reverse approach, which is mainly carried out using software, can produce elements with different characteristics, and this requires several iterations [16]. The components or subcomponents must still be tested to increase knowledge of their characteristics and to obtain a formulation close to the original. However, the number of tests is not as high as in forward engineering. After the characteristics of the elements are known, the technology concept will be adjusted to the following TRL sequence. In its implementation, elemental tests require quite a large amount of facilities, which are often found at other R&D institutions. Although it is more focused than forward engineering, reverse engineering requires regulation by the government to ensure success [22].

In 2005, PT. Pindad marketed a new product (SS-2) with several advantages, such as a more robust primary component material, more extensive gas pipe cross-section to minimise bolt circuit congestion, and several facilities to add new applications, such as binoculars and an ergonomic shape. Furthermore, some components, such as the magazine (see 601 Appendix C) and trigger circuit (see 501 Appendix C), can still be used on the SS-2. Before constructing the SS-2, PT. Pindad had some preliminary information, including user needs and competitors’ technologies, and they knew what essential elements had to be developed so that they could address weaknesses in existing technology. PT. Pindad could first use forward engineering by formulating a technology concept (TRL2 in Appendix B) and activities that are more focused on improving the performance with the condition that the selection of components and sub-components to be used can better utilise the raw materials available at the local level. The use of local materials can also be one way to reduce production costs. Furthermore, local materials can also increase the market potential at the local level.

This case study shows that the TRL calculator can be used as a reference to master technology using reverse engineering. However, reverse engineering must be carried out in stages. First, TRL definitions and indicators should be repeated to ensure that no procedures and processes are missed. This is carried out because reverse engineering moves towards greater uncertainty (a low TRL). This condition aligns with the theory presented by Makins (2009) and Ben Hicks et al. (2009).

Table 1. Additional models to master technology using reverse engineering.

Definition	Indicator
The technology has been observed.	Controlling organisation has been formed; Superior technology has been identified; Components and subcomponents have been modified.
The risk has been assessed.	A critical technology risk assessment has been carried out; Rearranging elements are known. Characteristics of recomposing elements are known and predictable.

shows an additional model consisting of definitions and indicators for mastering the technology through reverse engineering using the TRL calculator. The first level in Appendix B is replaced by an observation and risk assessment. However, the second level until the ninth level of TRL is still the same as in the case of forward (new) technology development (see Appendix B).

In addition to reducing the risk, additional activities, according to Table 1, can reduce the HR, material, energy, and financial investments in the process of mastering and developing technology.

6. Conclusions

Reverse engineering is a widely used method to accelerate the mastery of technology. The aim of this acceleration is to meet sustainable needs and increase competition. Furthermore, reverse engineering can save investment and the use of resources. Therefore, it must be carried out gradually to minimise the uncertainty derived from the lack of mastering the technology.

The TRL calculator is a tool to measure technological maturity in a forward engineering approach. From the literature study on the TRL, forward engineering, and reverse engineering, and using insights obtained from a case study in one defence industry in Indonesia, it is concluded that reverse engineering requires additional activities, such as observation, variant development, and technology redesign, to master the technology as a whole. These additional activities must be strengthened by establishing an organizational unit that can plan the reverse engineering activities, such as monitoring critical spare parts; maintaining existing equipment, facilities, and HR; collaboration and reviewing elements and their calculations. These activities are carried out so that reverse engineering can run efficiently.