Evaluating 3D Printers Using Data Envelopment Analysis

: Data Envelopment Analysis (DEA) is an established powerful mathematical programming technique, which has been employed quite extensively for assessing the efﬁciency/performance of various physical or virtual and simple or complex production systems, as well as of consumer and industrial products and technologies. The purpose of the present study is to investigate whether DEA may be employed for evaluating the technical efﬁciency/performance of 3D printers, an advanced manufacturing technology of increasing importance for the manufacturing sector. For this purpose, a representative sample of 3D printers based on Fused Deposition Modeling technology is examined. The technical factors/parameters of 3D printers, which are incorporated in the DEA, are investigated and discussed in detail. DEA evaluation results compare favorably with relevant benchmarks from experts, indicating that the suggested DEA technique in conjunction with technical and expert evaluation could be employed for evaluating the performance of a highly technological system, such as the 3D printer.


Introduction
In the 21st century global competition for manufacturing enterprises has become even harder; manufacturers of every size are trying to develop and sustain competitive advantages that will enable them to protect or increase their market share and enter new markets. The increased competitive pressure translates to higher requirements for both the production and product development processes, where process speed, flexibility, and continuous innovation are of increasing importance, besides the fundamental goals of high cost efficiency and quality. Additive Manufacturing and 3D Printing technologies are considered as key enablers for achieving the above goals at product design and development, as prototyping and visualization methods, as well as in the production floor, where they are commonly employed for the construction of production jigs and fixtures, and increasingly as the basic manufacturing method of the actual final parts and components [1][2][3].
The field of Additive Manufacturing (AM), encompasses an expanding and rather versatile group of manufacturing technologies where parts are gradually formed/fabricated through the successive stacking/joining of thin layers. Associated technologies, such as Stereolithography (SLA), Fused Deposition Modeling (FDM) or Fused Filament Fabrication (FFF), Material Jetting (MJ), Selective Laser Sintering (SLS), Selective or Electron Laser Melting (SLM/EBM), etc., usually process a limited set of materials that possess certain technology-specific characteristics (e.g., photocurable resins for SLA) and are supplied in a specific form (powder for SLS, filament for FDM/FFF etc.). The term "3D Printing" was initially (early 1990s) associated with the binder jetting AM process developed at the Massachusetts Institute of Technology. In the following years, its meaning has expanded, and it is often used as a synonym to Additive Manufacturing, especially in mainstream media and non-technical context, but this usage is quite inaccurate. In the present study, we employ the term "3D Printing" to refer to the fabrication of objects through the deposition ers [18], automobiles [19,20], and cell phones [21,22], as well as of advanced manufacturing technological systems and equipment, such as robots [23][24][25][26][27][28][29], CNC machines [30][31][32], expert systems [33], and flexible manufacturing systems [34][35][36][37]. Based on the above, DEA seems a suitable method for assessing the relative efficiency of currently available 3D printers. The corresponding results and information could prove useful both for potential users of the technology, in order to navigate the rapidly changing and evolving landscape of available systems and manufacturers, as well as for developers and manufacturers of systems, in order to assess the technical efficiency of their systems compared with that of competitors and aptly adjust their market and product development strategy. In this context the application of DEA for the assessment of 3D printers is investigated in the present paper.
Since it is difficult to compare 3D printing systems that are based on different technologies, mostly due to the associated material and technical differences, the study focuses on FDM/FFF technology, which is one of the most widely adopted and marketed 3D Printing methods. Materials used in FDM/FFF are typically based on thermoplastic polymers, such as ABS, PLA, and PETG, which are processed either in pure form or serve as the thermoplastic matrix for composites with several types of reinforcement (wood/metal particles, carbon fibers, glass fibers, etc.). The material is supplied in the form of filaments which is extruded from a heated moving nozzle, constructing thereby the thin layers that compose the part. Compared with other AM technologies, FDM/FFF is relatively simple from a technical point of view and allows the use of a relatively wide variety of materials (some metals and ceramics can also be processed), reasons which partly explain the popularity of the method and the high number of systems available in the market.
The rest of the paper is organized as follows. For the sake of completeness and integrity of presentation, a brief presentation of the theoretical background regarding the DEA method and its variations/models that are employed in the present study, is provided in the following section. Subsequently, the results from the investigation of a set of 38 3D printers based on FDM technology are presented and discussed in detail. In order to check the reliability of the proposed methodology the analysis results are compared with those of relevant benchmarks on the Internet. Finally, the basic concluding remarks and directions for further research are briefly discussed in the last section.

Theoretical Framework
DEA is one of the most popular and best documented techniques of mathematical programming, and several variations of the technique have been developed [38,39]. In the context of the present study, three DEA models/variations were examined: the CCR (Charnes-Cooper-Rhodes) model [13], the BCC (Banker-Charnes-Cooper) model [40] and the context-dependent DEA model [41]. The CCR or 'constant returns to scale' model is the oldest and most commonly employed variation of DEA [42]. In the CCR formulation of DEA, the technical efficiency θ * of each DMU (DMU 0 ) is evaluated by solving the following minimization problem [43].
where n is the number of DMUs, m is the number of inputs, s the number of outputs and λ j are the corresponding input/output coefficients (nonnegative scalars) assigned to each DMU. The above model is, also, commonly referred as the Farrell model [44]. The above formulation of the problem implies that the value of θ * cannot exceed 1. Accordingly, all DMUs for which θ * = 1 are considered efficient, while all other DMUs are considered inefficient. An efficiency score of 1 does not guarantee 'full efficiency' in the sense that some inputs could be reduced or some outputs could be increased without affecting the need for other inputs or the production of other outputs [45]. To assess this possibility slack values for DMUs are usually evaluated. The slack values s− and s+ of DMU 0 represent the amount that resource consumption or output production should be improved for the specific unit to reach the efficient frontier, and are evaluated by solving the following maximization problem.
Following slack evaluation, the identification of fully efficient DMUs is possible, for which both efficiency θ * = 1 and all slack values are equal to zero (s − * i = s + * r = 0). The BCC or 'variable returns to scale' model is formulated as the CCR model described above, but incorporates the following additional constraint: Efficient units, in this model also, are those that have the maximum efficiency score and zero slack values.
Classic DEA models help in evaluating the efficiency of DMUs and provide an indication of the distance to the best practice frontier, but do not provide specific guidance regarding how inefficient units may become efficient, i.e., whether outputs should be increased, inputs be reduced or both, and to what extent. Furthermore, as has been stressed in several studies [46,47], the performance ranking is often influenced by the context, i.e., the set of DMUs a specific unit is rated against. Especially in the case of product selection it has been argued that the choice context, i.e., what other options are available, is especially important for determining the "attractiveness" of an option [48]. For addressing the above considerations, the method of context-dependent DEA has been proposed [41]. In context-dependent DEA DMUs are classified in several subgroups/-sets which are considered efficient in specific contexts. This is achieved through successive optimization runs (levels), where efficient DMUs are assigned to a context-specific group and are subsequently removed from further DEA evaluation. The process is repeated until all DMUs are assigned to a specific group. The mathematical formulation of the corresponding linear programming model is the following [49].
where θ * (l, k) is the optimum efficiency of DMU k at level l, J l is the set of all DMUs at level l (J 1 is the initial set of all n DMUs), and x ij , y rj and λ j represent the inputs, outputs coefficient of DMU j respectively, following the previously adopted notation. Since the set of DMUs under evaluation at each level changes, the notation j ∈ F J l is employed for defining the relevant DMUs.
As described earlier the subsequent categorization of DMUs in context-specific DMUs is a repetitive process which begins at level l = 1 by evaluating the initial set J 1 , which is comprised of all n DMUs. Employing the standard DEA methodology described above, the subset E 1 of efficient DMUs at the first evaluation level, is defined. Subset E 1 is then removed from the initial evaluation set J 1 in order to obtain the second-level evaluation set J 2 and a new DEA evaluation is performed. The process is repeated until all DMUs are assigned to a level-specific efficient set, i.e., the set of DMUs for further evaluation is empty.
With context-dependent DEA it is, also, possible to further compare the efficiency of DMUs within each level-specific efficient set, by evaluating their attractiveness. A high attractiveness score indicates that a DMU is compared favorably with most other units, while a low attractiveness score will indicate a DMU with more referent DMUs from the inefficient stratum [50]. Attractiveness is, therefore, a measure of a DMU's relative efficiency within an efficient group, and is evaluated according to the following linear programming model [49].
The H * q (d) is called the d-degree attractiveness of DMU q for a specific level E l o . As noted earlier, this measure helps the decision-maker in comparing alternatives to others that belong to the same efficient frontier, as well as within various evaluation contexts, i.e., evaluate the attractiveness of an alternative for different peer sets. Another measure that can be evaluated in context-dependent DEA is that of progress. The progress score indicates the substantial improvement that a DMU needs to acquire to reach the frontier; hence the lower, the better. Progress for a specific DMU q is evaluated in a similar fashion, according to the following linear programming model [49].
The corresponding g-degree progress of a DMU q from a specific level E l o is evaluated in this case as follows.
Progress values are computed at each level and for all DMUs, except those belonging in the first-level frontier, and represent the incremental improvement a DMU must achieve in order to ascend to higher levels of efficiency [51].

Data Collection and Implementation
There is currently a plethora of 3D printers based on FDM/FFF technology that are available on the market. A rough estimation of available systems and machine vendors according to several specialized and general industrial web sites and sources (directindustry.com, 3Dhubs.com, 3dprintingindustry.com, 3ders.org, all3dp.com; all web sites accessed on various dates between March 2018-October 2019) shows that well above 200 models and systems from more than 40 vendors are currently offered worldwide. The corresponding prices for the great majority of systems falls mostly in the low to medium range (a few hundred to a few thousand dollars) but there are also high-end industrial scale systems available priced at several tens of thousands dollars. FDM/FFF printers can be supplied either fully assembled or as Do-It-Yourself (DIY) kits that are assembled by the user. In the case of DIY kits, the buyer will usually take into account the perceived difficulty of assembly and whether sufficient instructions are provided by the manufacturer; both factors that depend on the level of user's experience with 3D printers and are, therefore, quite subjective. In order, therefore, to avoid this non-objective parameter, printers provided as assembly kits were excluded from the analysis.
Based on the results of the preliminary analysis, a homogeneous initial population of assembled FDM/FFF printers, i.e., printers based on the thermoplastic material extrusion technology, offered at a price below USD 5000, has been constructed (273 printers). Low cost 3D printers based on other technologies, such as SLA or DLP, were excluded from the analysis because they are quite different in terms of materials and technical features. The corresponding final list contains several models from various companies including, but not limited to, the following (in alphabetical order): Anet, Craftunique, Creality, Dremel, Flashforge, Leapfrog, Lulzbot, Makerbot, Makergear, Monoprice, Prusa3D, Qidi, Raise3D, Ultimaker, and Wanhao. The majority of these printers are developed and offered according to the "low-cost online business model", whose main aim is to provide quality printers at low cost, with emphasis being placed at speed, ease-of-use, and durability [52].
In terms of materials, all identified printers can process standard PLA and ABS filaments. Depending on the configuration and capabilities of each printer, several other thermoplastic materials can be used. Some of the most commonly used thermoplastic materials for FDM/FFF printers and their basic properties and price range are summarized in Table 1. Printers equipped with two or more nozzles are considered capable of employing a secondary material, such as water-soluble PVA, for sacrificial support structures. Identified printers do not incorporate a heated chamber, since this a patent-protected feature of Stratasys and would, therefore, significantly limit the initial population. They can, however, incorporate a heated platform or an enclosed chamber. Other non-standard features, such as a moving/conveyer build surface or a rotating build table for continuous printing and automatic part removal, were not considered too, for the same reason. In terms of software, the corresponding printers are usually accompanied by speciallydeveloped slicing software but can, also, be programmed employing popular, freely available slicers such as Cura (ultimaker.com/software/ultimaker-cura, accessed on 26 April 2021), Prusaslicer (www.prusa3d.com/prusaslicer/, accessed on 26 April 2021), Craftware (craftbot.com/software, accessed on 26 April 2021) and/or Repetier (www.repetier.com/, accessed on 26 April 2021). The above software tools allow the user to appropriately define basic printer properties and parameters (workspace shape and dimensions, number of nozzles, filament diameter, etc.) to fit the specific printer. They offer, also, basic slicing features such as the definition of build parameters (build orientation, layer thickness, extrusion/bed temperatures, infill pattern and percentage, deposition speeds, etc.) and the generation of support structures. They include, also, modules for simulating/previewing the generated G-code file. Advanced features such as multi-material support, mesh modifiers, break-away support structures, variable layer height definition and special infill patterns (e.g., gyroid, Hilbert, or adaptive cubic) are, also, available.
For the identified printers, technical data concerning specific parameters were collected from relevant web sources, mostly from the manufacturers' official web sites but also from various 3D printing forums and sites. The technical characteristics, which were employed as parameters in the proposed DEA evaluation, were: build volume (effective machine volume), number of extruder heads, layer thickness (Z resolution), nozzle diameter, the existence or not of a heated plate, resolution of movement in the X-Y (layer) plane, variety of materials that can be processed by the printer, nominal power, printing speed, whether the printer is open source or not, weight, and, finally, the printer's market price. The 3D printer has been considered as a system whose inputs are associated with cost features, whereas its outputs are associated with its technical characteristics and capabilities. Therefore all the above parameters, but the last one, were evaluated as outputs, whilst market price was considered the sole input parameter. Taking the above statement into consideration, the primary objective of the evaluation is to evaluate whether DMUs can proportionally reduce their excess resources (cost) while keeping their characteristics constant. Thus input oriented DEA has been employed in the current analysis.
It is evident that regarding certain parameters, enhanced 3D printer performance is associated with higher corresponding values. It is preferable, e.g., for a printer to have a higher printing speed or a larger printing volume. Nevertheless, there are also parameters related to the printer's performance that we prefer to be as small as possible, e.g., the minimum layer thickness or X-Y resolution possible. To overcome the different, ascending or descending, nature of the parameters, which cannot be handled by DEA, a simple transformation of values for descending-nature parameters was performed. In this case, the 'transformed' values of parameter were computed by subtracting the actual parameter value from the highest corresponding value that was observed at the selected group of printers. This way, the 'transformed' parameter obtains its lowest value, i.e., zero, when the actual parameter attains its worst, i.e., its maximum, value. Ascending parameters are retained unchanged. Binary parameters, such as whether a printer is open source or is equipped with a heated platform, were given a value of one, when the answer was negative, and two otherwise. Being open-source was considered advantageous because it allows easier and lower-cost equipment maintenance and modifications. Finally, the parameter associated with material variety obtained values from 1 to 4 according the number of different materials that the printer could reportedly process.
Since the final number of systems, for which the majority of data were available, was quite high a smaller sample should be selected, for further analysis with prices that span the whole price range. The size of the final sample was determined by taking into account the number of outputs and inputs and the corresponding guidelines given in the studies of Bowlin [58] and Dyson et al. [59]. A random selection was performed after dividing the existing sample into six groups. Each group had a price range of a thousand of dollars (e.g., the first group from USD 0 to 999, the second from USD 1000 to 1999, etc.). From each group the 14% was randomly selected. For this purpose Minitab 17 was used. The final set of 30 3D printers and their technical specifications/parameters and prices are presented in Table 2.
For DEA according to the CCR and BCC models, an in-house DEA software was used, the code of which has been developed by the authors and runs in a Microsoft Excel environment. For context-dependent DEA, DEA Excel Solver was used as proposed by Zhu [60].

Results and Discussion
Before proceeding to the actual evaluation of the 3D printers with DEA, a correlation analysis of the 11 output parameters was performed in order to identify the minimum number of parameters that could be efficiently employed in DEA, i.e., investigate whether some of the initial selected parameters could be ignored without affecting the evaluation results. A reduction in the number of parameters is desirable because it lowers the associated computational burden and generally reduces the time required for the analysis.
The results of the correlation analysis of output parameters, performed with Minitab 17, are presented in Table 3. It can be observed that no strong correlation between any two of the output parameters exists. Nevertheless, there are pairs of parameters which seem to be somewhat correlated, as indicated by the fact that the corresponding p-values are lower than 0.05, but weakly, as indicated by the corresponding correlation coefficient values.
Among the initial set of output parameters, four seem to be more often correlated with others: build volume, nominal power, X-Y resolution, and speed. In order to check whether some of these parameters could be omitted from the analysis, a preliminary DEA evaluation of the printers' set was conducted. This evaluation involved two sets of computations, the first taking into account all eleven outputs while the second only seven of them, excluding the four correlated parameters. The sole input parameter in both evaluations was the printers' price and both CCR and BCC, input-oriented DEA was performed. The corresponding preliminary evaluation results are presented in Table 4.  Examining the values in Table 4 is quite obvious that when taking into account all output parameters the BCC model leads to a very high number of efficient systems (28 out of 30), i.e., almost all printers are evaluated as efficient. The CCR model, on the other hand, provided a broader distribution of efficiency scores that permits a more useful assessment of printer efficiency. This was, to some extent, expected since the BCC frontier is more flexible and envelops the data in a tighter manner. Another observation is that omitting the four output parameters (build volume, nominal power, X-Y resolution, and speed) leads to a reduction of efficiency scores for most printers in both models. This reduction affects also the number of efficient units identified for both models; something that is exhibited more vividly in the case of BCC DEA where efficient units are reduced from 28 to 16. A possible explanation for this deterioration of scores for some of the efficient printers is that these systems have relatively high values for the omitted output parameters.
Since the exclusion of all four parameters led to significant changes in the initially obtained scores, it has been decided to further explore whether some of the four output parameters under investigation could be excluded from further analysis. This investigation entailed the examination of different combinations of outputs, which included the seven output parameters that were definitely uncorrelated according to previous analysis (see Table 3), plus one or two of the parameters that exhibited weak correlations. The investigated combinations of outputs are presented analytically in Table 5. It should be noted that triple combinations of the four outputs under investigation were not considered simply because it does not make a lot of sense, computation-wise, to ignore just a single output/specification.
The DEA evaluation results, employing CCR and BCC models for the 10 different combinations of output parameters (shown in Table 5) are presented in Tables 6a and 6b respectively. Comparing the results for the CCR model presented in Table 6a to that of the analysis performed with all 11 outputs (Table 4) indicated that out of the 10 different output combinations the one that gives efficiency scores closest to those calculated using the complete set of output parameters, is combination (AD), i.e., employing the seven 'uncorrelated' outputs and the parameters of building volume and printing speed. A similar examination of efficiency scores for the BCC model (compare the corresponding column of Table 4 with the figures in Table 6b) shows that the 'closest to the complete set of outputs' combination is (CD), i.e., adding to the analysis the parameters of nominal power and printing speed. It is interesting to note, however, that in the BCC model almost all the 3D printers achieve maximum efficiency, same as observed in the preliminary analysis. Obviously, such a result has limited practical value as a filtering tool for assessing the performance of a 3D printer. The CCR method seems, therefore, more suitable for the problem at hand. Furthermore, as suggested in previous studies [61], the CCR model seems the most suitable method for evaluating sets of relatively homogeneous units, such as those investigated in the current study.
To further assess the suitability of the CCR method, statistical testing has been performed on the specific printer sample, as described in [45]. The test was implemented in R program language and employed the bootstrapping method for testing the following hypotheses: Hypothesis 0 (H0). 3D Printers excibit constant returns to scale (CCR).

Hypothesis 1 (H1). 3D Printers excibit variable returns to scale (BCC).
As indicated by the test results, H0 can be accepted at a 5% confidence level; hence the CCR model has been employed for further evaluation.
Having excluded the BCC model from further analysis and reduced the number of outputs to nine (building volume, number of extruders, layer thickness, nozzle diameter, presence of heated bed, number of materials, printing speed, open-source character, and weight) the analysis proceeded to the evaluation of an extended sample of 3D printers. The extended sample included eight additional 3D printers, numbered as printers 31-38 (see Table 7), which were highly rated and received favorable reviews from experts at specialized web sites (all3dp.com, www.aniwaa.com, 3dforged.com; all web sites accessed on various dates between July 2018-October 2019). The actual number of highly rated printers in the sample was nine, since printer 24, which was randomly included in the initial sample, received favorable reviews, also. Table 6. Evaluated efficiency of 3D printers, according to the CCR (a) and the BCC (b) model, for the output combinations of Table 5.  The DEA evaluation results for the extended sample, according to the CCR model, are presented in the second column of Table 8. The results of the analysis show a wide distribution of efficiency scores among printers and a rather clear distinction between efficient and less efficient units. In an effort to further reduce the number of output parameters to simplify the data collection and analysis process, the influence of nozzle diameter and open-source character on the evaluation results was examined. The evaluated efficiencies, excluding both and each of the above parameters are presented also in Table 8. Comparing the results considering all nine parameters (second column) to those obtained without considering nozzle diameter (third column), hardly any difference is observed. This is definitely not the case for the efficiency ratings evaluated when the opensource character (fourth column) or both parameters are ignored. Interestingly enough, the efficiency scores in the last two columns of Table 8 are almost identical. These two observations indicate that ignoring the nozzle diameter does not notably alter the overall 3D printer evaluation results and implies that the specific parameter could be also omitted. Such a behavior could be probably attributed to the fact that there is only a very limited variation of this particular output/specification among the 38 3D printers examined (the great majority of printers is equipped with nozzles of 0.40 mm diameter).
Examining the efficiency scores for the nine top-rated printers in the sample (printers 31-38 and printer 24), we observe that only five of them were evaluated as highly efficient. These are printers 35, 32, and 31, which display maximum efficiency, as well as printers 33 and 38, which achieve efficiency scores close to one. The remaining four top-rated systems (printers 24, 34, 36, and 37), on the other hand, failed to achieve high efficiency scores in the DEA evaluation. It should also be noted that all other printers, which were not considered as top choices from experts, did not obtain high efficiency scores, with the sole exception of printer 1 who was evaluated relatively favorably.
Some interesting observations can also be extracted by examining the relationship between efficiency and the price. Further investigation of this relationship is of interest because price is the sole input parameter in efficiency evaluation. As is clearly obvious in Figure 1, where the plot of a printer's price against its efficiency score is presented, relatively expensive 3D printers (price > USD 2000) are not evaluated as highly efficient. On the contrary, all printers that achieved high efficiency scores were relatively low priced; the three printers that achieved maximum efficiency were, in fact, among the cheapest ones. Accordingly, the low efficiency scores of the four underperforming top-rated printers (printers 24, 34, 36, and 37) seems to be due to their relatively high price. Focusing on the correlation between price and efficiency we observe a significant negative correlation, with −0.621 correlation coefficient and 0.000 p-value. It is interesting to note that the specific correlation coefficient value is quite close to the corresponding value computed by Doyle and Green [18] for dot matrix printers (−0.45). Above results could, also, be interpreted as an indication of higher economies of scale for manufacturers of relatively cheap printers, who can presumably withstand smaller profit margins due to higher numbers of units sold. This competitive advantage could explain the corresponding higher efficiency scores [30]. Possible correlations between efficiency scores and output parameters have also been examined, but no significant correlation values were observed.
Another interesting observation from the analysis of DEA results is that among the three efficient printers, printer 35 appears to be the efficient peer, or equivalently serves as the reference, for 31 'inefficient' printers, whilst printers 32 and 31 are the efficient peers for 22 and 11 'inefficient' printers respectively. A possible reading of this could be that printer 31 could be targeted at a specific market segment (niche) while printer 35 could have a broader market appeal. It should be noted that this characterization refers to the number of market competitors rather to the number of potential buyers.

Context-Dependent DEA
To further investigate the applicability of the DEA approach in the problem of assessing the efficiency of 3D printers, context-dependent DEA was applied. As described earlier, context-dependent DEA classifies DMUs in successive efficient frontiers/groups and assigns at each DMU attractiveness and progress values that can help in further assessing the technical efficiency of a DMU and how this can be improved.
The context-dependent evaluation of the extended sample of 3D printers resulted in the formation of eight efficient frontiers/groups, which are presented in the first column of Table 9, where the corresponding printers' attractiveness scores for the successive levels of analysis are also included. It can be observed that as the evaluation context level increases so does a printer's attractiveness score. It should also be noted that the relative ranking of printers according to the attractiveness score in each group, does not necessarily remains the same through all eight levels of analysis. Printer 32, for example, at the first group of printers E 1 , has higher attractiveness from printer 35 at the first six levels of analysis level, but this ranking is reversed at the last two levels. The relative ranking of printers in the second group E 2 , on the other hand, remains the same at all levels. It is, therefore, obvious that the evaluation context affects the ranking, as has been observed and in previous studies [62]. An examination of the attractiveness score of top-rated printers shows that the five of them that achieved high efficiency in previous analysis are also found in the first two efficient frontiers/groups, as expected. Among them, printer 31 is the most attractive choice at all levels of analysis. The remaining four top-rated printers are accordingly placed in the fourth (printers 37 and 34) and the fifth efficient frontiers (printers 24 and 36).
In Table 10 the printers' progress values are presented. An examination of progress and attractiveness values of printers in group E 2 reveals that printer 38 is the most efficient unit of the group. It should be noted that the specific printer obtained the second-best score at the initial input-oriented DEA, as can also be seen in Figure 2. As noted, however, in the study of [63], this is not necessarily the case. Printer 17, for example, is the sole member of the last context level, which could very well mean that it is the worst choice, despite the fact that it did not attain the worst efficiency score, printer 16 did. An examination of progress scores for both printers also shows that at some context levels (levels 4, 5, and 7) printer 16 requires more effort to improve than printer 17. Finally, it should be noted that the efficiency scores for printers belonging in the E 2 group are significantly lower from the corresponding scores for the rest of the printers (see Figure 2). This assessment is validated by the fact that E 2 printers exhibit the lowest progress scores at the first context level.

Conclusions
In the present study, DEA is proposed as a method of evaluating the technical efficiency of 3D printers. In order to identify the most suitable DEA approach for the specific task, both CCR and BCC input-oriented models of DEA are employed for assessing the efficiency of a sample of 38 3D printers, which are based on FDM/FFF technology and priced at less than USD 5000. The specific sample comprised mostly of systems randomly selected from a wide list of currently available systems, plus a small number of systems that received top ratings and reviews from 3D printing experts. The initial DEA evaluation model involved the assessment of price as the sole input parameter and of eleven 3D printer characteristics/specifications as output parameters. Employing correlation analysis and successive DEA evaluations for different combinations of outputs, their number was reduced to eight, namely, building volume, number of extruders, layer thickness, the presence of a heated bed, number of materials, printing speed, open-source character, and weight. According to the results of the analysis, CCR DEA seems to be the most appropriate model for the specific problem.
Employing CCR DEA, only three out of nine top-rated 3D printers achieved maximum efficiency, while another two exhibited efficiency scores greater than 0.9. The remaining four top-rated 3D printers were evaluated with efficiency scores lower than 0.5. To further investigate the efficiency, context-dependent DEA has been applied. According to the results of context-dependent DEA, five of the top-rated printers were assigned in the top-two efficient frontiers-a result that confirmed previous evaluations and rankings-while four were evaluated as relatively attractive in lower context levels. A possible interpretation of the result is that printers in the top-two levels are good all-around alternatives that combine good technical features with low price, while the remaining four are more attractive at specific contexts, where several other factors such as brand name, user base, and customer support are also considered. An examination of the price-efficiency relationship for the specific sample showed that there is a significant reciprocal relationship between the two values.
Regarding the applicability of the DEA method for the problem at hand, it could be argued that the results of the study show that DEA can be a useful approach for a preliminary assessment of the technical efficiency of 3D printers, based on data which are readily and easily available from the world-wide-web and without the need for subjective judgments regarding their values or relative importance. Relying on online sources of data may be considered a weakness of the proposed approach, but we believe that data reliability can be assured with a careful selection of the employed online sources. Considering the relatively large number of 3D printers offered world-wide and the rapidly changing landscape of the corresponding market, DEA may be employed for identifying a relatively small set of efficient systems out of an initial high number of possible alternatives, in relatively short time. This way 3D printer manufacturers may evaluate in short time the strengths and weaknesses of their products, compared to those offered by the competition, and adjust their product development and marketing strategies. Potential buyers could, also, consider the identified set of efficient units as a list of candidates for further evaluation where other quite important factors such as production quality and repeatability, system robustness, ease-of-use, and quality of after-sales service are assessed. The evaluation of all these factors requires extensive study and testing, and their importance to the buyer is of a more-or-less subjective nature. In this context, the proposed approach could be enhanced by incorporating a second evaluation step, where other multi-criteria decision-making methods are employed to further assess the performance of efficient printers. Future research could, also, focus on the investigation of other DEA approaches, which address some of the limitations of the classic models. In this context, a comparative study of convex and non-convex DEA approaches, would be particularly of interest.