A Joint Comparative Analysis of Routing Heuristics and Paperless Picking Technologies Using Simulation and Data Envelopment Analysis

Featured Application: The framework presented in this study can be served as a decision tool to help warehouse managers choose the right picking technology. Abstract: Recent literature demonstrates that warehouse order picking performance is reﬂected in the logistics performance of downstream retailers. Warehouse solutions and policies signiﬁcantly contribute to the improvement of distribution and delivery to retailers. This paper therefore reports an analysis of the joint performance of routing policies and picking technologies, and provides insights into the best ways to combine routing strategies and paperless solutions in order to optimize cost e ﬃ ciency. We follow a multistage approach that combines mixed integer linear programming algorithms, data envelopment analysis (DEA), and ranking and selection. The results show that traversal-voice picking and midpoint-voice picking combinations are equally distributed over the most e ﬃ cient subsets and that superior technology can enhance picking e ﬃ ciency only to a certain level. The study provides guidelines for logistics managers on ways to combine warehouse solutions and policies in order to better streamline the operations. It o ﬀ ers an original framework to analyze the joint performance of picking routing and picking solutions by considering the e ﬀ ect of picking errors. subsets contain either midpoint or traversal heuristic. In order words, based on the selection method,


Introduction
Order picking is one of the most studied topics in warehouse management due to the amount of resources involved in the process and its role in warehouses' performance. It is the most labor-intensive, expensive, and time-critical part of a warehouse or distribution center [1]. In fact, it can be viewed as a source of non-value-adding activities, which hinders the distribution and delivery to retailers [2]. It was estimated that 55% of warehouse cost can be attributed to order picking activity [3]. Due to this observation, researchers have sought to establish numerous methods and solutions to optimize the order picking process. The main target is reducing the travel time or travel distance, with or without consideration of human factors [4,5]. It is important to note that travel time constitutes 50% of the average picking time [6]. However, the most effective way identified to optimize the order picking process is the reduction of inefficiencies linked to human factors and operators' travel time by adopting the right routing policies and technologies. Against this backdrop, this paper contributes to the literature a report of the joint performance of routing policies and picking technologies, and insights on the best ways to combine routing strategies and paperless solutions in order to optimize cost efficiency. system does not guarantee a decrease in operating cost due to picking errors. The authors of Ref. [13] demonstrated that the introduction of modern technological solutions alone in the warehouse does not successfully eliminate errors. They devised a strategic framework that centers around three aspects of the warehouse: organizational, human, and technological. Unless the warehouse has a clear logistic strategy, well-trained order pickers, and adequate technologies, minimizing the number of errors will be limited. In order words, identical warehouses with the same technologies may differ in picking accuracy. Picking errors are therefore still existent in picker-to-parts order picking systems and may cause additional time to fulfill an order depending on the handheld or paperless technology.
In both practical and laboratory experiments, it has been shown that pickers' probability to make piking errors also depends on the type of device used and the warehouse configuration. For instance, Ref. [14] made a comparison of voice, handheld, and paper technologies and found that the use of handheld technologies was associated with lower errors than voice technologies. They concluded that handheld technologies could detect upstream errors, such as receiving, replenishment, or inventory control, better than the voice technology, since voice technologies require selectors to query the computer for additional information, which does not easily happen. The authors of Ref. [15], on the other hand, found that a low-level picking warehouse favors voice technologies comparative to handheld technologies based on the cost efficiency when they control for error occurrence, while Ref. [16] rank voice and handheld technologies equally in terms of errors' interception.
The literature has also proposed multiple new technologies to further improve productivity and accuracy during the order picking process. Although many of those technologies are still in the laboratory phase, they show promising results in terms of error reduction and productivity improvement. One of the proposed systems is the head-up display (HUD). In this system, the picker wears an HUD that contains the pick charts needed for each shelving unit. The next pick chart is shown when the picker drops items into the order bin. In all of the surveyed literature, the pick-by-HUD system surpassed pick-by-light and pick-by-paper for all the metrics analyzed. In terms of average task time, Ref. [17][18][19] reveal that the pick-by-HUD method was faster than the pick-by-light and the pick-by-paper methods. Likewise, the pick-by-HUD method was more accurate than all other methods considered. Overall, the pick-by-HUD method can significantly improve the performance of order picking processes.
Some other studies have also explored the joint role of picker personality and the use of picking technologies. They found that picker's personality has a significant effect on picking errors and throughput and that those effects vary with picking technology, such as voice picking and RF-terminal picking. For example, Ref. [20] showed that extraversion and neuroticism relate to picking errors, but the effect is dependent on the picking technology employed. They also found that voice picking is significantly more productive than RF-terminal and produce on average 21.4% less errors.
The authors of Ref. [21] reported on the acceptance of order picking support systems, such as pick-by-vision, pick-by-voice, or pick-by-light. All those systems outperform the paper-based system in terms of accuracy and productivity. However, it was revealed that adopting those technologies is not easy based on the perception of warehouse workers. In their consolidated review, Ref. [21] found that there are seven barriers to order picking support systems adoption: (1) an overwhelmingly high subjective task load, (2) loss of autonomy, (3) loss of social interaction, (4) negative influences from co-workers, (5) high complexity in handling the technology, (6) a lack of training, and (7) a lack of maturity of the technology. However, these may not be the only barriers to adoption. Implementation cost can also limit the adoption of paperless picking. Therefore, our study particularly considers the cost efficiency of paperless technologies with regard to picking accuracy or picking errors.
The literature has identified two kinds of picking errors: detectable and propagating [15,22]. The latter directly involved the distance or time travelled by the picker, since when a wrong item or quantity is picked, the picker has to travel back to the storage location to correct the error. The distance travelled is therefore associated with the routing method used, and with the picking technology adopted due to errors' occurrence. As the selected literature shows, routing methods and picking technologies differ in terms of performance. However, the joint performance of picking technology and routing methods is still unknown. Especially, how the joint performance of routing heuristics and paperless picking affect the cost efficiency of the order picking system is lacking in the literature. Therefore, this paper aims to help fill this gap.
Based on the selected literature, the S-shape or traversal is the most frequently used in practice [23]. This is primarily due to its intuitive nature to order pickers [5], and its route is less likely to be influenced by pickers' behavior, such as blocking. Therefore, in this study, our focus is on evaluating and comparing the performance of S-shape, return, and midpoint heuristics.
Furthermore, the order picking system is a highly cost-driven process involving labor and equipment costs. Although the main focus of the warehouse management resides on picking time reduction or productivity growth, the efficiency at which productivity is achieved is also very crucial due to budget constraint. For example, will a warehouse be better off investing in a voice picking system to increase the picking time and decreasing error occurrence? Considering that decision makers face multiple choices regarding paperless solutions, choosing the right one is proven to be a challenge. The performance of paperless solutions is largely influenced by several factors, including the size of the warehouse, the business activity, and the routing policy. Measuring the effect of routing decisions and technology on order picking cost efficiency is therefore very important.
Thus, the study reported in this paper assessed the joint performance of routing heuristics and paperless picking and their effects on order picking cost efficiency. We assumed that blocking is negligible, and deviation is inexistent due paperless technologies, and therefore focused on the picking errors. Most importantly, this analysis provides a framework to evaluate multiple warehouse configurations given different characteristics and budget constraints. It also offers a way to evaluate the performance of picking technologies and routing heuristics in different warehouse configurations. Finally, we propose a modified cost function of Ref. [15] to determine the input costs of the order picking systems. The rest of the paper is organized as followed: Section 2 contains the problem formulation and methodology; Section 3 deals with experiment's results and discussions; and Section 4 concludes the paper.

Materials and Methods
In this section, we describe the process undertaken to evaluate the routing heuristics, the paperless solutions, and the corresponding order picking parameters. The analysis is planned and structured based on the literature reviewed.
In practice, warehouse managers decide on a set of routing policies matching the existing order picking technologies, or businesses assess investment alternatives given different aspects of order picking systems, such as paperless solutions and expected order volumes and sizes. Therefore, a subset selection method is used to determine the best policies and parameters' combinations. It allows the significance of routing policies and parameters in efficient systems to be seen. The order picking systems are selected based on their efficiency values. The efficiency determination considers significant inputs, such as labor and equipment costs, incurred during the order picking process. Most importantly, the warehouse budget is taken into account. The system efficiency is consequently subject to the warehouse budget, size, and technological factors.

Stage 1: Parameters Selection
First, we specify the factors to be experimented. Based on the literature [24], we consider four different warehouse configurations (see Table 1). The distance between aisles is equal to 2.4 m. Each warehouse contains one block and two cross aisles, one at the front and one at the back. The depot is situated in front of the first aisle, the leftmost one. For each batch, pickers start at the depot and return to the depot. Items can be stored at both sides of the aisles. The picker travels to each storage position based on the sequence presented by the paperless device, which in turn depends on the routing method. Each warehouse is assumed to have different order distribution parameters, and the number of items per order varies with the warehouse and is uniformly distributed (see Table 2). According to our framework, orders are batched randomly from the pool of orders scheduled to be shipped at the earliest date. Likewise, orders are stored randomly within the warehouse.

Stage 2: Inputs and Outputs Determination
The costs associated with the warehouse operation typically include space, direct and indirect labor (fixed and variable), equipment (fixed and variable), overhead, and miscellaneous costs. According to the literature, labor can account approximately for as much as 60% of the warehouse costs. A large percentage of the labor cost goes into the order picking cost. The equipment costs, on the other hand, constitute 15% of the warehouse costs [8,25]. Since we assume a sort-while-pick batch picking system, sorting costs are not considered in the analysis.
The three input costs included in our study are therefore labor, storage, and equipment. One additional cost we consider is due to picking error that occurred during the picking process. According to the literature, there are two types of error that might occur when using a paperless picking device: detectable errors and propagating errors [15]. The detectable error is self-explained; it can easily be intercepted. It occurs when the order picker picks the right item but confirms the wrong one (e 1 ), or picks the wrong item and confirms the wrong one (e 2 ). The second type is, however, hardly recognizable. It occurs when either the wrong item is picked and the right one is confirmed (e 3 ), or when the wrong quantity is picked (e 4 ). The first type of error can be corrected on spot at the storage location, while the second one can only be seen at the end of the tour. In the first case, only the confirmation time or retrieval time will be increased, but in the second, both travel and retrieval time will eventually be increased. All four errors were considered to study their impact on the routing heuristics and picking efficiency.
In the literature, researchers have used different metrics to measure warehouse outputs, including movement, accumulation, and service level [26][27][28]. Movement is the number of items handled by the pickers. Here, we assumed that all items picked are shipped in their respective orders. Accumulation is the ratio between the total number of items and the total number of orders. Finally, the service level is the percentage of orders picked before their due time. We assumed that all orders are shipped the morning after they are picked. The three outputs used in our experiment are therefore movement, accumulation, and service level.

Order-Picking: Batching Strategy
In order to determine the number of items picked in a day and the associated costs, we need to find the average picking rate for each joint routing-batching strategy. Assuming a picker speed of respectively 45, 65, and 80 m per minute, the picking rate is the average number of items picked by a picker in an hour. We use the mixed integer linear programming (MILP) model developed by Ref. [24] to first simulate the average picking distance traveled by a picker per batch given the picking cart capacity, and then use Equation (1) to get the picking rate. In Equation (1), wij designates the average picking rate at warehouse w when routing policy i and technology j are used. S w , K w , and D wij are respectively the picker's speed, the picking cart capacity, and the average distance traveled per batch. t wij net represents the time of getting information about the product, searching, picking, and confirming the product picked. Basically, to determine the picking rate, we randomly generated each order and its items following a uniform distribution. Orders and items are respectively assumed to be between 1 and 14, and between 5 and 19. Each cart picked constitutes a batch. The distance travelled is approximately proportional to the number of batches and t wij net , allowing the picking rate to be applicable to any distance used in Equation (1): The picking rate was determined for each policy set presented in Table 1. As Equation (1) shows, the higher the capacity of the picking cart, the higher is the picking rate. Likewise, the longer the travel distance, the lower the picking rate.
To minimize the relative error corresponding to the travel distance estimation and thus to the picking rate, we used the following equation: where γ = X−µ |µ| and γ = γ 1−γ . Equation (2) yields the number of replications to make of each simulated travel distance to make sure that the relative error γ does not exceed the confidence level α, since µ, the travel distance, is unknown. X represents the estimated travel distance. S 2 (n 0 ) is the variance of the first n 0 replications.
i replications are made until is less or equal to γ . We chose γ = 0.1 and α = 0.05.

Input Costs Functions and Outputs Variable
Now, using the picking rate and the batching model proposed by [27], we were able to generate the number of daily fulfilled orders and an estimation of the associated costs incurred by the warehouses.
The outputs are defined as followed: where M wij is the sum of daily items fulfilled from each order and batch given the warehouse, the routing policy, and the type of paperless technology. We assumed that there is an 8-h shift and a maximum of 2 h overtime per picker (if all the orders are not fulfilled during the 8-h shift). If an order is not picked in either time period, it is considered unfulfilled and the service level for that day will be less than 1 (the highest value). L wij is the service level of the daily picking process and takes values between 0 and 1, where Q wij is the total daily ordered items. A value of 1 indicates that the total order made in a day has been fulfilled and ready to be shipped on schedule. A wij is the accumulation level, where O wij represents the total orders. It reflects the level of effort required of labor inputs to fulfill the sum of orders in a working day. We present a modified cost function C wij d of [15], which comprised four main daily cost components: The cost function in Equation (6) can be written based on the notation in Table 3 as follows: The total daily required time T j ek to fix errors is subject to the paperless technologies used during the picking process. For example, for barcodes handheld, if the error e 1 occurs, the picker will need some more time to confirm the right item and will therefore spent t j c unit of time to do so. If the e 2 occurs the picker will spend 2·t j net time to correct the error. This is because the picker will need twice as much time to get information about both the right and wrong product, to search, to pick, and to store. On the other hand, if e 3 occurs, additional 2·t j net + t trav time will be needed. For the third error type, additional travel time is required because this type of error is usually detected at the depot. Finally, in the case of e 4 , as much as t j tot time will be needed. The reader can refer to [15] for a detailed explanation of the different errors and corresponding time. The time needed during the picking process are expressed as follows: where t wij tot is the total daily time spent picking orders, which equals to the number of lines or items fulfilled divided by the picking rate. t wij net is the total time required to get information about a specific batch, search its product, pick, and confirm. t wij trav represents the total time needed for all pickers to travel over to the storage locations: where t j i represents the average amount of time needed to get the information of the products from a batch to pick; t j s is the average search time; t j p is the average pick time, which is the time to pick the item from the storage location and store it in the cart that is multiplied by the average number of items to pick M wij ; and t j c is the average confirm time. b wij is the total number of batches picked in a day.

Stage 3: Efficiency Determination Using DEA
The data envelopment analysis is a non-parametric tool used to evaluate the relative efficiency of decision-making units (DMUs). It was first developed by Ref. [29] in their seminal work "Measuring the efficiency of decision making units". Due to the few assumptions considered in DEA, it is an appropriate instrument for evaluating warehouse order-picking systems and policy sets. The warehousing literature lacks strong hypotheses about the warehousing production function and there is no reliable production function specification test [30].
For our problem, we have a set O = {O 1 , O 2 , . . . , O n } of n policy sets. Assuming variable return to scale, the efficiency of the lth policy set in a warehouse can be defined as follows. For the sake of simplicity, we replace C wij d by C wl d , by changing index i and j to l. l represents routing policy i given the technology j: min St : M ≤ l w l w The linear programming seeks for a set of weights (β wl ) that minimize the cost function in Equation (10). If it is not possible to further reduce the cost function C d , in other words, if the minimized cost is equal to the actual cost, the policy set O l is efficient. Otherwise, it is considered to be inefficient. Equations (11)- (14) make sure that the virtual cost is less or equal than the actual cost, while Equations (15)-(17) restrain the virtual output to be greater or equal than the actual output. Equation (18) deals with the return to scale assumption, and assumes variable returns of activities.

Stage 4: Statistical Sampling and Subset Selection
Since our goal is to find the best routing policy and technology combinations, we need to make sure that the selected systems really are the best. To do so, we used ranking and selection approaches. Usually, they determine the amount of data to sample or the number of replications from each population, or allocate the number of replications to each alternative from a fixed total computing budget. There are several approaches developed by researchers over the years, including the indifference-zone approach, the optimal computing budget allocation sequential approach (OCBA), and the enhanced two-stage selection (ETSS) [31][32][33]. However, they all present different drawbacks. Due to the least configuration assumption (LFC), the indifference-zone approach is known to be conservative as LFC is rarely encountered in reality. The OCBA, on the other hand, assumes that the true standard deviation of each alternative is known. Lastly, for the ETSS, there is no guarantee that the probability of correct selection is minimal [31]. Although the indifference-zone approach is conservative, it presents the advantage of being able to pre-specify a probability of correct selection, and set up a mean difference threshold based on the data at hand. Our goal is to rank the different policy combinations based on specific differences in the efficiency values. Therefore, we adopt the indifference-zone approach, more specifically the method developed by [34].
The Koenig and Law method is a two-stage approach of selecting a subset of systems containing the best performing ones. The first stage consists of implementing a number of initial replications and finding the first-stage sample means. Based on the specified indifference amount and the desired probability of correct selection, additional random observations and the second-stage sample means are determined. Finally, the weights are defined to obtain the weighted sample means. Two systems are considered equal in performance if the difference between their mean efficiencies is less than the indifference amount. Figure 1 summarizes the process followed to estimate outputs and cost components to determine the cost efficiency and select the efficient policies' combinations.
Appl. Sci. 2020, 10, x FOR PEER REVIEW 9 of 18 budget. There are several approaches developed by researchers over the years, including the indifference-zone approach, the optimal computing budget allocation sequential approach (OCBA), and the enhanced two-stage selection (ETSS) [31][32][33]. However, they all present different drawbacks. Due to the least configuration assumption (LFC), the indifference-zone approach is known to be conservative as LFC is rarely encountered in reality. The OCBA, on the other hand, assumes that the true standard deviation of each alternative is known. Lastly, for the ETSS, there is no guarantee that the probability of correct selection is minimal [31]. Although the indifference-zone approach is conservative, it presents the advantage of being able to pre-specify a probability of correct selection, and set up a mean difference threshold based on the data at hand. Our goal is to rank the different policy combinations based on specific differences in the efficiency values. Therefore, we adopt the indifference-zone approach, more specifically the method developed by [34]. The Koenig and Law method is a two-stage approach of selecting a subset of systems containing the best performing ones. The first stage consists of implementing a number of initial replications and finding the first-stage sample means. Based on the specified indifference amount and the desired probability of correct selection, additional random observations and the second-stage sample means are determined. Finally, the weights are defined to obtain the weighted sample means. Two systems are considered equal in performance if the difference between their mean efficiencies is less than the indifference amount. Figure 1 summarizes the process followed to estimate outputs and cost components to determine the cost efficiency and select the efficient policies' combinations.

Results and discussions
In this section, we present the performance analysis of the routing heuristics based on the four warehouse configurations and three paperless picking solutions. The cost inputs are determined based on a 2-year investment horizon and reflect the technological structure of each picking system. As indicated in Table 1, each warehouse has distinct characteristics. For example, warehouse layout 4 contains 20 aisles with 40 storage positions each, but pickers in that warehouse can travel 65 m per minute on average. This speed reflects the type of picking devices used in warehouse 4. Likewise, the picking cart capacity is 48. The capacity and speed are chosen to correspond with the size of the warehouse. Warehouse 4 uses a larger cart to offset the long distance that a picker might need to travel per batch. The more order a picker can pick per batch, the smaller will be the average total distance. However, due to weight constraint and safety concerns, pickers in warehouse 4 travel at a lower speed than the middle size warehouses (2 and 3). We assume that picker blocking is negligible. Following [15], the cost components are derived as in Table 4. Our cost values are adjusted to reflect

Results and Discussions
In this section, we present the performance analysis of the routing heuristics based on the four warehouse configurations and three paperless picking solutions. The cost inputs are determined based on a 2-year investment horizon and reflect the technological structure of each picking system. As indicated in Table 1, each warehouse has distinct characteristics. For example, warehouse layout 4 contains 20 aisles with 40 storage positions each, but pickers in that warehouse can travel 65 m per minute on average. This speed reflects the type of picking devices used in warehouse 4. Likewise, the picking cart capacity is 48. The capacity and speed are chosen to correspond with the size of the warehouse. Warehouse 4 uses a larger cart to offset the long distance that a picker might need to travel per batch. The more order a picker can pick per batch, the smaller will be the average total distance. However, due to weight constraint and safety concerns, pickers in warehouse 4 travel at a lower speed than the middle size warehouses (2 and 3). We assume that picker blocking is negligible. Following [15], the cost components are derived as in Table 4. Our cost values are adjusted to reflect the warehouse size and new information obtained from the industry catalogue and the literature. The usage hours are calculated using an eight-hour work shift for 220 days a year. We used CPLEX 12.10 (with default options and 60 s time limit for each run) to solve the MILP models Ref. [24,27] and calculate the cost efficiency values, Matlab R2020a to implement the method of [34], and Microsoft Office Professional Plus 2019 to compile the data generated. The software are run on an ASUSPRO with 3.00 GHz processor and 24 GB RAM with a Microsoft Windows 10 Enterprise operating system. As previously mentioned, the time required to process a batch varies with the paperless technology and the number of items. Likewise, the time needed to fix an error depends on the technology used. The three paperless solutions considered in our experiment are barcodes handheld, RFID tags handheld, and voice picking. Those three technologies are chosen because they allow order pickers to move freely within the warehouse.
For the ranking and subset selection, the parameters are chosen as follows:

Picking Rate
In a preliminary analysis, the picking rates, calculated using mixed integer linear programming, offer some insights on the performance of the routing heuristics. The traversal or S-shape routing yields the highest number of items per hour on the average of warehouses and paperless technology, followed by the midpoint policy. Midpoint, however, outperforms traversal in warehouse 1. As expected, the medium size warehouses have a higher picking rate than warehouse 1 and 4. Although warehouse 2 and 3 are bigger than warehouse 1, they have a faster picking device and can handle larger batch sizes. In terms of paperless technology, the voice picking largely outperforms both handheld devices whose performance looks similar (see Table 5).

Order Picking Performance
According to the subset selection, the 10 most efficient subsets are presented in Tables 6 and 7. The picking errors are assumed to occur with a mean of 4.5% (for error e 1 , e 2 , and e 3 ) or 5% (for error e 4 ) probability, and represent the baseline error probability. In the experiment, multiple scenarios were analyzed to investigate how each routing policy performs in each one of them. Each scenario depicts how picking errors affect the routing policies and picking technologies. Table 6. Distribution of the 10 best performing subsets.

Performance of Routing Policies
In terms of subset performance, midpoint and traversal heuristics perform equally well in 6 (E, ERb = O, E = 0, e 3−4 = 0, R e 3 = 0, and R e 4 = 0) of the 10 scenarios. In each case, equal number of subsets contain either midpoint or traversal heuristic. In order words, based on the selection method, they are equally likely to be selected by a decision maker based on the 10 best performing subsets. Furthermore, in terms of the efficiency level, 7 of the 10 scenarios yield equivalent results for both midpoint and traversal heuristics. This is contrary to the literature, which show either one of those heuristics outperforms each other in many instances. This result may be due to the fact that, since items are almost evenly distributed across the warehouse, in both cases, both sides of the aisles are equally likely to be traveled by a picker. Return policy remains the least performing heuristic. Over all the scenarios, most of the return policies are present in subsets containing small warehouse layouts. This confirms the results of the literature, which found that return routing to perform the best in cases of small pick density or order length [35]. However, to evaluate the effect of errors on its performance, we assess its efficiency without errors given that errors are observed in both midpoint and traversal routing. According to the results, return routing without propagation errors and midpoint routing with errors equally perform. Likewise, return policy without error outperforms both midpoint and traversal policies with errors. Therefore, picking errors largely contribute to the underperformance of return policy.

Performance of Solutions Technology
Voice picking technology is far the most competitive overall (see Table 6). Except in one case, voice picking is present in all of the subsets. This result indicates that voice picking technology contributes more to the efficiency of order picking systems irrespective of the warehouse configuration and routing policy. The voice picking has an edge over the handheld devices because confirmation activity is done during picking and therefore takes a shorter time to process orders. Error costs are therefore relatively lower for voice picking technology. When handheld without e 3 or e 4 and voice picking with errors are evaluated, voice picking technology still outperforms the handheld devices (sixth and eighth columns in Table 6). However, a further analysis of the picking errors effect shows that handheld without propagating errors and voice picking with errors equally perform (10th column in Table 6). This shows that only when handheld devices are propagating errors free that they can perform as well as voice picking technologies. Therefore, reducing propagating errors can play a significant role in enhancing order picking efficiency. Reducing propagating errors leads to less travel time and higher service level. Given that voice picking technologies are more expensive, decision makers can reduce the operating cost by implementing handheld technologies and at the same time eliminating or reducing propagation errors.

Performance of the Warehouses
Warehouse 1 and 3 made up the majority of the most efficient subsets in almost all of the scenarios. In every scenario, at least 6 out of 10 subsets are from warehouse 1 and 3. They have a shorter aisles length, which might contribute to their good performance as order pickers spend less travel time within shorter aisles, especially when the traversal routing method is used where the picker has to cross the entire aisle that has at least one pick. In fact, the literature demonstrates that having shorter aisles by increasing the number of cross aisles can, to a certain level, decrease the order picking distance and be beneficial to the effectiveness of routing policies [36]. Our results therefore suggest that warehouse managers should consider adopting warehouse configurations that have short aisles by increasing either the number of aisles or cross aisles.

Sensitivity Analysis of the Effect of Error on the Order Picking Performance
We evaluated the effect of the stochastic behavior of the picking errors on order picking (see Table 7). The probability of error is set to vary around the mean with standard deviation of respectively 0.01, 0.02, 0.03, 0.04, and 0.05. Not many changes were observed when the errors probability changes over the experimented samples compared to previous results where error probability is fixed. Midpoint and traversal routings performance, in terms of subset presence, remain equally the same; except when the standard deviation of error probability is equal to 0.02. In that case, the midpoint performs the best. However, very small reductions were observed in the efficiency level. As previously found, at least 9 out of 10 subsets from each scenario contain voice picking technology. Likewise, warehouse 1 and 3 perform the best as previously found. In sum, the performance of routing policies and solution technologies is robust against error deviation.

The Effect of Order Quantity on the Cost Efficiency Level
We evaluated how order quantity affected the performance of cost efficiency (see Table 8). In warehouse 1, the midpoint heuristic is the most efficient for all the order quantity considered, followed by the traversal heuristic. This is in line with our previous results in Table 6 where almost half of the subsets for warehouse 1 contain the midpoint heuristic. This supports the literature Ref. [12] that the midpoint outperforms the traversal and return policies for orders containing a small number of picks. The midpoint-voice picking combination results in the best order picking performance for warehouse 1.
On the other hand, traversal routing yields the best performance in terms of efficiency levels in warehouse 2 to 4. These are considered to be respectively medium-and big-sized configurations in our study. As previously found, warehouses 2 to 4 have more subsets containing traversal routing. Therefore, the results are in line with the selected literature Ref. [12] that for a large number of picks traversal routing performs the best. In warehouses 2 to 4, the traversal-voice picking combination performs the best. In terms of solution technologies, voice picking still dominates the results. It should, however, be noted that handheld barcode outperforms voice picking in cases of a large amount of orders relative to the warehouse considered. For example, for 500 orders, the barcode handheld leads to the highest level of efficiency when either return or traversal routing is implemented. On the other hand, for 800 and 900 orders, the barcode handheld outperforms voice picking when either midpoint or return routing is used. Although voice picking yields relatively higher picking rate, handheld picking is more cost efficient when warehouse activities reach a certain level due to a greater increase in the voice picking cost compared to the handheld picking cost. Further analysis shows that large order sizes cause order pickers to do overtime shift and thus further exacerbate the operating cost. To reach the optimal service level, in that case, warehouse managers have to trade off cost efficiency. This implies that warehouses are able to provide a service level to a certain level at a maximum possible efficiency.
Beyond that level, the warehouse managers will have to sacrifice cost efficiency in order to attain a 100% service level. However, due to a budget constraint and operation capacity, a 100% service level is not attainable. Therefore, the joint performance of routing policies and solution technologies also depends on the warehouse activity level.

Conclusions
Although several paperless technologies have been introduced to help reduce picking errors in manual order picking, they still represent a challenge for warehouses and their occurrence depends on the type of paperless technology used, and the routing methods utilized. In other terms, operating cost can substantially be decreased if the right routing policy is implemented with the right paperless technology. This paper therefore developed a framework to evaluate the performance of routing heuristics in the presence of picking errors. To do so, we followed multiple stages, including a careful selection of the parameters, efficiency determination, and statistical sampling and subsets' selection. We proposed a modified cost model following Ref. [15] composed of four main components capturing the main cost involved in order picking. Contrarily to [15], our modified approach emphasizes the travel time used by pickers given the routing policies.
Our results offer several insights, some of which are new to the literature, on the joint effect of routing policies and technology solutions. The subset selection approach reveals that traversal-voice picking and midpoint-voice picking combinations are equally distributed over the most efficient subsets but benefit one type of warehouse size compared to others. Therefore, managers have a certain flexibility in the selection process. In certain cases, the most efficient policy sets may not be the robust one. The reduction or elimination of picking errors, most particularly the propagation errors, is crucial to the efficiency of the order picking system. Superior technology can enhance the picking efficiency only to a certain level due to cost implementation and warehouse activity. Short-aisle warehouses perform the best and therefore should be considered as alternatives, either by introducing longer rectangular warehouses or multiple cross aisles in the warehouse.
This study shows that integrating picking errors in the analytical framework provides a strong practical way to assess order picking problems. It also highlights the importance of selecting key parameters in an analytical framework. Further study can integrate other human factors, such as picker blocking and deviations, together with picking errors, and other storage assignment methods, and so on.

Conflicts of Interest:
The authors declare no conflict of interest.