Industry Survey on the Current State of Stope Design Methods in the Underground Mining Sector

: Stope design is a core discipline within mining engineering. This study analyzes the current state-of-the-art of stope design through a survey addressed to mining industry professionals. In stope design research the dominance of empirical methods has slowly shifted towards numerical methods. Recent advancements have mostly focused on the development of stope optimization algorithms. The survey consisted of 19 questions and was distributed to stope design experts via email, LinkedIn messages, and the Mining Industry Professionals network forum. In total, 36 responses of satisfying quality from 20 countries were received and analyzed. No dominance of a single stope design method was recognized. Empirical methods and personal expertise are still used widely. However, a readiness for change in stope design practice was indicated in 87% of responses. The current needs of the stoping-based underground mining sector are to increase the amount of geotechnical data, automate stope design and implement related software, and integrate these into general mine planning. According to 70% of the participants, acquired geotechnical data should be available within three days to be employed in design practice. The industry is ready to implement more efﬁcient stope design methods if they offer results proven in case studies.


Introduction
Underground mining operations are progressing into deeper and increasingly more complex deposits, associated with more challenging geological conditions, while the demand for minerals and metals continues to grow rapidly.Additionally, ore grades are decreasing, and thus, valuable metals will more likely occur as refractory or trace minerals in future mining operations [1].With the goal of transforming the mining industry into a more sustainable one, the necessity to utilize mining methods with limited impact on the surface is becoming ever more urgent.To satisfy this need, the application of non-caving underground mining methods offers a convenient solution and presents a promising prospect for increasing the importance of these mining methods for the mining industry.Brady and Brown [2] divide mining methods into three main categories: pillar supported, artificially supported and unsupported.Each category is further divided into specific mining methods with varying degrees of rock mass response to mining.The pillar and artificially supported, so-called stoping-based, mining methods, with no or very limited subsidence, are the solutions of choice for the future challenges of the mining industry [3].The lack of visible environmental damage from these mines allows for a more positive public perception of the mining industry.These stoping-based mining methods rely on diving the orebody into stopes for ore production and have already become amongst the most common methods in modern underground mining.This is due to their inherent operational safety, reliable design, and cost efficiency [4].Therefore, this study focuses on stoping-based mining methods.Additionally, increasingly challenging geological conditions can potentially result in more hazardous working environments.To prevent this, adequate consideration during mine Energies 2022, 15, 240 2 of 20 planning and design are deemed necessary [5].Stoping-based mining methods are prone to dangers related to instability of the surrounding rock mass.This creates a considerable danger to both people and equipment [6].Thus, stoping-based mining methods are reliant on adequate estimations of the stability of the rock mass in which they are applied, to allow their application.This condition results in considerable efforts being applied to enable reliable estimations, predictions and dimensioning of the stopes to be mined.
Empirical methods have been used in geomechanics since their development in the 1970s.Most commonly they are used for preliminary design in the early stage of mining [7][8][9][10].The empirical methods include, but are not limited to: rock mass rating (RMR) system [11], Q-system [12], geological strength index (GSI) [13], mining rock mass rating (MRMR) system [14], Laubscher block caving rules [15], hard rock pillar design chart [12,16], and the tributary area method for naturally supported mining.A comprehensive overview of the most common empirical methods within geomechanics can be found in [7].For stope design, empirical methods can be grouped within three commonly used design methods [17], the stability graph method and its adaptations [18], the Hoek and Brown failure criterion [19,20], and the span design graph [21,22].The most well-known empirical method for stope design is the stability graph method.Over time, several issues have been identified as intrinsic to the stability graph (e.g., limited database entries) [23,24].Thus, the original stability graph was extended, modified, and the underlying system adapted to different needs and challenges of stope design, for various mining methods, mine environments, and geological conditions .The Hoek and Brown failure criterion is an empirical failure criterion that has undergone repeated revisions since its introduction in the 1980s [19,[46][47][48][49][50].Even though it is an effective tool for estimating the strength of jointed rock masses, there is a high degree of uncertainty of estimations related to the evaluation of in situ rock mass conditions being heavily reliant on subjective interpretation of qualitative guidelines [51].The span design graph [21,22] method relies on comparing rock mass data and geometry of stope designs to case histories and uses the RMR76 (rock mass rating) system.The RMR76 classification is plotted against the desired span while considering, in a rudimentary way, the joint orientations and stress levels.One of the most recent developments in empirical methods was the creation of novel empirical software that enables the creation of regional and mine specific case studies [43].
Numerical modeling methods for modeling underground stopes have become ever more important with the advancements in computing power which allows numerical modeling of the behavior of rock mass for increasingly complex scenarios [44][45][46][47][48][49][50][51][52][53][54].Both continuum-based [55][56][57][58][59], and discrete numerical methods [60][61][62][63][64][65] are used in stope design.An important advancement in numerical approaches to stope design is the ability to simulate jointed rock masses to analyze stope stability [66,67].Stability analysis of stopes is based on the characteristic mechanical properties of jointed rock masses.Therefore, deformability and strength of a fractured rock mass must be evaluated first.The simplest approach is to use the equivalent continuum (EC) method to estimate strength and deformability properties, using empirical, analytical, or numerical methods and to up-scale them for the region of interest [68].In a more comprehensive approach, geometrical fracture properties are characterized by field mapping, geostatistical, and geophysical methods [69][70][71][72] or remote sensing [73,74], and the mechanical properties of both intact rocks and rock discontinuities are measured in the laboratory or field.This enables large-scale modeling and stability analysis of multiple stopes, and the stability of each stope can be analyzed by numerical deterministic or probabilistic approaches e.g., [75][76][77].Another approach for formulating a fractured rock mass model uses explicitly defined discontinuities with properties back-calculated using in situ measurements and the EC method for the surrounding rock mass.This can allow for efficient modeling of the whole mine and stope optimization [78].Another method implemented numerically is the back-analysis method for stope design aimed at analyzing how previous stope designs behaved compared to their expected behavior.This allows for a reconciliation process, where the results from back-analysis are used to adapt the design of future stopes [79].Recent research related to stope design has not advanced to the same extent as research in other mining-related areas [80][81][82][83].The research is mainly focused on optimization of stope boundaries, production schedule, and development activities [9].An overview of specific research topics with a selection of featured research papers is given here:

•
The incorporation of grade uncertainty and minimization of associated risk on stope design optimization [84,85

•
Improvements in the efficiency of stope optimization algorithms [83] Many studies have focused on stope design optimization algorithms due to the larger computing power available [96][97][98][99][100][101][102][103][104][105][106][107][108][109][110].The first group of stope optimization algorithms is level-oriented algorithms that are applied at the single mine level, including Riddle dynamic programming [111], the branch and bound method [99,100], mixed integer programming [84], optimum limit integrated probable stope (OLIPS) [112][113][114], and the global optimization for underground mining area (GOUMA) algorithm [115].The second group is field-oriented algorithms where the entire mining area is considered for the determination of mining limits, including the floating stope algorithm [103,116], multiple pass floating stope process (MPFSP) [101], maximum value neighborhood method, nested stopes [104], heuristic approaches [97,106,117,118], network flow method [119], octree-division [120], and integrated method [88].However, the transition from algorithms working in 2D towards algorithms realizing a truly 3D optimization is still advancing as there are only a limited number of 3D algorithms available [83].A wide variety of underground mine planning and optimization software tools have been implemented recently [121].In general, the recent developments can be understood as a growing trend towards developing new software tools to overcome problems in design, scheduling and optimization.Software for stope design and mine planning and their working principles are explained in more detail in [121].
Even though a large amount of stope design research on various aspects is published in the literature, the level of implementation of this research in the mining industry is not clear.No existing study has been found in the literature that addresses this problem.Therefore, this study aims to adequately identify the degree of stope design research implementation in the mining industry and to characterize the most urgent challenges related to stope design.Thus, a questionnaire, following principles of questionnaire design and statistical science [122,123], was created and responded to by mining industry professionals who work with stope design.The key value of this work relates to the assessment of the current research focus on the one hand, and the identification of industry needs on the other, as well as the relationship between these.

Questionnaire Development
The WebRoPol electronic survey system [124] was utilized for the development of the questionnaire.This allowed for the creation of various interactive features and facilitated the distribution of the questionnaire, as well as enabling evaluation of the results.
The questionnaire consisted of 19 questions with multiple choice and free answer options addressing 19 topics (see Table 1) and was designed based on state-of-the-art survey methodology [123].The design adhered to scientific survey principles, such as balanced multiple-choice response options, non-mandatory responses, randomized answer option Energies 2022, 15, 240 4 of 20 order, and use of a multilayer funnel approach.The questionnaire started with general questions, evolving into detailed questions about stope design.While each question required specific responses, each question also allowed for detailed free-text answers.The questionnaire was represented on several small pages, based on specific topics.To minimize information overload, additional answer options were only shown when the previous option was selected.
Table 1.Questions asked in the industry survey on stope design methods.

Q1.
Provide your contact details and position Q2.
Do you wish to be notified about the results of the survey?Q3.
Which exact stoping/mining methods do you apply?Q4.
Have there been changes in your applied mining/stoping methods and if so why and when?Q5.
Which mining methods did you use previously?Q6.
Which methods are you currently applying to design your stopes?Q7.
How long does the process of designing stopes usually take and how many people are commonly involved?Q8.
How important are different parameters for the stope design (1 = very important; 5 = very unimportant)?Q9.
How challenging are the geological conditions in your deposit?Q10.
Are your stopes checked for accuracy and stability after excavation?Q11.
Would you recommend your current stope design and stability estimations to other mines?Q12.
What would you consider the most valuable improvement to the stope designing process?Q13.
Have you utilized any geochemical, geophysical and/or geomechanical investigation methods for stope design before?Q14.
Are you planning to utilize geochemical, geophysical and/or geomechanical methods for your stope design?Q15.
Would you be interested in applying new rock/ore characterization methods? Q16.
Would you be interested in applying new in-situ rock mass characterization methods (geophysical/geomechanical tests) in your mine?Q17.
How fast would new geochemical/geophysical/geomechanical information have to be available to be implemented in the stope designing process?Q18.
Would you consider implementing changes in your stope designing process and under what conditions?Q19.
Please feel free to leave any remarks or recommendations here.

Questionnaire Target
The Mining Intelligence database [125] was drawn from to create a database containing 250 international mines that use stoping methods.The database created included continent, country, commodity, mining method, owner, and contact information (if available).Once the population size for the survey had been estimated (250 mines), the aim was to contact as many of the mines as possible to exceed the target sample size needed to represent the population adequately.Following the principles of survey science, the desired sample size was estimated according to the following formula [126]: where N = population size, e = margin of error, z = z-score (standard deviation), and p = expected proportion.With a confidence level of 90% and a margin of error of 14%, with a population size of 250, this resulted in the estimated sample size desired to exceed 31 responses.

Participant Details
About 1000 potential participants were identified and contacted, and about 500 persons received an invitation to participate in the survey.Additionally, the survey invitation was advertised on the Mining Industry Professionals network forum on the LinkedIn platform.The invitation to the survey was additionally featured in LinkedIn's June 2019 newsletter of the Mining Industry Professionals network with about 300,000 members.To allow for tracing which form of contact delivered the best results, the survey was split into three groups (direct contact via emails, LinkedIn messages, and Mining Industry Professionals network forum on LinkedIn platform).In total, approximately 500 people were contacted, and a total of 36 responses were gathered.Therefore, the targeted sample size of 31 was exceeded by 16% and was considered a good representation of the studied population.
The survey was undertaken between May and July 2019.Due to the creation of different response origin sub-groups, it was possible to analyze where most responses originated, and which form of contact achieved the best results.The origin of the respondents by contact form and region can be seen in Figure 1.Mines from all continents were represented with the most answers received from European mines.However, the participants from Europe represented Nordic mines that are technologically advanced.A broad variety of different areas of expertise of respondents was achieved; the job titles of the respondents covered a broad range including mine managers and mining engineers, rock mechanics engineers, geotechnical engineers, project engineers, and geologists.The participants represented mines of varying sizes and commodities, such as gold, silver, copper, zinc, and lead.The annual ore production amongst survey respondents ranged between 0.2 Mt to 10.7 Mt with an average of 1.8 Mt.
sons received an invitation to participate in the survey.Additionally, the survey invitation was advertised on the Mining Industry Professionals network forum on the LinkedIn platform.The invitation to the survey was additionally featured in LinkedIn's June 2019 newsletter of the Mining Industry Professionals network with about 300,000 members.To allow for tracing which form of contact delivered the best results, the survey was split into three groups (direct contact via emails, LinkedIn messages, and Mining Industry Professionals network forum on LinkedIn platform).In total, approximately 500 people were contacted, and a total of 36 responses were gathered.Therefore, the targeted sample size of 31 was exceeded by 16% and was considered a good representation of the studied population.
The survey was undertaken between May and July 2019.Due to the creation of different response origin sub-groups, it was possible to analyze where most responses originated, and which form of contact achieved the best results.The origin of the respondents by contact form and region can be seen in Figure 1.Mines from all continents were represented with the most answers received from European mines.However, the participants from Europe represented Nordic mines that are technologically advanced.A broad variety of different areas of expertise of respondents was achieved; the job titles of the respondents covered a broad range including mine managers and mining engineers, rock mechanics engineers, geotechnical engineers, project engineers, and geologists.The participants represented mines of varying sizes and commodities, such as gold, silver, copper, zinc, and lead.The annual ore production amongst survey respondents ranged between 0.2 Mt to 10.7 Mt with an average of 1.8 Mt.
The responses from participants contacted via LinkedIn messages showed the highest response quality, while the responses from the Mining Industry Professionals network forum showed the longest average response times.A more detailed survey description is given in [127].

Applied Mining Methods
The distribution of the applied mining methods is illustrated in Figure 2. The survey shows a strong dominance of the sublevel stoping mining method, with approximately 46% of the responses reporting it as their applied mining method, followed by cut and fill with 22% of the responses.
To enable improved insight into the consistency of the applied mining methods, whether the currently applied mining methods had been used since the beginning of the The responses from participants contacted via LinkedIn messages showed the highest response quality, while the responses from the Mining Industry Professionals network forum showed the longest average response times.A more detailed survey description is given in [127].

Applied Mining Methods
The distribution of the applied mining methods is illustrated in Figure 2. The survey shows a strong dominance of the sublevel stoping mining method, with approximately 46% of the responses reporting it as their applied mining method, followed by cut and fill with 22% of the responses.
Energies 2022, 15, x FOR PEER REVIEW 6 of 22 life of the mine or changes had been applied to them was assessed.Figure 3 shows that changes in the mining method were reported in 69% of the responses.The reasons for the changes in the applied mining methods varied between economic, geological, technical, and geotechnical factors.To enable improved insight into the consistency of the applied mining methods, whether the currently applied mining methods had been used since the beginning of the life of the mine or changes had been applied to them was assessed.Figure 3 shows that changes in the mining method were reported in 69% of the responses.The reasons for the changes in the applied mining methods varied between economic, geological, technical, and geotechnical factors.

Geological Conditions
The mining professionals were asked to evaluate the geological conditions in their mines.More than 80% of the respondents described their mine's geological conditions as "challenging" or "very challenging" (Figure 4).

Geological Conditions
The mining professionals were asked to evaluate the geological conditions in their mines.More than 80% of the respondents described their mine's geological conditions as "challenging" or "very challenging" (Figure 4).

Stope Design Methods
Concerning the various methods for stope design, it was important to assess which of these were utilized in the industry.As shown in Figure 5, personal expertise and numerical methods represented a significantly higher proportion than empirical methods.
The empirical methods used were mainly limited to the application of Mathew´s stability graph or the modified Mathew´s stability graph and utilization of the rock mass rating system (Figure 6).Personal expertise was utilized in the form of a broad variety of personnel from different disciplines including mine planners, drill and blast engineers, geologists, specialists in rock mechanics, and mining and geotechnical engineers.Commonly, they utilized their personally acquired experience within specific geological settings.This can be viewed as a personal empirical approach.
The most utilized software in stope design were Micromine, Map3D, Surpac, Maptek Vulcan, Gantt Scheduler, Deswik (Microstation), Auto Stope Designer and Stope Optimizer, Rocscience (RS2, Slide, Phase2, Examine, DIPS, Unwedge and CPillar) and Recursos Mineros.Quantifiable values are not available here since these methods were only assessed in the free-text answers.Of note is that no participants mentioned any of the ITASCA software, such as FLAC, UDEC or 3DEC, that are widely used for rock mechanical numerical simulations.
Figure 7 illustrates in which state of mining the principal stope design was carried out.It is to be noted, that an iterative approach (stope design applied at multiple stages) was executed by only 6 respondents.
The survey respondents were asked how long it took to design stopes.Figure 8 provides an overview of the duration of the stope design process.Based on the survey, stope

Stope Design Methods
Concerning the various methods for stope design, it was important to assess which of these were utilized in the industry.As shown in Figure 5, personal expertise and numerical methods represented a significantly higher proportion than empirical methods.The empirical methods used were mainly limited to the application of Mathew s stability graph or the modified Mathew s stability graph and utilization of the rock mass rating system (Figure 6).Personal expertise was utilized in the form of a broad variety of personnel from different disciplines including mine planners, drill and blast engineers, geologists, specialists in rock mechanics, and mining and geotechnical engineers.Commonly, they utilized their personally acquired experience within specific geological settings.This can be viewed as a personal empirical approach.The most utilized software in stope design were Micromine, Map3D, Surpac, Maptek Vulcan, Gantt Scheduler, Deswik (Microstation), Auto Stope Designer and Stope Optimizer, Rocscience (RS2, Slide, Phase2, Examine, DIPS, Unwedge and CPillar) and Recursos Mineros.Quantifiable values are not available here since these methods were only assessed in the free-text answers.Of note is that no participants mentioned any of the ITASCA software, such as FLAC, UDEC or 3DEC, that are widely used for rock mechanical numerical simulations.
Figure 7 illustrates in which state of mining the principal stope design was carried out.It is to be noted, that an iterative approach (stope design applied at multiple stages) was executed by only 6 respondents.The survey respondents were asked how long it took to design stopes.Figure 8 provides an overview of the duration of the stope design process.Based on the survey, stope design mainly required less than a week; some stopes though were reported to be designed as quickly as in a matter of minutes, whereas other responses stated several weeks.The relationships between stope size and geological complexity and the time needed to perform stope design were frequently mentioned as important influencing factors.An additional aspect of stope design is the number of people involved in the process.The average number reported was three, though numbers as low as a single person and a maximum of 15 people were reported.Several responses indicated that the number of people involved depends on how much of the previous planning work was counted as part of the stope design process.
Finally, respondents were asked to rank parameters influencing stope design (Figure 9).Cutoff grade and dilution were reported as the most important parameters, though no single factor was of outstanding importance.Several additional factors were stated to have a considerable influence on stope design, though these cannot be assessed quantitatively as they were provided as individual free-text answers.These factors were: productivity, ventilation, mining cost, sequencing, drill and blast, flexibility in mine planning, rock mass, labor efficiency, selective mining unit (SMU), and safety.A discussion of these factors is not provided here, but they should be considered important factors sometimes affecting stope design.

Stope Performance
The prevalence of stope stability monitoring is shown in Figure 10.According to the survey, almost all (97%) respondents stated that stability monitoring occurred, most often (77%) using cavity monitoring systems (CMS).The other methods used relied on comparing planned and mucked tons, visual inspections, seismic controls, borehole extensometers, and laser scanning with point cloud data.

Most Important Improvement Suggestions for Stope Design
Expert suggestions on the most important improvements for stope design were assessed.Figure 11 illustrates that the implementation of additional geotechnical data, improvements in software, and integration into general mine planning, were reported as the An additional aspect of stope design is the number of people involved in the process.The average number reported was three, though numbers as low as a single person and a maximum of 15 people were reported.Several responses indicated that the number of people involved depends on how much of the previous planning work was counted as part of the stope design process.
Finally, respondents were asked to rank parameters influencing stope design (Figure 9).Cutoff grade and dilution were reported as the most important parameters, though no single factor was of outstanding importance.Several additional factors were stated to have a considerable influence on stope design, though these cannot be assessed quantitatively as they were provided as individual free-text answers.These factors were: productivity, ventilation, mining cost, sequencing, drill and blast, flexibility in mine planning, rock mass, labor efficiency, selective mining unit (SMU), and safety.A discussion of these factors is not provided here, but they should be considered important factors sometimes affecting stope design.An additional aspect of stope design is the number of people involved in the process.The average number reported was three, though numbers as low as a single person and a maximum of 15 people were reported.Several responses indicated that the number of people involved depends on how much of the previous planning work was counted as part of the stope design process.
Finally, respondents were asked to rank parameters influencing stope design (Figure 9).Cutoff grade and dilution were reported as the most important parameters, though no single factor was of outstanding importance.Several additional factors were stated to have a considerable influence on stope design, though these cannot be assessed quantitatively as they were provided as individual free-text answers.These factors were: productivity, ventilation, mining cost, sequencing, drill and blast, flexibility in mine planning, rock mass, labor efficiency, selective mining unit (SMU), and safety.A discussion of these factors is not provided here, but they should be considered important factors sometimes affecting stope design.

Stope Performance
The prevalence of stope stability monitoring is shown in Figure 10.According to the survey, almost all (97%) respondents stated that stability monitoring occurred, most often (77%) using cavity monitoring systems (CMS).The other methods used relied on comparing planned and mucked tons, visual inspections, seismic controls, borehole extensometers, and laser scanning with point cloud data.

Most Important Improvement Suggestions for Stope Design
Expert suggestions on the most important improvements for stope design were assessed.Figure 11 illustrates that the implementation of additional geotechnical data, improvements in software, and integration into general mine planning, were reported as the

Stope Performance
The prevalence of stope stability monitoring is shown in Figure 10.According to the survey, almost all (97%) respondents stated that stability monitoring occurred, most often (77%) using cavity monitoring systems (CMS).The other methods used relied on comparing planned and mucked tons, visual inspections, seismic controls, borehole extensometers, and laser scanning with point cloud data.
Energies 2022, 15, 240 9 of 20 most urgently desired improvements.There was considerable diversity within the listed methods, which indicates that there are several aspects requiring improvement in the stope design process.Several other areas for improvement were mentioned, such as the implementation of more reliable grade data, a variable cut-off, reliable performance indication for dilution-cable-bolt utilization, and utilization of hanging-wall offset to decrease dilution.

Utilization of Geotechnical Methods
The utilization of rock mass characterization methods was assessed.This served as an indication of how likely potential implementation of specific methods within the industry would be considered.Figure 12 shows that 45% of the respondents have utilized rock mass investigations for stope design.Most of the applied methods dealt with geomechanics, while minor parts involved geophysical and geochemical investigations.The geomechanical methods reported included RQD, RMR, MRMR, life of mine (LOM) stress model, deformation monitoring, core logging, Q-system, equivalent linear overbreak

Most Important Improvement Suggestions for Stope Design
Expert suggestions on the most important improvements for stope design were assessed.Figure 11 illustrates that the implementation of additional geotechnical data, improvements in software, and integration into general mine planning, were reported as the most urgently desired improvements.There was considerable diversity within the listed methods, which indicates that there are several aspects requiring improvement in the stope design process.Several other areas for improvement were mentioned, such as the implementation of more reliable grade data, a variable cut-off, reliable performance indication for dilution-cable-bolt utilization, and utilization of hanging-wall offset to decrease dilution.most urgently desired improvements.There was considerable diversity within the listed methods, which indicates that there are several aspects requiring improvement in the stope design process.Several other areas for improvement were mentioned, such as the implementation of more reliable grade data, a variable cut-off, reliable performance indication for dilution-cable-bolt utilization, and utilization of hanging-wall offset to decrease dilution.

Utilization of Geotechnical Methods
The utilization of rock mass characterization methods was assessed.This served as an indication of how likely potential implementation of specific methods within the industry would be considered.Figure 12 shows that 45% of the respondents have utilized rock mass investigations for stope design.Most of the applied methods dealt with geomechanics, while minor parts involved geophysical and geochemical investigations.The geomechanical methods reported included RQD, RMR, MRMR, life of mine (LOM) stress model, deformation monitoring, core logging, Q-system, equivalent linear overbreak (ELOS)-chart and the Hoek-Brown failure-criterion.The geophysical methods reported

Utilization of Geotechnical Methods
The utilization of rock mass characterization methods was assessed.This served as an indication of how likely potential implementation of specific methods within the industry would be considered.Figure 12 shows that 45% of the respondents have utilized rock mass investigations for stope design.Most of the applied methods dealt with geomechanics, while minor parts involved geophysical and geochemical investigations.The geomechanical methods reported included RQD, RMR, MRMR, life of mine (LOM) stress model, deformation monitoring, core logging, Q-system, equivalent linear overbreak (ELOS)-chart and the Hoek-Brown failure-criterion.The geophysical methods reported included acoustical and optical televiewer and gravimetric testing, and geochemical methods, which were mostly limited to atomic absorption spectroscopy and fire assay.ment novel methods.The geophysical and geochemical methods included in Figure 13b refer to collecting the rock mass data for stope design specifically, such as for location and orientation of discontinuities, or determination of their properties, the ore/waste boundary, or in situ rock strength.It was concluded that there was a marked trend towards the utilization of geophysical and geochemical methods for rock mass characterization in stope design in the future.The planned future implementation of rock mass characterization methods into the stope design process was assessed (Figure 13).The term "new rock mass characterization methods" was meant to address the willingness of the participants of the survey to implement new methods for rock mass characterization that are different from their currently applied methods.The results indicated that half of the participants were willing to implement novel methods.The geophysical and geochemical methods included in Figure 13b refer to collecting the rock mass data for stope design specifically, such as for location and orientation of discontinuities, or determination of their properties, the ore/waste boundary, or in situ rock strength.It was concluded that there was a marked trend towards the utilization of geophysical and geochemical methods for rock mass characterization in stope design in the future.
Energies 2022, 15, x FOR PEER REVIEW 11 of 22 included acoustical and optical televiewer and gravimetric testing, and geochemical methods, which were mostly limited to atomic absorption spectroscopy and fire assay.The planned future implementation of rock mass characterization methods into the stope design process was assessed (Figure 13).The term "new rock mass characterization methods" was meant to address the willingness of the participants of the survey to implement new methods for rock mass characterization that are different from their currently applied methods.The results indicated that half of the participants were willing to implement novel methods.The geophysical and geochemical methods included in Figure 13b refer to collecting the rock mass data for stope design specifically, such as for location and orientation of discontinuities, or determination of their properties, the ore/waste boundary, or in situ rock strength.It was concluded that there was a marked trend towards the utilization of geophysical and geochemical methods for rock mass characterization in stope design in the future.Figure 13.The planned implementation of rock mass and ore characterization methods for stope design (a) and the type of method that is planned to be used to collect rock mass and ore data (b).

Improvement Implementation and Conditions
Next, the participants were asked if they would be interested in applying new methods in stope design.This is different from the previous question (Figure 13a) where the participants were asked if they were already planning to implement new methods.Figure 14 illustrates that the interest in the implementation of novel ore characterization, such as LIBS (laser-induced breakdown spectroscopy) and novel in-situ rock mass characterization methods, ranged between 65% and 70%.This is considerably higher than response rates for the already planned implementation of new methods, as assessed in the previous questions.This should be understood as an indication that by explaining the specific methods and their associated benefits in detail, the potential for implementation for the proposed methods could be increased considerably.
questions.This should be understood as an indication that by explaining the specific methods and their associated benefits in detail, the potential for implementation for the proposed methods could be increased considerably.
The question was set to assess whether there is a general willingness to implement changes in the stope design process, and which conditions and requirements would be associated with these changes.In addition, the acceptable timeframe for possible improvements was identified.The general willingness to implement changes in the stope design process was very high, reaching a value of 87% (Figure 15).However, respondents noted that changes must be proven in feasibility studies and implemented into the software.Other requirements identified were realized benefits, reliability, predictability, faster process, and new personnel.Potential changes to stope design were not only limited to geotechnical methods but other methods, such as geophysical methods, were also desired.The question was set to assess whether there is a general willingness to implement changes in the stope design process, and which conditions and requirements would be associated with these changes.In addition, the acceptable timeframe for possible improvements was identified.The general willingness to implement changes in the stope design process was very high, reaching a value of 87% (Figure 15).However, respondents noted that changes must be proven in feasibility studies and implemented into the software.Other requirements identified were realized benefits, reliability, predictability, faster process, and new personnel.Potential changes to stope design were not only limited to geotechnical methods but other methods, such as geophysical methods, were also desired.The timeframe for the availability of additional geotechnical data to be acquired was identified as a potential key issue for the implementation, utilization, and ease of use of the new technology.Therefore, the tolerable timeframe for obtaining additional measurement data to be used in stope design was assessed (Figure 16).It was found that data from additional measurements available within three days would satisfy the needs of more The timeframe for the availability of additional geotechnical data to be acquired was identified as a potential key issue for the implementation, utilization, and ease of use of the new technology.Therefore, the tolerable timeframe for obtaining additional measurement data to be used in stope design was assessed (Figure 16).It was found that data from additional measurements available within three days would satisfy the needs of more than 70% of the respondents.However, a considerable number of responses stated that the timeframe for obtaining the data could be extended if the data added substantial value to the stope design process.The timeframe for the availability of additional geotechnical data to be acquired was identified as a potential key issue for the implementation, utilization, and ease of use of the new technology.Therefore, the tolerable timeframe for obtaining additional measurement data to be used in stope design was assessed (Figure 16).It was found that data from additional measurements available within three days would satisfy the needs of more than 70% of the respondents.However, a considerable number of responses stated that the timeframe for obtaining the data could be extended if the data added substantial value to the stope design process.

Hypothesis Analysis
To achieve a deeper level of understanding of survey responses, the results can be analyzed using either correlational or hypothesis analysis [128].In this study, correlational analysis was not implemented due the limited availability of quantifiable values.Therefore, a hypothesis analysis was performed as it is of value in surveys with a large number of free-text answers and where there is elaborated complexity in the questionnaire [129].Two hypotheses related to the surveyed subject were defined and evaluated by assessing the responses from the survey.The first hypothesis tested assumed that the more challenging the geological conditions, the more rock mass and ore investigation methods would be applied for stope design.To test this hypothesis, it was necessary to

Hypothesis Analysis
To achieve a deeper level of understanding of survey responses, the results can be analyzed using either correlational or hypothesis analysis [128].In this study, correlational analysis was not implemented due the limited availability of quantifiable values.Therefore, a hypothesis analysis was performed as it is of value in surveys with a large number of free-text answers and where there is elaborated complexity in the questionnaire [129].Two hypotheses related to the surveyed subject were defined and evaluated by assessing the responses from the survey.The first hypothesis tested assumed that the more challenging the geological conditions, the more rock mass and ore investigation methods would be applied for stope design.To test this hypothesis, it was necessary to analyze the correlation between the geological conditions and the current and planned implementation of rock mass and ore investigation methods.The results of this analysis are represented in Figure 17.There was a clear trend towards an increase in the utilization of rock mass and ore investigation methods with increasingly challenging geological conditions.It can be concluded that the first hypothesis holds true and was confirmed.analyze the correlation between the geological conditions and the current and planned implementation of rock mass and ore investigation methods.The results of this analysis are represented in Figure 17.There was a clear trend towards an increase in the utilization of rock mass and ore investigation methods with increasingly challenging geological conditions.It can be concluded that the first hypothesis holds true and was confirmed.The second hypothesis that was tested assumed that the more challenging geological conditions result in lower utilization of empirical methods for stope design.For analysis, the geological conditions were compared to the preferred stope design method type.The results are presented in Figure 18.The only clear trend that could be extracted from ana-0% 50% 100%  The second hypothesis that was tested assumed that the more challenging geological conditions result in lower utilization of empirical methods for stope design.For analysis, the geological conditions were compared to the preferred stope design method type.The results are presented in Figure 18.The only clear trend that could be extracted from analyzing the comparison between these two factors was that under varying and unchallenging geological conditions the utilization of personal expertise in stope design showed an increase.However, these geological conditions were reported by only a very limited proportion of the survey population and, therefore, this hypothesis was rejected.The second hypothesis that was tested assumed that the more challenging geological conditions result in lower utilization of empirical methods for stope design.For analysis, the geological conditions were compared to the preferred stope design method type.The results are presented in Figure 18.The only clear trend that could be extracted from analyzing the comparison between these two factors was that under varying and unchallenging geological conditions the utilization of personal expertise in stope design showed an increase.However, these geological conditions were reported by only a very limited proportion of the survey population and, therefore, this hypothesis was rejected.

State of Stope Design Methods in the Industry
The general survey responses provided detailed insights on stope design methods.It must be noted that block-caving methods were not considered in this study, as the approaches to stability and design in block caving are beyond the scope of stope design methods.The sublevel stoping methods were dominant, and the geological conditions were mostly challenging according to responses.The results demonstrated that a great interest in improvements for stope design exists.Many respondents highlighted the need to gather more geotechnical data as early as possible to allow for more optimal stope design.This would also limit the negative impact resulting from the limited-data issue of stope design and mine planning integration.Other challenges indicated by the participants were the need for development and improvement of software packages for stope design, integration into general mine planning and automation of the design process to increase its speed and reduce human bias (Figure 11).This indicates that machine and deep learning approaches for stope design (e.g., methods proposed by [93,94]) have the highest potential to be implemented in stope design practice in mines due to their speed, automation, and data utilization capabilities.The substantial interest in improvements in stope design processes should be considered by researchers.The timescale to effect stope design based on measurements within a timeframe of three days should be considered as guidance for future development.

Survey Sample Size and Response Rate
The targeted sample size of 31 was estimated using Equation (1).Based on previous experience of collecting survey data in the mining industry [90,130,131], it was anticipated that the response would be minimal.Therefore, a large number of invitations were sent to exceed the targeted sample size.The 36 responses received exceeded the estimated sample size by 16%, and therefore the results are considered representative of the studied sector comprising a population of 250 mines.
From the meta-data of the survey responses, the most successful approach for conducting the survey, which also resulted in the best answer quality, was a direct message through LinkedIn.The responses from the Mining Industry Professionals network forum were limited, possibly because it was not possible to directly address the possible reader.The best answers were received mainly from unknown individuals, who most likely believed in the value of related research and this survey; thus, they filled in the questionnaire out of goodwill rather than feeling obliged to do so.Support for this assumption is provided by the increased quality of answers within this group.In general, it was interesting to observe how much time people were willing to spend on responding to the survey and which kind of high-quality answers, allowing for valuable insights, were received.Some excellent answers were created in only 20 min, though on average an excellent answer required 39 min to be created.It was fortunate that so many industry professionals from different fields of stope design replied.The high degree of internationality and different positions within the management hierarchy were adequately represented.

Questionaire Response Quality Assessment
During the results evaluation process, the responses were initially assessed for their information value.Due to an identified high degree of variety in the answer complexity and quality, a response quality evaluation was performed.To minimize the subjective impact on the evaluation process, response quality/complexity indicators (average free-text-answerlength, or total response time) were chosen.These indicators were then used to create a response quality classification system.As part of the system s implementation, every response was assigned to a single class and the responses were divided into four categories, as can be seen in Table 2. Table 3 shows the results from the response quality evaluation grouped by the means of contact.The overall response quality of all participants was satisfactory on average and the share of excellent responses was 28%.The responses from participants contacted via LinkedIn showed the highest average response quality and the highest share of excellent responses, while the forum responses showed the lowest quality.

Conclusions
This study analyzed the current state-of-the-art of stope design methods through an industry survey distributed among mining professionals.The survey sought to assess the level of implementation of novel methods for stope design into stope design practice and to characterize current challenges.In total, 36 responses of satisfactory response quality were received, and an international audience was reached, adequately representing the diverse environment of the stope-based underground mines.Based on the survey results, stope design is considered an important focal point for improvements.The most important challenges are to increase the amount of geotechnical data for stope planning, to improve stope design software, and to integrate stope design into general mine planning to a larger extent.Other challenges identified in the survey are the development of software packages for stope design, integration into general mine planning, and automation of the design process to increase its speed.Therefore, machine and deep learning methods for the stope design process have strong potential for implementation in the industry as they can: (1) utilize a large amount of geotechnical data, (2) be automated and implemented into a software package, and (3) increase speed and reliability.
The survey showed a strong dominance of the sublevel stoping mining method currently being applied as the primary mining method in underground stoping-based mines.The results suggest that there is no dominant stope design method and in some mines stope design is still commonly based on personal expertise or empirical methods, even though this may yield very sub-optimal results.The reliability of personal expertise or empirical methods as stand-alone approaches is not considered to be sufficient.On the other hand, a variety of numerical methods for stope design have been developed and many different methods and algorithms exist and were applied in 37% of the mines.
The stope design process occurs either within mine planning or during the production stage.The period of stope design varies from less than a week to more than a month.The utilization of an iterative approach for stope design at multiple mining phases is not yet well established.
The important factors in stope design are cut-off grades and ore dilution.Most professionals consider the geological conditions of the mine they represent to be challenging or very challenging.Those who reported very challenging conditions suggest improved methods to gather more geotechnical data.The newly acquired data is desired to be compatible with the mine planning software.A time frame of three days to make new geotechnical data available for stope design is sufficient for about 70% of the specialists.Changes in the mining method were reported in 69% of the responses due to various reasons and the industry is ready to implement new and more efficient stope design methods.The feasibility of the new method should be proven by case studies.New design processes do not have to substitute old design methods but can rather be seen as potentially complementary actions.
The strong interest in improvements in stope design processes suggests great potential for incorporation of recent scientific findings into the current practice of stope design.However, the benefits must first be showcased by case studies.

Figure 2 .
Figure 2. Applied mining methods at mines participating in the survey.

Figure 2 .
Figure 2. Applied mining methods at mines participating in the survey.

Figure 2 .
Figure 2. Applied mining methods at mines participating in the survey.

Figure 3 .
Figure 3. Ratio of the mines where changes were made to the applied mining methods, which indicates the process is adjusted to the conditions.

Figure 3 .
Figure 3. Ratio of the mines where changes were made to the applied mining methods, which indicates the process is adjusted to the conditions.

Figure 4 .
Figure 4. Geological conditions present at mines represented in the survey.

Figure 4 .
Figure 4. Geological conditions present at mines represented in the survey.

22 Figure 5 .
Figure5.Ratio of the method that is used for design stopes in the mines.

Figure 5 .
Figure5.Ratio of the method that is used for design stopes in the mines.

Figure 5 .
Figure 5. Ratio of the method that is used for design stopes in the mines.

Figure 6 .
Figure 6.Ratio of the applied empirical method for designing stopes: Mathew's stability graph, Mathews modified stability graph, and other (e.g., rock mass rating, RMR).

Figure 7 .
Figure 7. Mining stage at which the stope design process takes place.The iterative stage means the stope design takes place in multiple stages with a feedback loop.

Figure 6 .
Figure 6.Ratio of the applied empirical method for designing stopes: Mathew's stability graph, Mathews modified stability graph, and other (e.g., rock mass rating, RMR).

22 Figure 5 .
Figure 5. Ratio of the method that is used for design stopes in the mines.

Figure 6 .
Figure 6.Ratio of the applied empirical method for designing stopes: Mathew's stability graph, Mathews modified stability graph, and other (e.g., rock mass rating, RMR).

Figure 7 .
Figure 7. Mining stage at which the stope design process takes place.The iterative stage means the stope design takes place in multiple stages with a feedback loop.

Figure 7 .
Figure 7. Mining stage at which the stope design process takes place.The iterative stage means the stope design takes place in multiple stages with a feedback loop.

Figure 10 .
Figure 10.The application of stope performance and control after excavation takes place (a) including the cavity monitoring system (CMS) to measure the stopes (b).

Figure 11 .
Figure 11.Most important improvement suggestions for stope design.

Figure 10 .
Figure 10.The application of stope performance and control after excavation takes place (a) including the cavity monitoring system (CMS) to measure the stopes (b).

Figure 10 .
Figure 10.The application of stope performance and control after excavation takes place (a) including the cavity monitoring system (CMS) to measure the stopes (b).

Figure 11 .
Figure 11.Most important improvement suggestions for stope design.

Figure 11 .
Figure 11.Most important improvement suggestions for stope design.

Figure 12 .
Figure 12.The current utilization of rock mass and ore investigation methods in the mines (a) and the type of method that is used to collect rock mass and ore data (b).

Figure 13 .Figure 12 .
Figure13.The planned implementation of rock mass and ore characterization methods for stope design (a) and the type of method that is planned to be used to collect rock mass and ore data (b).

Figure 12 .
Figure 12.The current utilization of rock mass and ore investigation methods in the mines (a) and the type of method that is used to collect rock mass and ore data (b).

Figure 13 .
Figure13.The planned implementation of rock mass and ore characterization methods for stope design (a) and the type of method that is planned to be used to collect rock mass and ore data (b).

Figure 14 .Figure 14 .
Figure 14.Interest in new rock and ore classification methods (a) and rock mass characterization methods (b).

Figure 15 .
Figure 15.Willingness to implement changes in the stope design process (a) and the acceptance conditions to implement new methods (b) (other requirements identified were realized benefits, reliability, predictability, faster process, and new personnel).

Figure 15 .
Figure 15.Willingness to implement changes in the stope design process (a) and the acceptance conditions to implement new methods (b) (other requirements identified were realized benefits, reliability, predictability, faster process, and new personnel).

Figure 15 .
Figure 15.Willingness to implement changes in the stope design process (a) and the acceptance conditions to implement new methods (b) (other requirements identified were realized benefits, reliability, predictability, faster process, and new personnel).

Figure 16 .
Figure 16.Number of responses on the tolerable time for measurement data to become available for stope design.

Figure 16 .
Figure 16.Number of responses on the tolerable time for measurement data to become available for stope design.

Figure 17 .
Figure 17.Correlation of the current (left) and planned (right) utilization of rock mass and ore characterization methods with geological conditions.

Figure 17 .
Figure 17.Correlation of the current (left) and planned (right) utilization of rock mass and ore characterization methods with geological conditions.

Figure 17 .
Figure 17.Correlation of the current (left) and planned (right) utilization of rock mass and ore characterization methods with geological conditions.

Figure 18 .
Figure 18.Correlation of geological conditions and the applied stope design method.

Table 2 .
Response quality classification system.Very complex answers, stretching over several phrases, respondent shows effort to explain his point of view and allows for thorough understanding

Table 3 .
Overview of response quality results grouped by means of contact.