Next Article in Journal
A Critical Assessment of the Resource Depletion Potential of Current and Future Lithium-Ion Batteries
Previous Article in Journal
Feasibility of Small Wind Turbines in Ontario: Integrating Power Curves with Wind Trends
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Taking the Step towards a More Dynamic View on Raw Material Criticality: An Indicator Based Analysis for Germany and Japan

by
Simon Glöser-Chahoud
1,*,†,
Luis Tercero Espinoza
1,
Rainer Walz
1 and
Martin Faulstich
2
1
Fraunhofer Institute for Systems and Innovation Research ISI, 76139 Karlsruhe, Germany
2
CUTEC Institute, Technical University of Clausthal, 38678 Clausthal-Zellerfeld, Germany
*
Author to whom correspondence should be addressed.
Current address: Breslauer Strasse 48, 76139 Karlsruhe, Germany.
Resources 2016, 5(4), 45; https://doi.org/10.3390/resources5040045
Submission received: 30 September 2016 / Revised: 18 November 2016 / Accepted: 28 November 2016 / Published: 8 December 2016

Abstract

:
Due to mounting concerns about the security of raw material supplies, numerous studies dealing with the quantification of supply risks and material criticality at the national level have been carried out in previous years. Regarding these studies, most approaches are indicator based static screening methods analyzing large numbers of raw materials and identifying those which are most critical for an economy. The majority of these screening methods quantify supply risks and vulnerabilities for one base year without taking into account temporal changes. Dynamic approaches for specific raw materials analyzing affected value chains in detail have been introduced recently; however, these studies do not intend to provide a screening of larger numbers of commodities. In this paper, we present a simple dynamic screening approach to assess raw material criticality at the country level building upon methods from innovation economics. The indicators applied in this study are only based on broadly available production and trade data, which makes this approach relatively easy to apply. We test our methodology on the example of Germany and Japan—two economies with highly specialized industries and low domestic raw material deposits, and, hence, high import dependency. The results are comparable to those of previously conducted multi indicator based static screening methods. However, they provide additional insight into temporal developments over the previous decade.

1. Introduction

Against the backdrop of increasing demand for most metals and mounting concerns about the security of raw material supply, analyses of supply risks and criticality of different metallic and mineral raw materials have gained attention in recent years [1,2,3]. This development is forced by both the rapid growth of emerging economies associated with high raw material consumption [4] and the accelerating innovation cycles of new technologies for which specific hardly substitutable materials often are a prerequisite [5]. Consequently, the systematic evaluation of raw material criticality has been the subject of numerous studies in recent years (cf. Table 1). These studies either use an indicator based risk ranking (cf. Figure 1a), the concept of a criticality matrix as introduced by the U.S. National Research Council (NRC) in 2008 (cf. Figure 1b), or scenario and time series analyses in order to evaluate past and potential future market developments for specific raw materials (cf. Figure 1c).
The majority of current screening methods for assessing raw material criticality (Figure 1a,b) are static approaches covering the supply situation of different materials for one base year in order to identify those which are most critical for an economy. Some studies take indicators into account that refer to different scales in time such as expected future demand or historic market development [8]. However, projecting indicators from different points in time to the present might dilute the results for the current supply situation. This is because one raw material might be designated critical due to expected future demand which is linked to high uncertainty, while another raw material is designated critical due to the current supply situation, which is a fact at present. In this context, it is important to clearly distinguish between the aforementioned screening methods that generally determine numerous different materials and technology oriented studies of the supply system of specific commodities. When considering specific raw material markets and supply systems, dynamic criticality approaches including econometric time series analysis [4,9,10], agent based [11] or system dynamics modeling [12] have been presented in previous years (see also bottom part of Table 1); however, most current screening methods analyzing large numbers of materials and commodities do not take temporal changes into account (see also classification in Table 1). The European Commission has updated its 2010 criticality study [13] in 2014 [14], and there are publications about criticality assessments for two different years using the same set of indicators [15]. However, in order to identify fluctuations and volatility among different indicators, extended periods of time need to be analyzed. Hence, regarding screening methods of critical raw materials, there is a need to better take into account temporal variations and volatility of indicator values. The US National Science and Technology Council (NSTC) has recently introduced a dynamic methodology for a first screening of potentially critical raw materials taking into account the three dimensions of supply risk (indicated by the concentration of production at the country level), market growth and market dynamics (indicated by the annual price volatility). The methodology discussed in the following sections may serve as a complement to currently ongoing efforts aiming at assessing the development of raw material criticality over a period of time.
Abbreviations in Table 1 are as follows: BERR: Department for Business Enterprise & Regulatory Reform; BMWI: German Ministry for Economic Affairs; BGR: German Geological Survey; CdTe: Cadmium telluride; CIGS: Copper indium gallium (di)selenide; DERA: German Mineral Resources Agency; JRC: Joint Research Council; KfW: German government-owned development bank; UNEP: United Nations Environment Programme.
In this paper, we present a simple indicator based method to assess raw material dependency and supply risks at the national level over a period of time using trade statistics and production data. This method is demonstrated and evaluated on the example of Germany and Japan—two established economies with low domestic mineral deposits, a high share of the manufacturing sector regarding total value added and specialized industries depending on imported high-tech metals. A key feature of this method is the good data availability and its simple applicability, making it suitable for a transfer to other countries and regions. This approach is not intended to replace static multi indicator based criticality assessments (Figure 1a,b), as it is highly simplified and only focuses on production and trade data that show significant volatility over time. However, this method provides additional insights into the historic development of vulnerabilities, supply risks and import dependency. Such historic developments might also be of interest when analyzing trends in demand in forward-looking studies. Nonetheless, historic change is no indicator for future development, and future demand for specific high-tech metals is mainly driven by emerging technologies that have to be analyzed in technology oriented studies (see Figure 3 for an example of technology-specific analyses).
As indicated by the following figures, particularly the markets of high-tech metals which are often characterized by comparatively low production volumes show high dynamics both on the supply and the demand side. The most important indicator to measure supply risks in recent studies was the concentration of production at the country level [8]. This is usually quantified by the Herfindahl–Hirschman Index (HHI), which is a standard approach to measure the relative concentration (see Equation (4)). Figure 2 displays the development of the concentration of production for different selected metals. Particularly for technology metals with comparatively low production levels (Figure 2 bottom), this indicator has shown high fluctuations. This is to some extent obvious, as due to the lower number of mining activities, an additional production site more strongly affects the overall concentration measure.
Regarding future development, several of these technology metals are especially expected to face a severe increase in demand due to their utilization for different emerging technologies [50,51]. Figure 3 combines the results of a recently published study about raw material demand for future technologies [51] with the evaluation of supply risks from the latest EU criticality assessment [14]. As shown in Figure 3, some of the most critical materials for the EU (particularly several heavy (HREE) and light (LREE) Rare Earth Elements for magnet production) are expected to face increasing demand in the coming years due to their technological importance.
Figure 3. Comparison of the supply risk value in the latest EU criticality study [14] and the expected increase in demand due to emerging technologies according to a current study commissioned by the German Mineral Resources Agency (DERA) [51]. The illustrated demand in 2013 and expected future demand in 2035 only refer to the emerging technologies that were analyzed in this study. The grey line represents the current production level.
Figure 3. Comparison of the supply risk value in the latest EU criticality study [14] and the expected increase in demand due to emerging technologies according to a current study commissioned by the German Mineral Resources Agency (DERA) [51]. The illustrated demand in 2013 and expected future demand in 2035 only refer to the emerging technologies that were analyzed in this study. The grey line represents the current production level.
Resources 05 00045 g003

2. Methodology

When regarding raw material criticality in the context of classical risk assessment within a criticality matrix, the two dimensions of economic importance (or vulnerability) and supply risk, which represent the likelihood of supply disruptions [3] need to be quantified by a set of indicators (cf. Figure 1b). The evaluation of the economic importance of different raw materials is especially challenging. This assessment often includes qualitative measures such as the vulnerability of affected value chains [25], complex sectoral economic analyses (based on sectors that might be affected by supply disruptions [13,14]), or qualitative assessments of the raw material substitutability in different industries (see Helbig et al. [56] for an overview of different approaches of the evaluation of economic vulnerabilities in recent criticality studies). While concentration measures which are most commonly used for the assessment of supply risks [8] can be quantified over a period of time using time series of production an trade data [57], this is not possible (or would be accompanied with high efforts) with the common multi indicator based approaches for the evaluation of the economic importance. Therefore, in the following section, we develop several indicators to assess the economic importance of a raw material on the basis of broadly available production and trade data using established methods from the field of innovation economics [58]. This approach enables a relatively simple analysis and comparison of raw material criticality at the national level over a historical time span.
In order to define the level of specialization of an economy and the competitive advantage of different industries, there are several well established indicators in economic theory that compare the performance of the economy in focus to global average numbers based on trade statistics (see, for example, [58,59,60]). The two most important indicators, which are not only applied to trade data but might also be used for the evaluation of patent databases [61], are explained in Table 2.
For the following assessment of the relative economic importance of different raw materials, the RXA when applied to commodity imports is of particular relevance. We transfer this basic principle of comparing the domestic imports and uses of different commodities to the overall global consumption while taking into account the size of the economy in focus to the raw material supply system. This allows us the development of several indicators which are readily accessible and which already provide a quantitative evaluation of the relative importance of different raw materials for an economy. Hence, this approach is based on the assumption that if the industrial or economic structure was homogenous among different countries, their material consumption would be in accordance with their economic performance, whereas the industrial specialization of a country leads to a concentration in raw material use. A concentration in use of a specific raw material indicates a higher economic importance of that commodity for the economy being analyzed. The different indicators assessing this relative economic importance of a specific raw material are described below.
From the share of the domestic consumption of a specific raw material regarding global consumption and the percentage that the country holds in the world economy (GDP share), a simple indicator can be developed that evaluates if the intensity of use of a specific raw material is disproportionately high. This indicates the level of specialization of different industries within the economy and the accompanied need for specific technology metals for which the global average use is significantly lower. Therefore, the global share in consumption of a specific raw material taking into account the economic performance of the country can be applied as a first indicator for the relative economic importance of a raw material. This is specified in Equation (1):
Indicator 1 = Share of global consumption country i regarding metal j Share in global economy country i = Im M j C i Ex M j C i + Dp M j C i Global production M j GDP C i Global GDP
where C i is the country being analyzed and M j is the specific metal, further abbreviations in the equations are as follows:
I m Imports , E x Exports , D p Domestic production .
A simplification of Equation (1) which no longer includes production data but focuses on pure trade data analysis is shown in Equation (2). This indicator puts the share of world trade regarding a specific metal in relation to the country’s economic performance. Particularly when the domestic raw material production is low (such as for the examples of Germany and Japan analyzed herein), the results of this indicator are quite similar to those of Equation (1):
Indicator 2 = World trade share country i regarding metal j Share in global economy country i = Im M j C i Global trade M j = i Im M j C i GDP C i Global GDP
Furthermore, an indicator that analyzes the import share of a specific metal regarding the total import of all metals seems reasonable in order to get a feeling for the relative importance of specific raw materials. By weighting this value with the global share of the metal in focus in relation to the global trade of all metals, an indicator is developed that directly relates to the RXA in Table 2. This indicator is described in Equation (3):
Indicator 3 = Import share of metal j regarding all metal imports to country i Global share of metal j regarding total trade of all metals = Im M j C i j Im M j C i i Im M j C i i , j Im M j C i
When analyzing raw material criticality as an abstraction of classical risk assessment within a criticality matrix (see Figure 1b and Figure 4), besides the vulnerability (or economic importance), an assessment of the supply risk is necessary. While, thus far, we have analyzed the relative economic importance of different raw materials at the national level based on the aforementioned indicators, the most common indicator for the assessment of supply risk in previous studies was the concentration of production evaluated by the HHI (as described in Equation (4), see Achzet and Helbig (2013) [8] for a detailed assessment of indicator based supply risk measures):
HHI = i = 1 n x i j = 1 n x j 2
Furthermore, particularly in countries with high import dependence, the concentration of raw material imports can be taken as an indicator for the supply risk. These concentration measures are classical risk indicators that already were relevant when analyzing a region’s dependence on strategic raw materials and geopolitical risks during the Cold War [62,63]. Habib et al. [57] and the US National Science and Technology Council [64] used the concentration of production at the country level in order to assess the development of geopolitical risks over time. In current multi indicator based static criticality studies, further subindicators such as the static depletion time of reserves, the byproduct share, the recyclability, the substitutability, the environmental performance, etc. are taken into account when analyzing the supply risk (a summary of indicators applied to quantify supply risks in current criticality studies is provided in Table S2 in the Supplementary Materials to this paper). However, the concentration of production at the country level still is the most important indicator for measuring supply risks, and import dependency plays an important role in current criticality studies [65]. As most other indicators usually do not show high volatility over time [15], it seems reasonable to take these concentration measures as a starting point to evaluate temporal changes of supply risks.
In order to get an understanding of the temporal development of raw material criticality at the country level, the indicators described above referring to the economic importance and the supply risk, respectively, are equally weighted and added up. This seems reasonable as there is no indication to give stronger weight to one specific indicator, and the assessment provided by this simple approach intends to emphasize the volatility among key variables rather than to provide a detailed assessment of raw material criticality as performed by static multi indicator based approaches (see the first two sections of Table 1). A summary of the indicator aggregation utilized to assess the two dimensions of the criticality matrix, which are the “Relative Economic Importance (REI)” and the “Supply Risk (SR)”, is provided in Equations (5) and (6):
REI = Indicator 1 + Indicator 2 + Indicator 3 3
SR = HHI production country level + HHI imports 2
Both of the indicators for quantifying the economic importance of a raw material and the supply risk are based on pure production and trade data. Hence, their development over time using historical data can be simply assessed. This procedure enables an enhancement of the criticality matrix by the dimension of time as shown in Figure 4.
Generally, the analyses of supply risks and raw material criticality are of interest for countries and regions with significant raw material demand across different industries and low domestic raw material production, and, hence, high import dependence. As Germany and Japan show the highest value added shares of the manufacturing sector among the G7 counties (see Figure 5) and both countries have highly specialized industries and low domestic raw material deposits, an evaluation of the defined indicators for these two economies seems suitable. Nonetheless, the approach described above could be transferred to any other country, even though this method particularly makes sense for countries and regions with high dependence on raw material imports.

3. Results

For the following indicator based assessments, we used production data from the USGS [55] and the UN Comtrade database for the analysis of trade flows (see Table S1 in the Supplementary Materials to this paper for background data and commodity codes). While all raw materials that have been indentified as potentially critical in previous studies were analyzed, Figure 6 displays the results of the indicator evaluation regarding the relative economic importance (see Equations (1)–(3)) for those materials with the highest values (significantly above 1, which indicates a concentration of use within the economies in focus).
The interpretation of these indicators is as follows, taking tungsten for Germany as an example: while Germany holds 5% of the world economy (German share of global GDP), the German industries are responsible for around 20% of the global tungsten consumption and trade flows (indicators 1 and 2 have values of around 4), and the share of tungsten imports regarding total metal imports to Germany is significantly higher than the global average (indicator 3). Hence, without knowing the exact industrial value chain of tungsten processing, it is obvious that this raw material plays a significant role for the German economy.
As shown in Figure 6, for Japan, the Rare Earth Elements (REE) as well as the Platinum Group Metals (PGM) such as platinum and palladium show the highest indicator values for the relative economic importance. This seems reasonable as Japan is definitely the largest consumer of rare earths outside China due to its strong magnet industry (NdFeB magnets) and its specialized chemical industry (cerium and lanthanum for catalysts and speciality chemicals or metal alloys) and has a strong automobile industry for which considerable amounts of platinum and palladium for autocatalysts are needed. Furthermore, lithium, which is an important raw material for battery production, as well as the semiconductor germanium have a significant concentration in the Japanese industry. Tungsten, which is mainly used in high performance alloys (tungsten carbides) for gas turbines, drilling, trimming and cutting machinery, shows the highest indicator value for Germany while the Rare Earth Elements play a minor role. Platinum Group Metals (PGM) or magnesium for lightweight construction components play a significant role for the German automotive industry. Additional high-tech metals that show a concentration of use in Germany are bismuth and germanium as well as titanium, which is an important component in speciality steels or which is used for different high performance titanium alloys.
Hence, without having to analyze the value chains and their sectoral distribution and by only taking into account the volume of primary raw material consumption, this simple indicator based approach offers valuable insight into the economic importance of specific raw materials for different countries. As a matter of fact, this approach is highly simplified and does not replace the detailed analysis of sectoral distribution of raw materials taking into account their substitutability in different end-use applications (as performed, for example, in the EU criticality studies). However, when comparing the results of raw material criticality assessments for different countries over a period of time, a simple indicator based approach seems very useful in order to guarantee the comparability of results.
In order to assess the development of raw material criticality over time, the results for the “Relative Economic Importance (REI)” from Equation (5) and the “Supply Risk (SR)” from Equation (6) were scaled to values from 1 to 10 (see Table S5 in the supplemenatry data for the normalized values). In a first step, we performed this procedure for the German economy taking into account those materials with high values regarding their economic importance (see raw materials in Figure 6). As illustrated in Figure 7, the temporal development may be displayed within three dimensions or within the common two-dimensional criticality matrix, distinguishing between different points in time by the size of the dots.
Figure 7 shows a general high volatility of the calculated criticality values. This is mainly due to variations in trade flows, which already becomes evident from the analysis of the relative economic importance in Figure 6. The criticality of Rare Earth Elements (REE) in Figure 7 seems to be slightly declining, which is due to increasing mining activities outside of China and declining import volumes in previous years. However, as indicated by Figure 2, the concentration of production at the country level (HHI) is generally related to fluctuations, especially for raw materials with comparatively low production volume. Nonetheless, as illustrated in Figure 7, several of the analyzed high-tech metals are showing a clear trend towards an increasing relative economic importance (such as palladium or further PGMs) or increasing supply risks (such as bismuth or magnesium). Such developments can only be detected with dynamic approaches taking into account temporal variations of indicator values.
Those commodities with high values regarding their relative economic importance (Figure 6) or their criticality level (Figure 7), respectively, have been indentified as potentially critical in previous static analyses (see [34,35] for Japan and [25] for Germany); hence, the results from this work are nothing completely new. Nevertheless, they are a proof of concept that this simple approach already provides profound information about a country’s dependence on imported raw materials, vulnerabilities, or raw material criticality over a period of time.

4. Conclusions

In this paper, we introduced a simple methodology of how a first approximation to raw material criticality (based on the economic importance and the supply risk which form the two dimensions of the criticality matrix) could be analyzed over a period of time using broadly available production and trade data. It is needless to say that this dynamic approach is a strong simplification of static multi indicator methods performed in previous criticality studies (see Table 1). It only focuses on the quantitative relevance of primary raw materials and does not take into account aspects of environmental risk, substitutability or recyclability of different materials. However, the approach introduced in this paper forms a well accessible manner to get a feeling of which raw materials might be critical for an economy and how this raw material dependency has changed in previous years without having to go into any detail regarding industrial use patterns, value chains or even sectoral economic analyses. Hence, such an analysis will not replace any static multi indicator approach but might provide additional insight into commodity dependency. Furthermore, the approach applied for the evaluation of the relative economic importance or the supply risk, respectively, is not intended to be a fixed set of indicators and equations. The supply risk measure could, for example, be enhanced by the World Governance Indicators (WGI, published by the World Bank) as performed by the European Commission [13,14]. Besides the supply risk, the price volatility as an indicator for market dynamics could be included as suggested by the US National Science and Technology Council [64]. Similar enhancements and modifications could be applied to the evaluation of the economic importance.
In order to evaluate political instruments or measures for how to reduce raw material dependency by substitution, higher efficiency of use or more efficient recycling, and raw material specific models are needed that cover the entire material life cycle including primary and secondary production [66,67]. Hence, indicator based screening methods, no matter if they are static or dynamic, will always be used to identify those materials among a large number of different elements for which more detailed analyses are needed [64]. In this context, the methodology presented here might contribute to previously performed static analyses and to ongoing efforts aiming at taking into account temporal variations in the assessment of raw material criticality.

Supplementary Materials

The following are available online at https://www.mdpi.com/2079-9276/5/4/45/s1, Table S1: HS codes (Harmonized System) relevant for the trade data analysis using the UN Comtrade database, Table S2: Indicators applied to assess raw material supply risk in different recent static criticality assessments, Table S3: Calculated values of Indicators 1-3 for Germany, Table S4: Calculated values of Indicators 1-3 for Japan, Table S5: Normalized values of the “Relative Economic Importance” and the “Supply Risk” for Germany, Figure S1: Results of the dynamic criticality assessment for Germany displayed as a variation of Figure 7.

Acknowledgments

The work leading to this publication was supported by the AERTOs Community (Associated European Research and Technology Organisations), which is gratefully acknowledged. Moreover, the authors would like to particularly thank Stefan Haag for his support in collecting relevant trade and production data.

Author Contributions

S.G. designed the research, performed the analysis, and wrote the manuscript; L.T.E. supported the graphics creation, manuscript layout and writing process; R.W. and M.F. supervised the research, gave suggestions and discussed the results.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Erdmann, L.; Graedel, T.E. Criticality of Non-Fuel Minerals: A Review of Major Approaches and Analyses. Environ. Sci. Technol. 2011, 45, 7620–7630. [Google Scholar] [CrossRef] [PubMed]
  2. Speirs, J.; Houri, Y.; Gross, R. Materials Availability: Comparison of Material Criticality Studies: Methodologies and Results; Technical Report; UK Energy Research Centre: London, UK, 2013. [Google Scholar]
  3. Glöser, S.; Tercero Espinoza, L.; Gandenberger, C.; Faulstich, M. Raw material criticality in the context of classical risk assessment. Resour. Policy 2015, 44, 35–46. [Google Scholar] [CrossRef]
  4. Rosenau-Tornow, D.; Buchholz, P.; Riemann, A.; Wagner, M. Assessing the long-term supply risks for mineral raw materials—A combined evaluation of past and future trends. Resour. Policy 2009, 34, 161–175. [Google Scholar] [CrossRef]
  5. Tercero, L. The Role of Emerging Technologies in Rapidly Changing Demand for Mineral Raw Materials; Polinares Working Paper; European Commission: Brussels, Belgium, 2012. [Google Scholar]
  6. Morley, N.; Eatherley, D. Material Security—Ensuring Resource Availability for the UK Economy: UK Environment Agency and Department for Business Enterprise; Technical Report; Oakdene Hollins Ltd.: Aylesbury, UK, 2008. [Google Scholar]
  7. National Research Council (NRC). Minerals, Critical Minerals, and the U.S. Economy: Committee on Critical Mineral Impacts of the U.S. Economy; Technical Report; Committee on Earth Resources, National Research Council: Washington, DC, USA, 2008. [Google Scholar]
  8. Achzet, B.; Helbig, C. How to evaluate raw material supply risks—An overview. Resour. Policy 2013, 38, 435–447. [Google Scholar] [CrossRef]
  9. Scholz, R.W.; Wellmer, F.W. Approaching a dynamic view on the availability of mineral resources: What we may learn from the case of phosphorus? Glob. Environ. Chang. 2013, 23, 11–27. [Google Scholar] [CrossRef]
  10. Habib, K.; Wenzel, H. Reviewing resource criticality assessment from a dynamic and technology specific perspective—Using the case of direct-drive wind turbines. J. Clean. Prod. 2016, 112, 3852–3863. [Google Scholar] [CrossRef]
  11. Knoeri, C.; Wäger, P.A.; Stamp, A.; Althaus, H.J.; Weil, M. Towards a dynamic assessment of raw materials criticality: Linking agent-based demand—With material flow supply modelling approaches. Sci. Total Environ. 2013, 461–462, 808–812. [Google Scholar] [CrossRef] [PubMed]
  12. Glöser, S.; Faulstich, M. Quantitative Analysis of the Criticality of Mineral and Metallic Raw Materials Based on a System Dynamics Approach. In Proceedings of the 30th International Conference of the System Dynamics Society, St. Gallen, Switzerland, 22–26 July 2012.
  13. European Commission. Critical Raw Materials for the EU; Technical Report; European Commission (Enterprise and Industry): Brussels, Belgium, 2010. [Google Scholar]
  14. European Commission. Report on Critical Raw Materials for the EU; Technical Report; European Commission (Enterprise and Industry): Brussels, Belgium, 2014. [Google Scholar]
  15. Ciacci, L.; Nuss, P.; Reck, B.; Werner, T.; Graedel, T. Metal Criticality Determination for Australia, the US, and the Planet—Comparing 2008 and 2012 Results. Resources 2016, 5, 29. [Google Scholar] [CrossRef]
  16. Gandenberger, C.; Glöser, S.; Marscheider-Weidemann, F.; Ostertag, K.; Walz, R. Die Versorgung der Deutschen Wirtschaft mit Roh- und Werkstoffen für Hochtechnologien: Präzisierung und Weiterentwicklung der Deutschen Rohstoffstrategie: Innovationsreprot; Technical Report; Büro für Technikfolgen-Abschätzung beim Deutschen Bundestag (TAB): Berlin, Germany, 2012. [Google Scholar]
  17. Buijs, B.; Sievers, H.; Tercero Espinoza, L.A. Limits to the critical raw materials approach. Proc. ICE Waste Resour. Manag. 2012, 165, 201–208. [Google Scholar] [CrossRef]
  18. Olivetti, E.; Field, F.; Kirchain, R. Understanding dynamic availability risk of critical materials: The role and evolution of market analysis and modeling. MRS Energy Sustain. 2015, 2. [Google Scholar] [CrossRef]
  19. Mancini, L.; Sala, S.; Recchioni, M.; Benini, L.; Goralczyk, M.; Pennington, D. Potential of life cycle assessment for supporting the management of critical raw materials. Int. J. Life Cycle Assess. 2015, 20, 100–116. [Google Scholar] [CrossRef] [Green Version]
  20. Dewulf, J.; Blengini, G.A.; Pennington, D.; Nuss, P.; Nassar, N.T. Criticality on the international scene: Quo vadis? Resour. Policy 2016, 50, 169–176. [Google Scholar] [CrossRef] [Green Version]
  21. Graedel, T.E.; Reck, B.K. Six Years of Criticality Assessments: What Have We Learned So Far? J. Ind. Ecol. 2016, 20, 692–699. [Google Scholar] [CrossRef]
  22. Jin, Y.; Kim, J.; Guillaume, B. Review of critical material studies. Resour. Conserv. Recycl. 2016, 113, 77–87. [Google Scholar] [CrossRef]
  23. U.S. Department of Energy. Critical Materials Strategy: 2010; Technical Report; U.S. Department of Energy: Washington, DC, USA, 2010.
  24. U.S. Department of Energy. Critical Materials Strategy: 2011; Technical Report; U.S. Department of Energy: Washington, DC, USA, 2011.
  25. Erdmann, L.; Behrendt, S.; Feil, M. Kritische Rohstoffe für Deuschland: Identifikation aus Sicht Deutscher Unternehmen Wirtschaftlich Bedeutsamer Mineralischer Rohstoffe, Deren Versorgungslage Sich Mittel- bis Langfristig als Kritisch Erweisen könnte; Technical Report; IZT/Adelphi im Auftrag der KfW: Berlin, Germany, 2011. [Google Scholar]
  26. Duclos, S.J.; Otto, J.P.; Konitzer, D.G. Design in an Era of Constrained Resources. Mech. Eng. 2008, 132, 36–40. [Google Scholar]
  27. Nassar, N.T.; Barr, R.; Browning, M.; Diao, Z.; Friedlander, E.; Harper, E.M.; Henly, C.; Kavlak, G.; Kwatra, S.; Jun, C.; et al. Methodology of Metal Criticality Determination: Criticality of the Geological Copper Family. Environ. Sci. Technol. 2012, 46, 1071–1078. [Google Scholar] [CrossRef] [PubMed]
  28. Graedel, T.E.; Nassar, N.T. The criticality of metals: A perspective for geologists. Geol. Soc. Lond. Spec. Publ. 2013. [Google Scholar] [CrossRef]
  29. Nuss, P.; Harper, E.M.; Nassar, N.T.; Reck, B.K.; Graedel, T.E. Criticality of Iron and Its Principal Alloying Elements. Environ. Sci. Technol. 2014, 48, 4171–4177. [Google Scholar] [CrossRef] [PubMed]
  30. Harper, E.; Kavlal, G.; Burmeister, M.; Erbis, S.; Espinoza, V.; Nuss, P.; Graedel, T. Criticality of the geological zinc, tin, and lead family. J. Ind. Ecol. 2015, 19, 628–699. [Google Scholar] [CrossRef]
  31. Nassar, N.T.; Graedel, T.E.; Harper, E.M. By-product metals are technologically essential but have problematic supply. Sci. Adv. 2015, 1, e1400180. [Google Scholar] [CrossRef] [PubMed]
  32. Harper, E.M.; Diao, Z.; Panousi, S.; Nuss, P.; Eckelman, M.J.; Graedel, T.E. The criticality of four nuclear energy metals. Resour. Conserv. Recycl. 2015, 95, 193–201. [Google Scholar] [CrossRef]
  33. Panousi, S.; Harper, E.M.; Nuss, P.; Eckelman, M.J.; Hakimian, A.; Graedel, T.E. Criticality of Seven Specialty Metals. J. Ind. Ecol. 2015. [Google Scholar] [CrossRef]
  34. Hatayama, H.; Tahara, K. Criticality assessment of metals for Japan’s resource strategy. Mater. Trans. 2015, 56, 229–235. [Google Scholar] [CrossRef]
  35. Hatayama, H.; Tahara, K. Evaluating the sufficiency of Japan’s mineral resource entitlements for supply risk mitigation. Resour. Policy 2015, 44, 72–80. [Google Scholar] [CrossRef]
  36. Kalvig, P.; Clausen, R.J.; Fold, N.; Hanghøj, K. To what extent is Denmark vulnerable to mineral supply shortage? Geol. Surv. Den. Greenl. Bull. 2014, 2014, 95–98. [Google Scholar]
  37. Verband der Bayrischen Wirtschaft e.V. (vbw). Rohstoffsituation Bayern: Keine Zukunft ohne Rohstoffe: Strategien und Handlungsoptionen; Technical Report; Verband der Bayrischen Wirtschaft e.V. (vbw): München, Germany, 2009. [Google Scholar]
  38. Verband der Bayrischen Wirtschaft e.V. (vbw). Rohstoffversrogung Langfristig Sichern: Verband der Bayrischen Wirtschaft e.V.; Technical Report; Verband der Bayrischen Wirtschaft e.V. (vbw): München, Germany, 2012. [Google Scholar]
  39. Moss, R.L.; Tzimas, E.; Kara, H.; Willis, P.; Kooroshy, J. Critical Metals in Strategic Energy Technologies: Assessing Rare Metals as Supply Chain Bottlenecks in Low-Carbon Energy Technologies; Technical Report; JRC Scientific and Technical Reports; European Commission: Brussels, Belgium, 2011. [Google Scholar]
  40. Moss, R.L.; Tzimas, E.; Willis, P.; Arendorf, J.; Tercero Espinoza, L. Critical Metals in the Path towards the Decarbonisation of the EU Energy Sector: Assessing Rare Metals as Supply Chain Bottlenecks in Low-Carbon Energy Technologies; Technical Report; JRC Scientific and Technical Reports; European Commission: Brussels, Belgium, 2013. [Google Scholar]
  41. Knašytė, M.; Kliopova, I.; Staniškis, J.K. Economic Importance, Environmental and Supply Risks on Imported Resources in Lithuanian Industry. Environ. Res. Eng. Manag. 2012, 60. [Google Scholar] [CrossRef]
  42. AEA Technology. Raw Materials Critical to the Scottish Economy; Technical Report; Scottland and Nothern Ireland Forum for Environmental Research: Edinborough, Schottland, 2011. [Google Scholar]
  43. British Geological Survey (BGS). Risk List of the British Geological Survey; Technical Report; British Geological Survey: London, UK, 2012.
  44. Buchholz, P. Angebotskonzentration bei mineralischen Rohstoffen und Zwischenprodukten—potenzielle Preis- und Lieferrisiken: DERA-Rohstoffliste 2012. In DERA Rohstoffinformationen; Deutsche Rohstoffagentur (DERA): Hannover, Germany, 2014; Volume 24. [Google Scholar]
  45. Helbig, C.; Bradshaw, A.M.; Kolotzek, C.; Thorenz, A.; Tuma, A. Supply risks associated with CdTe and CIGS thin-film photovoltaics. Appl. Energy 2016, 178, 422–433. [Google Scholar] [CrossRef]
  46. Patiño, D.; Alberto, E. Metallic Mineral Resources in the Twenty-First Century. I. Historical Extraction Trends and Expected Demand. Natl. Resour. Res. 2015, 25, 71–90. [Google Scholar] [CrossRef]
  47. Mayer, H.; Gleich, B. Measuring Criticality of Raw Materials: An Empirical Approach Assessing the Supply Risk Dimension of Commodity Criticality. Natl. Resour. 2015, 6, 56–78. [Google Scholar] [CrossRef]
  48. Habib, K.; Wenzel, H. Exploring rare earths supply constraints for the emerging clean energy technologies and the role of recycling. J. Clean. Prod. 2014, 84, 348–359. [Google Scholar] [CrossRef]
  49. Roelich, K.; Dawson, D.A.; Purnell, P.; Knoeri, C.; Revell, R.; Busch, J.; Steinberger, J.K. Assessing the dynamic material criticality of infrastructure transitions: A case of low carbon electricity. Appl. Energy 2014, 123, 378–386. [Google Scholar] [CrossRef]
  50. Angerer, G.; Erdmann, L.; Marscheider-Weideman, F.; Scharp, M.; Luellmann, A.; Handke, V.; Marwede, M. Rohstoffe für Zukunftstechnologien: Einfluss des Branchenspezifischen Rohstoffbedarfs in Rohstoffintensiven Zukunftstechnologien auf Die Zukünftige Rohstoffnachfrage; Technical Report; Fraunhofer ISI and IZT, Fraunhofer Verlag Stuttgart: Berlin, Germany, 2009. [Google Scholar]
  51. Marscheider-Weidemann, F.; Langkau, S.; Hummen, T.; Erdmann, L.; Tercero Espinoza, L.A.; Angerer, G.; Marwede, M.; Benecke, S. Rohstoffe für Zukunftstechnologien 2016: Auftragsstudie. In DERA Rohstoffinformationen; Deutsche Rohstoffagentur (DERA): Hannover, Germany, 2016; Volume 28. [Google Scholar]
  52. Buchholz, P.; Liedtke, M.; Gernuks, M. Evaluating Supply Risk Patterns and Supply and Demand Trends for Mineral Raw Materials: Assessment of the Zinc Market. In Non-Renewable Resource Issues, International Year of Planet Earth; Sinding-Larsen, R., Wellmer, F.W., Eds.; Springer: Dordrecht, The Netherlands; London, UK, 2010; pp. 157–181. [Google Scholar]
  53. Buchert, M.; Schüler, D.; Bleher, D.; Neurohr, N.; Hagelüken, L. Critical Metals for Future Sustainable Technologies and Their Recycling Potential: Sustainable Innovation and Technology Transfer Industrial Sector Studies; Technical Report; Öko Institut e.V.: Freiburg, Germany, 2009. [Google Scholar]
  54. Frondel, M.; Grösche, P.; Huchtemann, D.; Oberheitmann, A.; Peters, J.; Angerer, G.; Sartorius, C.; Buchholz, P.; Röhling, S.; Wagner, M. Trends der Angebots- und Nachfragesituation bei Mineralischen Rohstoffen; Technical Report; RWI, BGR, Fraunhofer ISI: Berlin, Germany, 2006. [Google Scholar]
  55. US Geological Survey (USGS). Mineral Yearbooks; Technical Report; US Geological Survey: Reston, VA, USA, 2016.
  56. Helbig, C.; Wietschel, L.; Thorenz, A.; Tuma, A. How to evaluate raw material vulnerability—An overview. Resour. Policy 2016, 48, 13–24. [Google Scholar] [CrossRef]
  57. Habib, K.; Hamelin, L.; Wenzel, H. A dynamic perspective of the geopolitical supply risk of metals. J. Clean. Prod. 2016, 133, 850–858. [Google Scholar] [CrossRef]
  58. Grupp, H. Messung und Erklärung des Technischen Wandels: Grundzüge einer Empirischen Innovationsökonomik; Springer: Berlin/Heidelberg, Germany, 1997. [Google Scholar]
  59. Bowen, H.P. On the theoretical interpretation of indices of trade intensity and revealed comparative advantage. Weltwirtschaftliches Arch. 1983, 119, 464–472. [Google Scholar] [CrossRef]
  60. Walz, R.; Marscheider-Weidemann, F. Technology-specific absorptive capacities for green technologies in Newly Industrialising Countries. Int. J. Technol. Glob. 2011, 5, 212–229. [Google Scholar] [CrossRef]
  61. Ostertag, K.; Sartorius, C.; Tercero Espinoza, L. Innovationsdynamik in rohstoffintensiven Produktionsprozessen. Chem. Ing. Tech. 2010, 82, 1893–1901. [Google Scholar] [CrossRef]
  62. Haglund, D.G. The new geopolitics of minerals. Political Geogr. Q. 1986, 5, 221–240. [Google Scholar] [CrossRef]
  63. Anderson, E.W.; Anderson, L.D. Strategic Minerals: Resource Geopolitics and Global Geo-Economics; Wiley: Chichester, West Sussex, UK; New York, NY, USA, 1998. [Google Scholar]
  64. National Science and Technology Council (NSTC). Assessment of Critical Minerals: Screening, Methodology and Initial Application; Technical Report; Subcommittee on Critical and Strategic Mineral Supply Chains of the Committee on Environment, Natural Resources, and Sustainability: Washington, DC, USA, 2016.
  65. Gemechu, E.D.; Helbig, C.; Sonnemann, G.; Thorenz, A.; Tuma, A. Import-based Indicator for the Geopolitical Supply Risk of Raw Materials in Life Cycle Sustainability Assessments. J. Ind. Ecol. 2016, 20, 154–165. [Google Scholar] [CrossRef]
  66. Müller, E.; Hilty, L.M.; Widmer, R.; Schluep, M.; Faulstich, M. Modeling Metal Stocks and Flows: A Review of Dynamic Material Flow Analysis Methods. Environ. Sci. Technol. 2014, 48, 2102–2113. [Google Scholar] [CrossRef] [PubMed]
  67. Glöser-Chahoud, S.; Pfaff, M.; Soulier, M. Modeling material flows, cumulative material demand and market dynamics of industrial metals within a system dynamics framework: An overview of concepts and exemplary models. In Proceedings of the 34th International Conference of the System Dynamics Society, Delft, The Netherlands, 17–21 July 2016; pp. 1–20.
Figure 1. Different approaches for quantifying and communicating raw material criticality or supply and market risks. While hierarchical risk ranking methods via indicator aggregation (a) and criticality matrix approaches as an abstraction of risk analysis in a risk matrix (b) are screening methods analyzing and comparing numerous commodities, detailed time series and scenario analyses (c) form individual approaches performed separately for each commodity and market. (a) hierarchical ’risk ranking’ based on different indicators, e.g., [6]; (b) criticality matrix over the axes of “vulnerability” and “supply risk” [7]; and (c) time series and scenario analysis, here on the example of copper, e.g., [4].
Figure 1. Different approaches for quantifying and communicating raw material criticality or supply and market risks. While hierarchical risk ranking methods via indicator aggregation (a) and criticality matrix approaches as an abstraction of risk analysis in a risk matrix (b) are screening methods analyzing and comparing numerous commodities, detailed time series and scenario analyses (c) form individual approaches performed separately for each commodity and market. (a) hierarchical ’risk ranking’ based on different indicators, e.g., [6]; (b) criticality matrix over the axes of “vulnerability” and “supply risk” [7]; and (c) time series and scenario analysis, here on the example of copper, e.g., [4].
Resources 05 00045 g001
Figure 2. Herfindal–Hirschman Index (HHI) of raw material production at the country level for different selected metals (cf. [17]). Particularly high-tech metals with comparatively low production rates show high volatility regarding the concentration of production, which is the most important indicator for supply risks. Data for this analysis are based on US Geological Survey (USGS) [55].
Figure 2. Herfindal–Hirschman Index (HHI) of raw material production at the country level for different selected metals (cf. [17]). Particularly high-tech metals with comparatively low production rates show high volatility regarding the concentration of production, which is the most important indicator for supply risks. Data for this analysis are based on US Geological Survey (USGS) [55].
Resources 05 00045 g002
Figure 4. Enhancing the criticality matrix by the dimension of time.
Figure 4. Enhancing the criticality matrix by the dimension of time.
Resources 05 00045 g004
Figure 5. Share of the manufacturing sector regarding total GDP for the G7 countries (based on World Bank data, “industry value added” (% of GDP)). Germany and Japan both have a relatively high industry share regarding total value added, and, hence, a higher economic vulnerability concerning disruptions of raw material supply.
Figure 5. Share of the manufacturing sector regarding total GDP for the G7 countries (based on World Bank data, “industry value added” (% of GDP)). Germany and Japan both have a relatively high industry share regarding total value added, and, hence, a higher economic vulnerability concerning disruptions of raw material supply.
Resources 05 00045 g005
Figure 6. Results of the indicators described above for Germany and Japan. All metals that have been identified as potentially critical in previous studies were analyzed, while only those metals for which an indicator value of significantly >1 was observed are shown in this Figure. The calculated indicator values are available in Tables S3 and S4 in the Supplementary Materials to this paper. (a) indicator results (Equations (1)–(3)) for Germany; and (b) indicator results (Equations (1)–(3)) for Japan.
Figure 6. Results of the indicators described above for Germany and Japan. All metals that have been identified as potentially critical in previous studies were analyzed, while only those metals for which an indicator value of significantly >1 was observed are shown in this Figure. The calculated indicator values are available in Tables S3 and S4 in the Supplementary Materials to this paper. (a) indicator results (Equations (1)–(3)) for Germany; and (b) indicator results (Equations (1)–(3)) for Japan.
Resources 05 00045 g006
Figure 7. Results of a dynamic criticality assessment based on Equations (5) and (6) for Germany. A variation of this illustration is available in Figure S1 in the Supplementary Materials to this paper. (a) criticality matrix in three dimensions assessing the values of supply risk and relative economic importance over time.; and (b) criticality matrix with temporal movements in two dimensions (the larger the dots, the more present the value).
Figure 7. Results of a dynamic criticality assessment based on Equations (5) and (6) for Germany. A variation of this illustration is available in Figure S1 in the Supplementary Materials to this paper. (a) criticality matrix in three dimensions assessing the values of supply risk and relative economic importance over time.; and (b) criticality matrix with temporal movements in two dimensions (the larger the dots, the more present the value).
Resources 05 00045 g007
Table 1. Key studies and publications from the previous ten years quantifying supply risks and raw material criticality (see also numerous review papers about this topic [1,2,3,8,16,17,18,19,20,21,22]). Particularly the list of raw material specific dynamic analyses (bottom part of the table) does not claim to be complete.
Table 1. Key studies and publications from the previous ten years quantifying supply risks and raw material criticality (see also numerous review papers about this topic [1,2,3,8,16,17,18,19,20,21,22]). Particularly the list of raw material specific dynamic analyses (bottom part of the table) does not claim to be complete.
MethodTitle (Year)Commissioner and Reference
Concept of the criticality matrix“Critical raw materials for the EU” (2010, 2014)European Commission [13,14]
“Critical Materials Strategy” (2010, 2011)U.S. Department of Energy [23,24]
“Critcal raw materials for Germany” (2011)KfW bank group [25]
“Minerals, Critical Minerals, And the U.S. Economy” (2008)National Research Council [7]
“Design in an Era of Constrained Resources” (2008)General Electric [26]
“Criticality space” (Criticality matrix enhanced by the third dimension of “Environmental Impact”)
“Criticality of the Geological Copper Family / Iron and its Principal Alloying Elements / Geological Zinc, Tin and Lead Family / Criticality of the Rare Earth Elements / The criticality of four nuclear energy metals” (2012–2015)Graedel et al., Yale University [27,28,29,30,31,32,33]
Single risk index based on multi indicator aggregation“Criticality Assessment of Metals for Japan’s Resource Strategy” (2015)National Institute of Advanced Industrial Science and Technology, Japan [34]
“Evaluating the sufficiency of Japan’s mineral resource entitlements for supply risk mitigation” (2015)National Institute of Advanced Industrial Science and Technology, Japan [35]
“To what extent is Denmark vulnerable to mineral supply shortage?” (2014)Geological Survey of Denmark and Greenland (GEUS) [36]
“Raw material supply in Bavaria” (2009, 2012)Bavarian industry association (vbw) [37,38]
“Ensuring resource availability for the UK economy” (2008)UK Environment Agency and BERR [6]
“Critical Metals in Strategic Energy Technologies” (2011, 2013)JRC European Commission [39,40]
“Economic Importance, Environmental and Supply Risks on Imported Resources in Lithuanian Industry” (2012)Kaunas University of Technology [41]
“Raw materials critical to the Scottish economy” (2011)Scottish Environment Protection Agency [42]
“Risk list of the British Geological Survey (BGS)” (2012)British Geological Survey (BGS) [43]
“Raw materials list for Germany” (2012)German Geological Survey (DERA) [44]
Raw material or technology specific dynamic assessments based on scenario or econometric analyses“Supply risks associated with CdTe and CIGS thin-film photovoltaics” (2016)Augsburg University [45]
“Metallic Mineral Resources in the Twenty-First Century. Historical Extraction Trends and Expected Demand” (2015)University of Georgia [46]
“Measuring criticality of raw materials: an empirical approach assessing the supply risk dimension of commodity criticality” (2015)Augsburg University [47]
“Exploring rare earths supply constraints for the emerging clean energy technologies and the role of recycling” (2014)
“Reviewing resource criticality assessment from a dynamic and technology specific perspective —using the case of direct-drive wind turbines” (2015)
University of Southern Denmark [10,48]
“Assessing the dynamic material criticality of infrastructure transitions: A case of low carbon electricity” (2014)University of Leeds [49]
“Raw Materials for Future Technologies” (2009, 2016)BMWI, BGR, DERA [50,51]
“Assessing the long-term supply risks for mineral raw materials” (2009)
“Evaluating supply risk patterns and supply and demand trends” (2012)
Volkswagen AG/BGR [4,52]
“Critical Metals for Future Sustainable Technologies and their Recycling Potential” (2009)UNEP [53]
“Trends in supply and demand of mineral raw materials” (2008)BMWI, BGR [54]
Table 2. Indicators from the field of innovation economics to assess the competitive advantage of an industry regarding different technologies or commodity groups [58,59,60]. Similar indicators are used to evaluate patent databases [58,61].
Table 2. Indicators from the field of innovation economics to assess the competitive advantage of an industry regarding different technologies or commodity groups [58,59,60]. Similar indicators are used to evaluate patent databases [58,61].
RXA for country C i and technology T j :
RXA = Ex T j C i j Ex T j C i i Ex T j C i i , j Ex T j C i
The RXA (Relative Export Activity) is an indicator that evaluates the degree of concentration of exports of a country regarding specific technologies or commodities, respectively. This indicator quantifies the share of exports of specific commodities regarding total exports of a country and compares this value to the share of the commodities in focus on total global trade:
RXA = export share of technology T j regarding total exports of country C i export share of technology T j regarding total global trade
An RXA-value > 1 indicates that a country has a higher export share of specific commodities as compared to the global average, and, hence, a higher degree of specialization in that field. Comparable calculations can be done with import data.
RCA for country C i and technology T j :
RCA = Ex T j C i Im T j C i j Ex T j C i j Im T j C i
The RCA (Revealed Competitive Advantage) additionally takes into account the export–import relations when analyzing the level of specialization in trade. Hence, this indicator further takes into account how well the domestic industry stands its ground when confronting the competition from international imports. The RCA is calculated from the country’s export/import ratio regarding a specific technology and the export/import ratio of all commodities in that country:
RCA = export / import ratio of technology T j in country C i export / import ratio of total trade ( all commodities ) in country C i
An RCA-value > 1 indicates a competitive advantage of a country regarding the production of the commodities being analyzed.

Share and Cite

MDPI and ACS Style

Glöser-Chahoud, S.; Tercero Espinoza, L.; Walz, R.; Faulstich, M. Taking the Step towards a More Dynamic View on Raw Material Criticality: An Indicator Based Analysis for Germany and Japan. Resources 2016, 5, 45. https://doi.org/10.3390/resources5040045

AMA Style

Glöser-Chahoud S, Tercero Espinoza L, Walz R, Faulstich M. Taking the Step towards a More Dynamic View on Raw Material Criticality: An Indicator Based Analysis for Germany and Japan. Resources. 2016; 5(4):45. https://doi.org/10.3390/resources5040045

Chicago/Turabian Style

Glöser-Chahoud, Simon, Luis Tercero Espinoza, Rainer Walz, and Martin Faulstich. 2016. "Taking the Step towards a More Dynamic View on Raw Material Criticality: An Indicator Based Analysis for Germany and Japan" Resources 5, no. 4: 45. https://doi.org/10.3390/resources5040045

APA Style

Glöser-Chahoud, S., Tercero Espinoza, L., Walz, R., & Faulstich, M. (2016). Taking the Step towards a More Dynamic View on Raw Material Criticality: An Indicator Based Analysis for Germany and Japan. Resources, 5(4), 45. https://doi.org/10.3390/resources5040045

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop