Next Article in Journal
Sustainable Supply Chain Based on News Articles and Sustainability Reports: Text Mining with Leximancer and DICTION
Previous Article in Journal
Integrating Ecosystem Services and Eco-Security to Assess Sustainable Development in Liuqiu Island
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Does Implementation of Big Data Analytics Improve Firms’ Market Value? Investors’ Reaction in Stock Market

1
School of Business, Ewha Womans University, 52 Ewhayeodae-gil, Seodaemun-gu, Seoul 03760, Korea
2
Sogang Business School, Sogang University, 35 Baekbum-ro, Mapo-gu, Seoul 04107, Korea
*
Author to whom correspondence should be addressed.
Sustainability 2017, 9(6), 978; https://doi.org/10.3390/su9060978
Submission received: 9 April 2017 / Revised: 1 June 2017 / Accepted: 3 June 2017 / Published: 9 June 2017
(This article belongs to the Section Economic and Business Aspects of Sustainability)

Abstract

:
Recently, due to the development of social media, multimedia, and the Internet of Things (IoT), various types of data have increased. As the existing data analytics tools cannot cover this huge volume of data, big data analytics becomes one of the emerging technologies for business today. Considering that big data analytics is an up-to-date term, in the present study, we investigated the impact of implementing big data analytics in the short-term perspective. We used an event study methodology to investigate the changes in stock price caused by announcements on big data analytics solution investment. A total of 54 investment announcements of firms publicly traded in NASDAQ and NYSE from 2010 to 2015 were collected. Our results empirically demonstrate that announcement of firms’ investment on big data solution leads to positive stock market reactions. In addition, we also found that investments on small vendors’ solution with industry-oriented functions tend to result in higher abnormal returns than those on big vendors’ solution with general functions. Finally, our results also suggest that stock market investors highly evaluate big data analytics investments of big firms as compared to those of small firms.

1. Introduction

Recently, due to the advent of social media, multimedia, and the Internet of Things, the total volume of data has tremendously increased [1]. Data have extended their territory over all parts of industries [2] and are considered a key factor of establishing productive processes, alongside capital and labor. Furthermore, as unstructured data have increased, the existing data analytics tools cannot effectively cover those unstructured data. With these environmental changes, big data analytics becomes one of the emerging technologies for business today [3]. Firms are considering implementation of big data analytics solutions, since more sizable and detailed data have become available [4]. Due to data analytics solutions, business organizations are capable of having a deep insight [5] so that they could enhance their performance with better managerial decision making. Furthermore, manufacturers can increase forecasting accuracy with big data analytics, minimizing their inventory cost [6]. Big data analytics and related service markets growth is projected to be at the compound annual growth rate (CAGR) of 23.1% from 2014 through 2019, with annual spending up to $48.6 billion in 2019 [7]. However, there are a number of companies hesitating to implement big data analytics solutions, because the utility of big data analytics implementation remains ambiguous. According to TCS’s survey, a quarter of the respondents said it was difficult for them to anticipate whether the implementation of big data analysis would be effective [8].
Based on previous studies demonstrating the positive impact of information systems implementations [9,10,11] on firms’ financial performance [12,13,14], the present study tries to uncover the economic benefits of big data analytics solutions by using stock market reactions. Specifically, we measure the value of big data analytics solution investment based on the following assumption: when a firm has an event which could affect its market value, stock price of firms will show abnormal returns because all information affects the market value of firms [15,16]. Our study empirically estimates a market value of a firm after announcing an investment in big data analytics solutions. The following research questions are addressed:
  • Do announcements about big data technology implementation increase firms’ market value?
  • What business condition can influence the size of abnormal returns on stock market prices?
To answer these research questions, we gathered the announcements of firms relating big data technology whose stocks are publicly traded in NASDAQ and NYSE. A total of 54 announcements from 2010 to 2015 were collected. Based on our analysis results, we found a substantial support to the prediction that firms’ investment on big data solution lead to positive stock market reactions.

2. Literature Review

“Big data” is a term referring to both structured and unstructured data with massive volume. The most widely accepted definition of big data was suggested by Gartner which defines the term “Big data” as follows: “Big data is high-volume, high-velocity and/or high-variety information assets that demand cost-effective, innovative forms of information processing that enable enhanced insight, decision making, and process automation” [17]. In addition, a bunch of “high velocity information” is another characteristic of big data [17]. When organizations manage big data, they must focus on volume, variety, and velocity of information (sometimes, complexity is considered to be another important feature of big data) [18,19]. Specifically, the volume of data is a critical issue for business, because a huge amount of data is difficult to store and analyze [18,19]. Big data have been far beyond the ability of the exiting database system to store, manage and analyze meaning that the concept of data has been moving towards to an entirely different level of dimensions [20]. Big data analytics can be understood as the advent of new technologies that enable capturing, analyzing, and discovering hidden but crucial information neglected before [7]. Chen and co-authors also defined big data analytics as “analytical techniques in applications that are so large (from terabytes to exabytes) and complex (from sensor to social media data) that they require advanced and unique data storage, management, analysis, and visualization technologies” [21] (p. 1166). Organizational leaders want to implement big data analytics, expecting to achieve competitive differentiation and effective use of data they possess [22].
As big data analytics is a one of the emerging information systems today, by implementing big data analytics, organizations expect to achieve excellence and more productiveness in their business [19]. Therefore, in the present study, we have examined previous literature on an implementation of information systems and its effectiveness (also see Table 1) [23,24,25]. Hayes et al. (2000) and Ranganathan and Samarah (2001) [26,27] estimated how stock market price is fluctuated with announcements of enterprise resource planning implementation. Furthermore, Hendricks et al. [14] examined stock market reactions to enterprise resource planning systems implementations, supply chain management implementations, and customer relationship management implementations, respectively. This study strongly highlighted that, when early adopters employed the supply chain management systems, profitability was escalated significantly. Compared with supply chain management system investment announcements which had as a consequence significant raises in stock returns and profitability, implement of customer relationship management systems did not report a significant impact on either stock price or profitability. In addition, Hunton et al. [12] used four measures of firms’ performance: return on asset, return on sales, asset turnover, and return on investment. They investigated the vertical impact of enterprise resource planning systems adoption on performance of each firms and found that return on asset, asset turnover and return on investment were by far outperformed for adopters, as compared to non-adopters [12]. Furthermore, Hayes et al. (2001) found significant positive effects in enterprise resource planning systems outsourcing announcements, by measuring stock market value [26]. The purpose of this research is to investigate economic benefits of a firm’s big data technologies implementation based on a firm’s stock market value. Since there is no sufficient evidence showing the actual financial value of big data analytics systems implementations, this study can suggest a significant empirical contribution to the body of knowledge on information systems adoption and its economic value.

3. Theoretical Background and Research Propositions

According to the efficient-market hypothesis, a stock price reflects all events and information [15]. For example, if a firm invests in big data analytics solutions, the value of investment is reflected in firm’s stock market price. The methodology is often used to measure the value of an event in a stock market. The event methodology is also used to measure an impact of an event on a firm’s market value [29], e.g., information system related events [16]. There are several studies examining a market value of information system investments [28,29,30,31].
Based on the efficient market hypothesis, this study deploys the event methodology to measure abnormal returns on the stock price taking firms’ big data analytics investments into account. Investment in big data analytics will cause in positive abnormal returns on the stock price, because investors positively evaluate the adoption of big data analytics bringing benefits to firms. By adopting big data analytics, firms can analyze various forms of data, including unstructured data [32]. Big data analytics systems are based on clustered computers with parallel processing power [33], so that a huge amount of data can quickly be processed. Companies adopting big data analysis solutions can expect improved operational efficiency through smart decision makings based on deeper insights into and better understanding of their business environment, as well as into their customers. In addition, they can experience cost reductions with regard to operation and inventory management after adopting the solutions [32]. In the present study, we assume that market investors expect performance improvement by big data adoptions. Positive reactions from investors towards a big data adoption lead to positive abnormal returns on the stock price of firms. Therefore, we posit the following proposition:
Proposition 1 (P1).
An announcement of a big data analytics solution investment results in positive abnormal returns of firms.
We categorized the firms into two distinct groups: tech firms and non-tech firms. Hardware, software, and electronic device manufacturers, information and communications technology (ICT) service firms, information technology consulting firms, and e-commerce firms [34] were categorized as tech firms. These firms need a high information technology (IT) capability to carry their business activities. Firm’s IT capability is an ability to allocate IT-based resources [35] composed of three capabilities: human IT resources, physical IT infrastructure, and intangible IT-enabled resources (p. 178, [35]). Intangible IT enabled resources include resources such as corporate culture, know-how, and environmental orientations. According to [35], knowledge assets are one of the intangible IT-enabled resources including skills and experiences of employees, processes, policies, and information repositories. Compared to non-tech firms, many tech firms are supposed to have high intangible IT-enabled resources [35], so that their employees have comparative skills in and experiences with regard to IT-technology. High intangible IT-enabled resources help employees’ knowledge and information to be shared across functional units [35]. Firms with high information capability achieve success in information technology adoption and use [36]. Therefore, firms with high IT-enabled resources could easily accept and share knowledge on big data analytics. According to an IBM survey, one of the biggest obstacles in adopting big data analytics solutions is the lack of understanding on the big data analytics technology [32]. When firms implement big data analytics solutions, tech firms easily adopt analytic skills and insight through active knowledge sharing. With these reasons in mind, tech firms have a better approach to using big data analytics solutions. Therefore, we argue that market investors evaluate abnormal returns higher for tech firms’ than non-tech firms’ when investing in big data solution adoptions. Hence, we posit the following preposition:
Proposition 2 (P2).
The amount of abnormal return is greater in tech firm than in non-tech firms.
Previous literature indicates that adopting information systems results in a better performance for firms in the manufacturing industry, as compared to those in the service industry. When firms adopt enterprise resource planning systems, firms in manufacturing industry show higher abnormal returns in stock market [28,30,31]. As an extension to previous study results, this study also expects that implementing big data analytics solutions can enhance firms’ performance in the manufacturing industry. By adopting big data analytics, performance checks for machinery and equipment can be automatically done with the self-learning knowledge base [37]. Demands forecasting accuracy can be improved. In addition, production cost and inventory management cost can also be saved through big data analytics implementation for manufacturing processes [32]. By contrast, when service firms adopt big data analytics, they mainly focus on enhancing customer-oriented services [32]. According to previous studies, customer relationship management systems adoptions for service firms do not significantly change either the market price of firms or their performance [38,39]. Although, in order to increase customer satisfaction, firms in the service industry make investments in big data analytic solutions, investors may still have doubt that investments on big data analytics solutions will not directly lead to improving firms’ performances. In this study, we propose that firms in the manufacturing industry acquire more positive market reactions from market investors. Therefore, the third preposition we will explore is as follows:
Proposition 3 (P3).
The amount of abnormal return is greater in the manufacturing industry than in the service industry.
Clear strategic goals and objectives are among the most important factors in successfully implementing information systems [40]. To fulfill their strategic goals, firms should bring their business objectives into alignment with the desired features of information systems [11]. Small vendors provide specialized and customized solutions to fit specific industries. Firms can make better decisions with data analysis results focusing on specific business objectives. For instance, small vendors tend to have industry-oriented solutions, such as clinical business analytics (e.g., Medidata), financial fraud management solution (e.g., Hortonworks), and data security intelligence software (e.g., Informatica). Large vendors provide general analytics solutions that can be applied in any industry (e.g., SAS and Oracle). Based on this phenomenon, we propose that market investors more positively evaluate small vendors’ application because those applications can deliver more industry-specific functions that help business achieve industry-oriented goals and objectives. Thus, we assume that small vendors’ products have more positive effects than those of large vendors’ in terms of the amount of the market value when firms announce that they will make investment in a big data analytics solution. Therefore, we formulate the following proposition:
Proposition 4 (P4).
The amount of abnormal return is greater in firms that implemented small vendors’ solutions than those that implemented big vendors’ solutions.
Firm size is often measured by the number of its employees [41] or total revenue. Previous studies recognize firm size as one of factors influencing the size of abnormal returns [28,31]. Big firms usually have sufficient capacity of human resources to operate largely spread business units. Compared to small firms, big firms are producing more business transaction data. Therefore, big firms possess larger transaction log data, compared to small firms. Large size of data is a necessary condition for big data analytics [42]. From this perspective, big firms have an advantage on utilizing big data analytics compared with small firms. When firms implement big data analytics solutions, they also need expert human resources who can understand and use analytics. However, as compared to big firms, small firms have a difficulty of acquiring human resources specialized in data analytics [43]. Hence, market investors expect that big firms can exploit big data analytics solutions better as compared to small firms. Since big firms are in a better position in terms of adapting and using big data analytics solutions, market investors place more value on big firms’ investment than on that of small firms. Therefore, we can formulate the following proposition:
Proposition 5 (P5).
The amount of abnormal return is greater in big firms than in small firms.

4. Method and Sample

4.1. Choosing Methodology

Overall, two main methodologies to measure the economic value of information technology investments are reported in information systems literature [12,14,27,28]. One is measuring return on assets (ROA) of firms or return on investment (ROI) and the other methodology is the event methodology measuring abnormal return on stock price. To measure return on assets or return on investment, pre-announcement performance and post-announcement performance should be compared [12]. Considering that big data analytics is an emerging technology [20], in this study, we adopted the event study methodology which is often used in information systems research [28,30,31]. For example, if Facebook adopts IBM’s big data analytics solutions, we measure the abnormal returns of Facebook in stock market.

4.2. Data

We focused on listed firms in the national association of securities dealers’ automated quotations (NASDAQ) and New York stock exchange (NYSE). Both of stock exchange locations are headquartered in New York City, United States. Announcements of big data analytics solution investments were collected by Capital IQ’s key development screening report function. A 6-year period (from 1 January 2010 to 31 December 2015) was included. We excluded the date or period of particular events which could affect to stock price (e.g., subprime mortgage crisis). We searched announcements by inputting combinations of keywords and vendor names [13]. We used key words such as “big data analytics”, “data analytics”, “business analytics”, “SAP”, “Oracle” together and executed the query. A total of 97 announcements were extracted (see Table 2 for an example). However, announcements from 43 firms were not indexed in the national association of securities dealers’ automated quotations (NASDAQ) or New York stock exchange (NYSE). Therefore, this study only used 54 announcements of big data analytics investments. As big data analytics is a relatively new technology, there are only few pioneer organizations adopting big data analytics. As we wanted to investigate the very first effect of big data analytics investment, we focused on 54 announcements. These announcements had a form of news articles. Stock prices were extracted from the center for research in security price (CRSP) database. As center for research in security price recorded stock price fluctuation every day, the data we collected are non-stationary data.

5. Measure

5.1 Categorizing the Data

We suggested demographic factors of 54 firms in Table 3. The average revenue of 54 firms was $27,964,710. On average, 54 firms hired 42,646 employees. Firms’ total assets were $ 27,965,000. In 2015, the frequency of announcing big data analytics was 18, which is the highest among six years. Of the 54 firms, 39 firms were classified as service industry and 15 firms as manufacturing industry. Likewise, 36 firms were non-tech firms and 18 firms were information technology firms. Of the 54 firms, 28 firms also adopted big vendors’ big data analytics solutions and 26 firms implemented small vendors’ solutions.
In this study, hardware, software and electronic device manufacturers, ICT service firms, information technology consulting firms, and e-commerce firms were categorized as tech firms, because their business activities are closely related to technology industry [34]. If firms do not belong to the technology industry, they were coded as non-tech firms. To compare stock market reactions of non-tech firms and tech firms, we coded “zero” for tech firms and “one” for non-tech firms.
Industry has been used in several studies as a control variable to investigate the economic value of information system investment [27,28,44]. In this research, we categorized firms into non-service industries and service industries to find out how industry affects abnormal returns in stock price. The service industry was coded as “one” and the non-service industry was coded as “zero”.
We further categorized vendors based on their market share. SAP, Oracle, SAS, and IBM are big vendors. Their market share amounts to 50% of the total market [45,46]. To compare the stock market reactions of the solution vendor size, we coded “zero” for the firms implementing big vendors’ solution and “one” for firms adopting small vendors’ solution.
Median value of total revenue was used to assess firm size. If revenue of a firm is bigger than the median, we categorized it as a big firm and, if it is smaller than the median, we categorized the firm as a small firm. To compare the stock market reactions of big and small firms, we coded “zero” for the big firms and “one” for small firms.

5.2. Computation of Cumulative Abnormal Returns (CAR)

Based on the event methodology, we establish the market model for 54 firms. The market model is suggested as follows (see Equation (1)):
Rit = αi + βi Rmt + εit
where, Rit is the return for the firm i on Day t, Rmt is the return for the market in Day t, αi and βi are the estimated parameters, and εit is the error term for firm i on Day t.
Furthermore, we conducted a regression analysis for the market model. We used 255 estimation days to find out the αi and βi parameters. To make sure that the announcement would not influence parameters αi and βi, we selected the estimation window from t = −300 to t = −45 [47]. We selected the estimation window to control various factors affecting stock price (e.g., sport event, tragic news, weather-related events) [48,49,50,51]. As MacKinlay [52] confirmed that the event window can be extended to before and after the announcement, we examined (−1,1), (−2,2), (−1,0), and (0,1) windows for this research. For example, (0,1) window means that we examined stock price for two days: the event day (0) and the day after the event day (1).
After running the regression and determining the market model, we found out the difference between Rit and the expected return. We named the difference between Rit and expected return abnormal return (AR). AR was calculated as ARit = Rit − (αi + βiRmt). As specified in Section 3, we used the event methodology to measure abnormal returns on the stock price towards firms’ big data analytics investments. Investment on big data analytics will generate AR on the stock price, because investors positively evaluate adoption of big data analytics. In this context, ARit is the amount of generated abnormal returns by positive evaluation of investors.
We then cumulated abnormal returns (AR) over a specific event window from t1 to t2. We defined cumulative abnormal returns over specific event window as cumulative abnormal returns (CAR). We also computed mean CARs for n stocks as shown in Equation (2).
CAR = t = t 1 t 2 AR it   Mean   CAR = i = 1 n CAR i / n
Following the study of Im et al. (2001), we computed Z-statistics with standardized abnormal returns and tested the statistical significance on abnormal returns of firms by SPSS 22.0 [31]. In addition to the already existing parametric tests, we carried out a nonparametric analysis and a sign test to improve the robustness of our research findings [52]. The sign test is a nonparametric test. The test compares one pair of measurements by counting the number of positive and negative differences [53]. To validate the statistical significance, we also calculated Z score based on the ratio of the number of positive signs to the number of negative signs.

6. Results

6.1. CARs Results

Table 4 presents the results of testing proposition 1. Figure 1 also presents average AR during the event window. We investigated the CARs and Z-statistics in various event windows. We found significant positive abnormal return in event window (−1,1) and (−2,2). Furthermore, the (−1,1) window shows the biggest positive abnormal return (0.26) with significant Z-test statistics (2.60). We could find a positive abnormal return in the other windows, but only at the 0.1 significance level. In addition, results of nonparametric test support our findings in (−1,1) and (−2,2). The results of the sign test revealed that the number of positive signs is bigger than the number of negative signs in the (−1,1) and (−2,2) windows.

6.2. Regression Results

We found out the impact of big data analytics solution variables on CARs. A regression model was used to establish the impact of big data analytics solution variables on CARs (see Equation (3)).
CAR(−1,1) = β0 + β1Sector + β2Industry + β3Size + β4Vendor + ε
The regression results are summarized in Table 5. We chose the (−1,1) window to run the regression model because it has the biggest positive abnormal return at the 0.05 significance level. CARs are dependent variables in this model. Comparing F-value and R2 value to previous information systems studies utilizing event methodology, this model is significant [44,54]. In event window (−1,1), Chai et al. (2011) reported that F-value was 4.120 and R2 value was 0.184 [54]. Ranganathan et al. (2006) also reported that F-value was 3.199 and R2 value was 0.084 [44].
We included four independent variables in this regression model: sector of firms, industry, firm size and vendors. We found that only firm size and vendors have a positive relationship with CARs. When big firms announce the implementation of big data analytics solutions, market investors positively evaluate the adoption. When firms adopt small vendors’ solution with clear business objectives and insight, market positively reacts. Sector of firms and industry were not significant in this regression model.

7. Comparison

7.1. Big Vendor vs. Small Vendor

With regard to vendor size, Table 6 presents the CARs and Z-statistics for each category. We also added the results of parametric and nonparametric tests in Table 6. We found out that firms implementing small vendors’ solutions have bigger abnormal returns. The number of small vendors’ positive signs is bigger that the number of negative signs in all event windows. As small vendors tend to have industry-oriented solution, while large vendors provide general analytics solutions supporting any industry, this implies that investors assess that solutions-specialized clients’ business objectives could enhance firms’ performance. This result also implies that market investors consider that solution implementation without clear business objective could not improve firms’ performance. Therefore, before implementing the solutions, decision makers of firms should be sure about objectives of adoption and choose the solutions corresponding to firms’ business objectives.

7.2. Big Firm vs. Small Firms

With regard to firm size, Table 7 shows the CARs and Z statistics results for each category. We also performed parametric and nonparametric tests. The results suggest that big firms have bigger CARs. The number of big firms’ positive signs is bigger than the number of negative signs in all event windows. This result implies that market investors evaluate that big firms have a better condition for using big data analytics solutions. By contrast, investors assess that small firms, compare to big firms, could not effectively use the solutions.

8. Conclusions and Discussion

Big data analytics solutions are considered to be an emerging technology that can help organizations to improve their performance. However, in allocating resources, firms need to make an investment decision on the big data analytics solutions with recognizing economic value of big data analytics solutions. The present research proposes that investors in stock market positively consider investments on big data analytics. This research attempts to investigate an impact of firms’ market value towards investment announcements of big data solution by measuring abnormal returns of stock price. It also tries to identify business conditions that can affect the size of abnormal returns on the stock price, such as sector of firms, industry, vendor, and firm size. This study provides empirical evidence that firms’ investment on big data analytics solutions leads to positive abnormal returns on the stock price. Therefore, our results provide managerial insights that managers need to consider implementing big data analytic solutions, since investment decisions lead to the increase of the firm’s market value reflected in firm’s stock market price. For managers, investment decisions in information systems are always critical, leading to the following question: “Does big data solution investments lead to improving firm’s performance?” This research provides managers with empirical evidence in support of positive impacts of big data solution investments on the firm’s market value.
Our research results also support the prediction that firm size and vendor size make a different impact on the size of abnormal returns in stock market price. As compared to small firms, big firms have higher abnormal returns, reacting to the investments of big data solutions. In dealing with huge data due to their firm size, big firms demonstrate a more proactive approach in using their human resources and insights for employing big data analytics solutions. Therefore, market investors recognize these advantages of big firms’ resource perspective. This result is opposed to result of Im et al. (2001) [31]. Im et al. (2001) founded out that small firms had greater positive abnormal returns caused by IT investment. The contribution our results make to the industry is that big data analytic solutions can be applied into business conditions matching the use of big data with firms’ capacity of human resources with good skills on big data software. Second, vendor size has a different impact on the size of abnormal returns on the stock market price. By providing specialized analytic functions to clients with specific business objectives, client firms can have benefits of maximizing the use of big data solutions. On the other hand, big vendors provide general analytics functions that can be applied in firms in any industry. In contrast, solutions of small vendors are more suitable and effective than those of the big vendors to achieve firms’ business objectives. According to Ranganathan et al. (2006), market investors did not positively evaluate enterprise resource planning systems adoption by leading vendors [44]. Therefore, it is obvious that firms must choose solution of vendors which has a good fit with achieving their business objectives.
Market investors also recognize the different approach of big data solutions, provided by vendors. Our results point to useful implications for managers when they decide to select vendors for big data solutions. When managers decide to make an investment in big data solutions, they need to consider how they will utilize the big data solutions for their firms. If they attempt to use big data solution with the general approach, they can choose a big vendor. If firms attempt to apply a big data solution to industry specific objectives, they need to choose small vendors to meet their needs. Based on the results of the present research, when organizations take into account their capability and objectives in implementing big data analytics, they could select the best solutions for them. By adopting the best solutions and big data analytics, organizations easily achieve competitive advantage and productivity and consequently sustainability in their business.
However, our research results do not support the prediction that CARs are not affected by the sector of firms and industry. The sector of firms and industry of firms did not make a significant difference on measuring abnormal returns on the stock market price towards big data solutions investments. Market investors consider that big data solutions can be effectively applied into firms, regardless of the industry.
For investors in the stock market, this research provides useful insights about firms’ big data analytics investment. When investors want to invest on firms which announce investment in big data analytics, investors should check the firm size and capability to utilize big data analytics. Likewise, investors should also check whether a firm clearly understands the objectives of big data analytics and whether firms choose the best vendors that have a good fit with their business objectives. By considering these conditions, stock market investors could identify firms which properly invested on big data analytics and achieve competitive advantage in their business.
The limitation of our study is the small number of samples. Previous research using the event methodology collected over 80 announcements over a longer time period [16]. As big data analytics is a new and emerging technology with a short period of use [20], this research cannot collect a sufficient number of announcements during the past six years. To overcome this limitation, future research would focus on measuring long-term effects of adopting big data analytics solutions by taking return on asset (ROA) and return on investment (ROI) of firms into account.

Author Contributions

Hansol Lee and Sangmi Chai developed the research model. Hansol Lee, Eunkyung Kweon and Minkyun Kim conceived and designed the method and analyze data. Hansol Lee and Eunkyung Kweon and Sangmi Chai wrote the paper.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Hashem, I.A.T.; Yaqoob, I.; Anuar, N.B.; Mokhtar, S.; Gani, A.; Khan, S.U. The rise of “big data” on cloud computing: Review and open research issues. Inf. Syst. 2015, 47, 98–115. [Google Scholar] [CrossRef]
  2. Provost, F.; Fawcett, T. Data science and its relationship to big data and data-driven decision making. Big Data 2013, 1, 51–59. [Google Scholar] [CrossRef] [PubMed]
  3. Eric, S. Gartner: Top 10 Strategic Technology Trends for 2013. Available online: http://www.forbes.com/sites/ericsavitz/2012/10/23/gartner-top-10-strategic-technology-trends-for 2013/ (accessed on 7 May 2016).
  4. Cukier, K.; Mayer-Schoenberger, V. The Rise of Big Data: How it’s changing the Way We Think about the World. Foreign Aff. 2013, 92, 28–40. [Google Scholar]
  5. Power, D.J. Using ‘Big Data’ for analytics and decision support. J. Decis. Syst. 2014, 23, 222–228. [Google Scholar] [CrossRef]
  6. Lee, J.; Lapira, E.; Bagheri, B.; Kao, H.A. Recent advances and trends in predictive manufacturing systems in big data environment. Manuf. Lett. 2013, 1, 38–41. [Google Scholar] [CrossRef]
  7. IDC. Worldwide Big Data Technology and Services Forecast, 2015–2019. 2015. Available online: http://www.idc.com/getdoc.jsp?containerId=prUS40560115 (accessed on 7 May 2016).
  8. Tata Consultancy Services. The Emerging Big Returns on Big Data: A TCS Global Trend Study; Tata Consultancy Services: Mumbai, India, 2013; pp. 1–106. [Google Scholar]
  9. Bradford, M.; Florin, J. Examining the role of innovation diffusion factors on the implementation success of enterprise resource planning systems. Int. J. Account. Inf. Syst. 2003, 4, 205–225. [Google Scholar] [CrossRef]
  10. Fui-Hoon Nah, F.; Lee-Shang Lau, J.; Kuang, J. Critical factors for successful implementation of enterprise systems. Bus. Process Manag. J. 2001, 7, 285–296. [Google Scholar] [CrossRef]
  11. Umble, E.J.; Haft, R.R.; Umble, M.M. Enterprise resource planning: Implementation procedures and critical success factors. Eur. J. Oper. Res. 2003, 146, 241–257. [Google Scholar] [CrossRef]
  12. Hunton, J.E.; Lippincott, B.; Reck, J.L. Enterprise resource planning systems: Comparing firm performance of adopters and nonadopters. Int. J. Account. Inf. Syst. 2003, 4, 165–184. [Google Scholar] [CrossRef]
  13. Nicolaou, A.I. Firm performance effects in relation to the implementation and use of enterprise resource planning systems. J. Inf. Syst. 2004, 18, 79–105. [Google Scholar] [CrossRef]
  14. Hendricks, K.B.; Singhal, V.R.; Stratman, J.K. The impact of enterprise systems on corporate performance: A study of ERP, SCM, and CRM system implementations. J. Oper. Manag. 2007, 25, 65–82. [Google Scholar] [CrossRef]
  15. Fama, E.F.; Fisher, L.; Jensen, M.C.; Roll, R. The adjustment of stock prices to new information. Int. Econ. Rev. 1969, 10, 1–21. [Google Scholar] [CrossRef]
  16. Konchitchki, Y.; O’Leary, D.E. Event study methodologies in information systems research. Int. J. Account. Inf. Syst. 2011, 12, 99–115. [Google Scholar] [CrossRef]
  17. Gartner IT Glossary. 2016. Available online: www.gartner.com/it-glossary/big-data (accessed on 7 May 2016).
  18. Beyer, M.O.J. Gartner Says Solving ‘Big Data’ Challenge Involves More Than Just Managing Volumes of Data. Available online: http://www.gartner.com/it/page.jsp?id=1731916 (accessed on 27 June 2011).
  19. Kaisler, S.; Armour, F.; Espinosa, J.A.; Money, W. Big data: Issues and challenges moving forward. In Proceedings of the 46th Hawaii International Conference on System Sciences, Wailea, Maui, HI, USA, 7–10 January 2013. [Google Scholar]
  20. McKinsey Global Institute. Big Data: The Next Frontier for Innovation, Competition, and Productivity; McKinsey&Company: New York, NY, USA, 2011; pp. 1–20. [Google Scholar]
  21. Chen, H.; Chiang, R.H.; Storey, V.C. Business intelligence and analytics: From big data to big impact. MIS Q. 2012, 36, 1165–1188. [Google Scholar]
  22. LaValle, S.; Lesser, E.; Shockley, R.; Hopkins, M.S.; Kruschwitz, N. Big data, analytics and the path from insights to value. MIT Sloan Manag. Rev. 2011, 52, 21. [Google Scholar]
  23. Hong, K.K.; Kim, Y.G. The critical success factors for ERP implementation: An organizational fit perspective. Inf. Manag. 2002, 40, 25–40. [Google Scholar] [CrossRef]
  24. Poon, P.; Wagner, C. Critical success factors revisited: Success and failure cases of information systems for senior executives. Decis. Support Syst. 2001, 30, 393–418. [Google Scholar] [CrossRef]
  25. Ravichandran, T.; Lertwongsatien, C.; Lertwongsatien, C. Effect of information systems resources and capabilities on firm performance: A resource-based perspective. J. Manag. Inf. Syst. 2005, 21, 237–276. [Google Scholar] [CrossRef]
  26. Hayes, D.C.; Hunton, J.E.; Reck, J.L. Information systems outsourcing announcements: Investigating the impact on the market value of contract-granting firms. J. Inf. Syst. 2000, 14, 109–125. [Google Scholar] [CrossRef]
  27. Ranganathan, C.; Samarah, I. Enterprise resource planning systems and firm value: An event study analysis. ALSeL 2001, 19, 157–158. [Google Scholar]
  28. Hayes, D.; Hunton, J.; Reck, J. Market reaction to ERP implementation announcements. J. Inf. Syst. 2001, 15, 3–18. [Google Scholar] [CrossRef]
  29. Brown, S.J.; Warner, J.B. Using daily stock returns: The case of event studies. J. Financ. Econ. 1985, 14, 3–31. [Google Scholar] [CrossRef]
  30. Roztocki, N.; Weistroffer, H.R. How do investments in enterprise application integration drive prices? In Proceedings of the 40th Hawaii International Conference on System Sciences (HICSS), Waikoloa, HI, USA, 3–6 January 2007; pp. 1–8. [Google Scholar]
  31. Im, K.; Dow, K.; Grover, V. Research report: A reexamination of IT investment and the market value of the firm—An event study methodology. Inf. Syst. Res. 2001, 12, 103–117. [Google Scholar] [CrossRef]
  32. Schroeck, M.; Shockley, R.; Smart, J.; Romero-Morales, D.; Tufano, P. Analytics: The real-world use of big data in financial services. IBM Inst. Bus. Value 2012, 1, 1–22. [Google Scholar]
  33. Wu, S.I.; Lu, C.L. The relationship between CRM, RM, and business performance: A study of the hotel industry in Taiwan. Int. J. Hosp. Manag. 2012, 31, 276–285. [Google Scholar] [CrossRef]
  34. Storey, V.C.; Straub, D.W.; Stewart, K.A.; Welke, R.J. A conceptual investigation of the e-commerce industry. Commun. ACM 2000, 43, 117–123. [Google Scholar] [CrossRef]
  35. Bharadwaj, A.S. A resource-based perspective on information technology capability and firm performance: An empirical investigation. MIS Q. 2000, 24, 169–196. [Google Scholar] [CrossRef]
  36. Caldeira, M.M.; Ward, J.M. Using resource-based theory to interpret the successful adoption and use of information systems and technology in manufacturing small and medium-sized enterprises. Eur. J. Inf. Syst. 2003, 12, 127–141. [Google Scholar] [CrossRef]
  37. Lee, J.; Kao, H.A.; Yang, S. Service innovation and smart analytics for industry 4.0 and big data environment. Procedia CIRP 2014, 16, 3–8. [Google Scholar] [CrossRef]
  38. Wu, X.; Zhu, X.; Wu, G.Q.; Ding, W. Data mining with big data. IEEE Trans. Knowl. Data Eng. 2014, 26, 97–107. [Google Scholar]
  39. Xu, Y.; Yen, D.C.; Lin, B.; Chou, D.C. Adopting customer relationship management technology. Ind. Manag. Data Syst. 2002, 102, 442–452. [Google Scholar] [CrossRef]
  40. Somers, T.M.; Nelson, K. The impact of critical success factors across the stages of enterprise resource planning implementations. In Proceedings of the 34th Annual Hawaii International Conference on System Sciences, Maui, HI, USA, 3–6 January 2001; pp. 10–20. [Google Scholar]
  41. Kumar, K.B.; Rajan, R.G.; Zingales, L. What determines firm size? Natl. Bur. Econ. Res. 1999, w7208, 1–54. [Google Scholar]
  42. Russom, P. Big data analytics. In TDWI Best Practices Report, Fourth Quarter; Transforming Data with Intelligence (TDWI): Renton, WA, USA, 2011; pp. 1–35. [Google Scholar]
  43. Statistical Analysis System. Big Data Analytics: Adoption and Employment Trends, 2012–2017; Statistical Analysis System: Marlow, UK, 2013; pp. 1–24. [Google Scholar]
  44. Ranganathan, C.; Brown, C.V. ERP investments and the market value of firms: Toward an understanding of influential ERP project variables. Inf. Syst. Res. 2006, 17, 145–161. [Google Scholar] [CrossRef]
  45. Gartner. Market Share Analysis: Business Intelligence and Analytics Software, 2013. 2014. Available online: https://www.gartner.com/doc/2723017 (accessed on 7 May 2016).
  46. Gartner. Market Share Analysis: Business Intelligence and Analytics Software, 2014. 2015. Available online: https://www.gartner.com/doc/3083322 (accessed on 7 May 2016).
  47. Armitage, S. Event study methods and evidence on their performance. J. Econ. Surv. 1995, 9, 25–52. [Google Scholar] [CrossRef]
  48. Hirshleifer, D.; Shumway, T. Good day sunshine: Stock returns and the weather. J. Financ. 2003, 58, 1009–1032. [Google Scholar] [CrossRef]
  49. Kamstra, M.J.; Kramer, L.A.; Levi, M.D. Winter blues: A SAD stock market cycle. Am. Econ. Rev. 2003, 93, 324–343. [Google Scholar] [CrossRef]
  50. Edmans, A.; Garca, D.; Norli, O. Sports sentiment and stock returns. J. Financ. 2007, 62, 1967–1998. [Google Scholar] [CrossRef]
  51. Kaplanski, G.; Levy, H. Sentiment and stock prices: The case of aviation distasters. J. Financ. Econ. 2010, 95, 174–201. [Google Scholar] [CrossRef]
  52. Mackinlay, A. Event studies in economics and finance. J. Econ. Lit. 1997, 35, 13–39. [Google Scholar]
  53. Dixon, W.J.; Mood, A.M. The statistical sign test. J. Am. Stat. Assoc. 1946, 41, 557–566. [Google Scholar] [CrossRef] [PubMed]
  54. Chai, S.; Kim, M.; Rao, H.R. Firms’ information security investment decisions: Stock market evidence of investors’ behavior. Decis. Support Syst. 2011, 50, 651–661. [Google Scholar] [CrossRef]
Figure 1. Average abnormal return during the event window (%).
Figure 1. Average abnormal return during the event window (%).
Sustainability 09 00978 g001
Table 1. Prior studies examining implementation of IS and its effectiveness.
Table 1. Prior studies examining implementation of IS and its effectiveness.
ReferenceIS ImplementationMethodologyVariableKey Findings
Hayes et al. (2000) [26]
Period: 1990–1997
Information system outsourcingEvent study methodology
-
Firm size (small/ large)
-
Industry (service/ non service)
-
Significant positive abnormal return 0.124.
-
Significant bigger returns for small firms (Standardized abnormal returns 12.840).
Hayes et al. (2001) [28]
Period: 1990–1998
Enterprise resource planning (ERP) implementationEvent study methodology
-
Firm size and financial health (small/healthy; large/healthy; large/unhealthy; small /unhealthy)
-
Vendor size (large/small)
-
Significant positive abnormal return for Day 0 (0.102) and Day +1 (0.170).
-
Significant bigger returns for small /healthy firms than small and unhealthy firms (Standardized cumulative abnormal returns 0.552).
-
Significant bigger returns for large vendors (Standardized cumulative abnormal returns 16.389).
Poon and Wagner (2001) [24]Executive information systemsPersonal interviews with several key personnel
-
Committed and informed executive sponsor
-
Operating sponsor
-
Appropriate information systems staff
-
Appropriate technology
-
Management of data
-
Clear link to business objectives
-
Management of organizational resistance
-
Management of system evolution and spread
-
Evolutionary development methodology
-
Carefully defined information and system requirements
-
Successful organizations managed all critical success factors properly.
-
Failed organizations managed all critical success factors poorly.
Hong and Kim (2002) [23]Enterprise resource planning (ERP) implementation
-
Survey
-
Regression analysis
-
Organizational fit
-
Enterprise resource planning adaptation level
-
Process adaptation level
-
Organizational resistance
-
Enterprise resource planning systems implementation success is significantly related to organizational fit.
-
Enterprise resource planning adaptation level and process adaptation has moderating effects on enterprise resource planning implementation success.
Hunton et al. (2003) [12]
Period: 1990–1998
Enterprise resource planning (ERP) implementation
-
Testing differences between pre- and post-adoption with measures of firms’ performance:
-
Return on assets, return on investment, asset turnover, and return on sales
-
Financial health(healthy/unhealthy)
-
Firm size(small/large)
-
Enterprise resource planning (ERP) adopters had significantly higher return on assets, return on investment, and asset turnover.
-
Firm size positively affected to the firms’ performance.
-
Firm health significantly and positively affected performance.
Ravichandran and Lertwongsatien(2005) [25]Implementation of information technologyPartial least squares (PLS) with objective performance data (Return on assets and return on sales, sales growth)
-
Information systems resource
-
Information systems capabilities
-
Information technology support
-
Firm performance is influenced by information technology which enhances a firm’s core competency.
-
An organization’s ability to utilize IT to support its competency is de-pendent on information systems functional capabilities.
-
Information systems capability is dependent on human, technology, and relationship resources.
Hendricks et al. (2007) [14]
Period: 1991–1999
Enterprise resource planning (ERP), supply Chain management, customer(SCM) relationship management systems(CRM)
-
Testing for differences between pre- and post-adoption with our measures of firms’ performance: Return on assets and return on sales
-
Testing long term stock price performance
-
Investment in enterprise resource planning
-
Investment in supply chain management
-
Investment in customer relationship management systems
-
Enterprise resource planning systems: improvements in profitability (1.03% in return on assets).
-
Supply chain management systems: improvements in profitability (1.78% in return on assets and 1.44% in return on sales).
Table 2. Example of big data analytics solution implementation announcement.
Table 2. Example of big data analytics solution implementation announcement.
CompanyText of Announcements
The Cheesecake Factory, Inc.IBM (NYSE: IBM) today announced that The Cheesecake Factory Incorporated (NASDAQ: CAKE), with over 175 restaurant locations across the United States and three licensed locations in the Middle East, is using IBM Big Data analytics to help deliver the highest-quality experience to its guests.
(NASDAQ: CAKE)
26 February 2013
Cardinal Health, Inc.
-
Hortonworks, a leading contributor to Apache Hadoop, today announced that Cardinal Health, a leading provider of products and services supporting the healthcare industry, has selected to deploy their next generation data architecture based on the Hortonworks Data Platform (HDP). The industry’s only true 100% open source data platform powered by Apache Hadoop, HDP enables Cardinal Health to better serve its customers’ needs for rapid data management, as part of the supply chain management and business intelligence processes.
-
Cardinal Health provides pharmaceuticals and medical products to over 60,000 locations and helps their customers focus on patient care while reducing costs, as well as improving efficiency and overall quality. With the use of big data, Cardinal Health is looking to deploy a next-generation data architecture that improves the speed and analysis of large volumes of structured and unstructured information that further increases the value it delivers to its customers across all business units and enhances its organizational agility.
(NYSE: CAH)
13 May 2013
BlackRock, Inc.Splunk Inc. (NASDAQ: SPLK), provider of the leading software platform for real-time operational intelligence, today announced that BlackRock is significantly expanding its usage of Splunk® Enterprise. BlackRock Technology initially deployed Splunk software in 2012 and saw immediate success. Due to the proven value from the initial deployment, BlackRock recently decided to triple the size of its Splunk Enterprise license to address a broader range of requirements. Leading security teams consider all data to be security-relevant and need a solution that can scale with the exponentially growing volumes of data and perform big data security analytics. The use of Splunk software helps firms create a more effective big data security infrastructure to counter emerging and advanced threats.
(NYSE: BLK)
5 November 2013
Verizon Communications, Inc.Verizon Enterprise Solutions today announced collaboration with Cloudera, a leader in enterprise analytic data management powered by Apache Hadoop, to provide cloud-based big data analytics services to Verizon Cloud clients. Cloudera is the latest in the growing list of enterprise-class services being configured to run on the new Verizon Cloud, providing clients the flexibility to develop their own big data analytics applications based on the Apache Hadoop open source framework.
(NYSE: VZ)
7 November 2013
Joy GlobalIBM (NYSE: IBM) and Joy Global (NYSE: JOY), a worldwide leader in high-productivity mining solutions, today announced that Joy Global selected IBM Big Data and Analytics technology—including IBM’s advanced predictive analytics software and optimization solutions—to enhance the ability of Joy Global Smart Services to improve mining machine performance, while reducing downtime and costs.
(NYSE: JOY)
28 October 2014
Cisco Systems, Inc.
-
Tableau Software (NYSE: DATA), a global leader in rapid-fire, easy-to-use business analytics software, today announced Cisco has deployed Tableau across multiple business divisions in EMEA, in order to drive efficiency and better inform every part of the business, from senior business planning to in-field sales.
-
Cisco is a worldwide leader in IT and prides itself in helping companies to seize business opportunities through connecting the previously unconnected. Looking inwards, Cisco’s business intelligence team recognized that the company needed to better connect the dots and provide useful insight into business data that everyone could use, from field roles through sales managers, directors, and even senior executives.
(NASDAQ: CSCO)
29 October 2014
Cerner CorporationCloudera, the leader in enterprise analytic data management powered by Apache Hadoop™, today announced that Cerner Corp. (CERN), a long-time leader in the health IT space, is powering its Big Data platform with a Cloudera enterprise data hub to create a holistic understanding of the healthcare system and to improve patient outcomes.
(NASDAQ: CERN)
12 February 2015
Yahoo! Inc.Splunk Inc. (NASDAQ: SPLK), provider of the leading software platform for real-time Operational Intelligence, today announced that Yahoo has implemented Hunk®: Splunk Analytics for Hadoop and NoSQL Data Stores and Splunk® Enterprise. With a Hunk enterprise adoption license, Yahoo employees use Hunk to explore, analyze and visualize data from its Hadoop environment, which stores more than 600 petabytes of data. Yahoo teams are also analyzing more than 150 terabytes of machine data per day in Splunk Enterprise for use cases including IT operations, applications delivery, security, and business analytics.
(NASDAQ: YHOO)
17 February 2015
Harte Hanks, Inc.Harte Hanks (NYSE: HHS), a leader in developing customer relationships, experiences and defining interaction led marketing, and MapR Technologies, Inc.; provider of the top-ranked distribution for Apache™ Hadoop®, today announced Harte Hanks is using the MapR Distribution including Hadoop to evolve Harte Hanks’ big data solutions. By adopting the MapR data platform, Harte Hanks enhances the performance, scalability, and flexibility of its solutions, enabling its clients to more easily and quickly migrate, analyze, and store massive quantities of data.
(NYSE: HHS)
13 August 2015
Table 3. Description of 54 firms.
Table 3. Description of 54 firms.
(a) Firm Size(Revenue)
Minimum$70,000
Maximum$233,715,000
Mean$27,964,710
Standard deviation$45,415,917
Median$6,538,600
(b) Number of Employee
Minimum81
Maximum239,000
Mean42,646
Standard deviation57,771.57
Median18,000
(c) Total Assets
Minimum$117,000
Maximum$2,417,121,000
Mean$27,965,000
Standard deviation$448,202
Median$11,462,000
(d) Frequency of Year
20103
20113
20129
201313
20148
201518
(e) Industry
Service39
Non-service15
(f) Big Data Analytics Solution Vendors
Large vendors such as SAP, Oracle, IBM and SAS28
Other small vendors26
(g) Sector
Information technology firms18
Non-tech firm36
Table 4. CARs and Z-statistics for big data analytics solution investment.
Table 4. CARs and Z-statistics for big data analytics solution investment.
Event WindowParametric TestsNonparametric Tests
Mean CAR (%)ZPositive: Negative Signs of CARsZ
(−2,2)0.112.94 **38:162.98 **
(−1,1)0.262.60 **36:182.35 **
(−1,0)0.221.95 *35:192.20 **
(0,1)0.251.84 *31:231.02
*: Significant at the 10% level // **: Significant at the 5% level.
Table 5. Model: Cumulative abnormal returns (CARs) (−1,1).
Table 5. Model: Cumulative abnormal returns (CARs) (−1,1).
VariablesCoefficientt-Scorep-Value
Sector of firms0.1040.810NS
industry−0.054−0.421NS
Firm size(revenue)−0.353−2.7380.009
Vendors0.3132.4130.02
R20.203
Adjusted R20.138
F-value3.123 (significant at 0.023)
Table 6. CARs and Z-statistics: big vendor vs. small vendor.
Table 6. CARs and Z-statistics: big vendor vs. small vendor.
Event Window * Sig. at 0.10. **Sig. at 0.05Parametric TestsNonparametric Tests
Mean CAR (%)ZPositive: Negative Signs of CARsZ
Big vendor (n = 22)
(−2,2)0.281.88 *14:81.31
(−1,1)0.130.8313:90.84
(−1,0)0.090.6813:90.84
(0,1)0.100.4110:120.38
Small vendor (n = 32)
(−2,2)0.722.20 **23:92.48 **
(−1,1)1.242.68 **23:92.48 **
(−1,0)0.742.28 **23:92.48 **
(0,1)0.721.79 *20:121.41
*: Significant at the 10% level // **: Significant at the 5% level.
Table 7. CARs and Z-statistics: big firm vs. small firm.
Table 7. CARs and Z-statistics: big firm vs. small firm.
Event Window * Sig. at 0.10. **Sig. at 0.05Parametric TestsNonparametric Tests
Mean CAR (%)ZPositive: Negative Signs of CARsZ
Big firms (n = 28)
(−2,2)0.812.74 **21:72.64 **
(−1,1)1.372.82 **19:91.90 *
(−1,0)0.791.7020:82.22 **
(0,1)0.712.08 **17:111.16
Small firms (n = 26)
(−2,2)0.281.3917:91.52
(−1,1)0.160.8317:91.52
(−1,0)0.061.0515:110.71
(0,1)0.260.4914:120.31
*: Significant at the 10% level // **: Significant at the 5% level.

Share and Cite

MDPI and ACS Style

Lee, H.; Kweon, E.; Kim, M.; Chai, S. Does Implementation of Big Data Analytics Improve Firms’ Market Value? Investors’ Reaction in Stock Market. Sustainability 2017, 9, 978. https://doi.org/10.3390/su9060978

AMA Style

Lee H, Kweon E, Kim M, Chai S. Does Implementation of Big Data Analytics Improve Firms’ Market Value? Investors’ Reaction in Stock Market. Sustainability. 2017; 9(6):978. https://doi.org/10.3390/su9060978

Chicago/Turabian Style

Lee, Hansol, Eunkyung Kweon, Minkyun Kim, and Sangmi Chai. 2017. "Does Implementation of Big Data Analytics Improve Firms’ Market Value? Investors’ Reaction in Stock Market" Sustainability 9, no. 6: 978. https://doi.org/10.3390/su9060978

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop