Next Article in Journal
Observation of Ionospheric Gravity Waves Introduced by Thunderstorms in Low Latitudes China by GNSS
Next Article in Special Issue
Monitoring Climate Impacts on Annual Forage Production across U.S. Semi-Arid Grasslands
Previous Article in Journal
Building Cultural Heritage Resilience through Remote Sensing: An Integrated Approach Using Multi-Temporal Site Monitoring, Datafication, and Web-GL Visualization
Previous Article in Special Issue
Changes in Meadow Phenology in Response to Grazing Management at Multiple Scales of Measurement
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Remote Sensing
Volume 13
Issue 20
10.3390/rs13204132
remotesensing-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

The Relationship between Satellite-Derived Vegetation Indices and Live Weight Changes of Beef Cattle in Extensive Grazing Conditions

by
Christie Pearson
1,*,
Patrick Filippi
2 and
Luciano A. González
1
1
Sydney Institute of Agriculture, School of Life and Environmental Science, Faculty of Sciences, The University of Sydney, Camden, NSW 2570, Australia
2
Sydney Institute of Agriculture, School of Life and Environmental Science, Faculty of Sciences, The University of Sydney, Sydney, NSW 2006, Australia
*
Author to whom correspondence should be addressed.
Remote Sens. 2021, 13(20), 4132; https://doi.org/10.3390/rs13204132
Submission received: 15 August 2021 / Revised: 9 October 2021 / Accepted: 12 October 2021 / Published: 15 October 2021
(This article belongs to the Special Issue Applications of Remote Sensing for Livestock and Grazing Land Management)
Browse Figures
Versions Notes

Abstract

The live weight (LW) and live weight change (LWC) of cattle in extensive beef production is associated with pasture availability and quality. The remote monitoring of pastures and cattle LWC can be achieved with a combination of satellite imagery and walk-over-weighing (WoW) stations. The objective of the present study is to determine the association, if any, between vegetation indices (VIs) (pasture availability) and the LWC of beef cattle in an extensive breeding operation in Northern Australia. The study also tests a suite of VIs along with variables such as rainfall and Julian day to predict the LWC of breeding cows. The VIs were calculated from Sentinel-2 satellite imagery over a 2-year period from a paddock with 378 cattle. Animal LW was measured remotely using a weighing scale at the water point. The relationship between VIs, the LWC, and LW was assessed using linear mixed-effects regression models and random forest modelling. Findings demonstrate that all VIs calculated had a significant positive relationship with the LWC and LW (p < 0.001). Machine learning predictive modelling showed that the LWC of breeding cows could be predicted from VIs, Julian day, and rainfall information, with a Lin’s Concordance Correlation Coefficient of 0.62 when using the leave-one-month-out cross-validation. The LW and LWC were greater during the wet season when VIs were higher compared to the dry season (p < 0.001). Results suggest that the remote monitoring of pasture availability, the LWC and LW is possible under extensive grazing conditions. Further, the use of VIs and other readily available data such as rainfall can be used to predict the LWC of a breeding herd in extensive conditions. Such information could be used to increase the productivity and land management in extensive beef production. The integration of these data streams offers great potential to improve the monitoring, management, and productivity of grazing or cropping enterprises.

Graphical Abstract

1. Introduction

The live weight gain of cattle in extensive grazing conditions is directly linked to pasture quantity and quality [1]. The distinct wet and dry seasons affect pasture growth throughout the year, being highest during and immediately following the wet season (December to March in the tropics of the Southern Hemisphere) [2]. A cow’s live weight (LW) and body condition score (BCS) reflect this, typically being highest at the end of the wet season, with declines in the LW occurring as pasture quantity and quality decline throughout the dry season [1].
Satellite imagery with high a spatial and temporal resolution allows the calculation of VIs, which can be correlated to pasture quantity and quality [3,4]. The growth and weight of cows are of interest because they impact cows’ reproductive efficiency, including their ability to conceive, gestate, and raise a calf to weaning [5,6]. Cow and calf loss in extensive systems is responsible for large productivity losses, with the live weight of cows believed to impact the extent of calf loss in these systems [6]. Cattle live weight change (LWC) can also be measured using remote automatic weighing systems [7,8]. This information could be useful to integrate with remotely sensed VIs. However, no studies have been published integrating data streams from the remote monitoring of both vegetation and cattle LWC in extensive systems. Such information could then be used to assess the relationship between these variables throughout the seasons. It could also be used to manage animals in regard to nutrition, through paddock change, and feed supplementation. Mating and weaning strategies, amongst others [8], could also benefit from this information.
VIs can assist in understanding the greenness and digestibility of a pasture [9]. VIs have been known to help in grazing management decisions in the past [10,11], making them a viable option for low-input, large-scale beef enterprises. Pairing VIs with LW and LWC data could help to determine thresholds linked to animal performance, such as LW loss in larger operations with low inputs where this is not measured. Machine learning techniques have been used in the industry to predict meat characteristics and calving time [12,13], and have the potential to predict production outcomes such as the LWC by utilising ancillary variables such as VIs and rainfall in extensive production systems. To the author’s best knowledge, there are no studies which incorporate the use of VIs to predict changes in the growth of breeding cows under commercial conditions in Northern Australia. An investigation into this area would be of benefit in increasing the productivity of breeding animals under extensive conditions. Due to the global availability of freely accessible data, such as rainfall and VI from satellites, predictive modelling can nowcast the growth of cows in the coming months and could have benefits for management interventions such as the stocking rate, provision of supplementation, early warning indicators for poor performance of animals, and predictions of mortality.
The hypothesis of the present study is that there is a relationship between VIs and cattle LW and LWC. The objectives of the present study are, firstly, to assess the relationship between VIs and cattle LW and LWC under extensive grazing conditions of northern Australia. Secondly, the study aims to utilise machine learning to develop a model which could predict the LWC of breeding cows using VIs and freely available data such as rainfall.

2. Materials and Methods

All methods for the present project were approved by The University of Sydney Animal Ethics Committee (Animal Ethics Project Number 2017/1296).

2.1. Cattle Handling

The study was conducted on a large commercial beef enterprise in the Western Victoria River Downs Region of the Northern Territory, Australia. The property is 2480 km2, and the trial paddock 33 km2 (Figure 1). The area has a tropical climate, with distinct wet and dry seasons. Long-term average rainfall and temperature in the region is 800 mm of rainfall per year, with an average daily maximum temperature of 35.0 °C and a minimum of 21.2 °C [14]. Rainfall is predominantly during the wet season from December to March. Vegetation is, predominantly, lightly wooded eucalypt, dominated by tropical tall grasses such as Mitchell grass (Astrebla spp.) [2]; however, the paddock is heterogenous with variation in vegetation and land type.
Animals were mustered in April 2018 with a helicopter (R-22, Robinson Helicopter Company, Torrance, CA, USA) as per regular mustering processes in the area. During the two-year trial running from April 2018 to April 2020, a total of 378 trial animals were monitored. Trial cattle from the larger mob mustered in April 2018 were processed and selected based on gestation stage as described in Pearson et al., 2020 [15]. Trial animals were mostly Brahman (Bos Indicus) (80%) with remaining animals being of shorthorn or composite crosses. Cows were aged between 2 and 10 years old.
Data collected for each individual cow in the larger herd included LW and LWC, measured using a walk-over-weighing (WoW) station, which was set up at the entrance to water and configured as previously described by Gonzalez et al., 2014 [7]. Trial cows were provided with a loose mix urea supplement from June 2018 throughout the year until mid-November and then provided with phosphorus supplementation at the onset of the wet season in November 2018. In 2019, cows were provided with loose mix urea at the onset of calving in September until December, but were not provided with phosphorus lick during the 2019–2020 wet season. The dry season urea loose mix was 36% and 32% urea for each year, respectively (Table 1; Red Range Stock Supplements (Kununurra, Western Australia, Australia), and the wet season phosphorus lick block was a custom mix of 16% phosphorus (Olsen’s, Yennora, New South Wales, Australia). Animals consumed an average of 150 g per head per day during the dry season in both years and 66 g per head per day of phosphorus supplement in the wet season of 2018–2019.
All animals were subjected to commercial conditions and treatments for the duration of the trial period, including provision of supplementation and mustering procedures. Continuous mating was practised on the trial property, with calving structured around segregation of cows according to gestation stages. Weaning of calves was generally performed twice per year, once during the April and May period and again before the wet season in September and October, depending on the time calves were born during the year. Calves were between 2 and 6 months old at weaning. In 2018, weaning was performed on the 14 October and, in 2019, weaning was conducted on the 26 November. There were 8 bulls in the trial paddock during both years, which, along with the cows, used the WoW stations to access water. In addition, there were 126 calves born mid-year in 2018 and 134 calves in 2019, born later in the year. The LW and LWC data for the present study included the presence of bulls and calves in the paddock if these were tagged at muster and not weaned. The expected calving date in 2018 for 67% of the herd was mid-August, with 27% due between June and July 2018 and the remaining cows due in May 2018. In 2019, 58% were due mid-September and the remaining cows were due to calve from October to December. Weaning percentage in both years was 70%.

2.2. Satellite Data and Calculation of Vegetation Indices

Satellite data were retrieved through Google Earth Engine (GEE) [16], which were used to access Sentinel-2 Surface Reflectance satellite imagery at 10 m spatial resolution and 5 day temporal resolution for the trial paddock from 24 April 2018 to 9 April 2020. Sentinel-2 satellite data were chosen primarily because of the high temporal resolution, and the number and type of spectral bands available. A cloud-masking filter was applied to remove all pixels that were affected by cloud cover as per Filippi et al. 2020 [17]. Images with more than 40% cloud cover of the study paddocks were removed from the analysis to ensure a representative portion of the paddock. The average value for each VI for each image was calculated for the study area.
For the remaining pixels within each image, the number of VIs was calculated, including: Enhanced Vegetation Index (EVI) [18,19], Normalised Difference Vegetation Index (NDVI) [20], Normalised Difference Red Edge (NDRE) [21], Colour Infrared (CIr) [22,23], Vogelmann Red Edge Index (VREI) [24], and Bare Soil Index (BI) [25]. All indices were fitted with a penalised B-spline [26] to smooth the data and interpolated using the cubic method to fill in the gaps with no satellite data across the 2-year period. The VI was estimated for each day, matching the daily cattle weight data.
These VIs were used to assess the relationship with LWC of animals in the present study. The value of VIs typically represented the density or quantity of green forage available in the trial paddock. Different indices often represented different aspects of vegetation or complemented each other in terms of their sensitivity to prevailing vegetation and soil reflectance. Therefore, a suite of VIs was assessed in the present study to determine their association with LWC and LW (Table 2). As an example, NDVI and VREI were chlorophyll sensitive [21], but NDVI often saturated at high levels of green vegetation and was impacted more by soil and shadow. In contrast, EVI addressed some of these issues and often had reduced interference from soil and atmospheric noise [27]. Similarly, NDRE is known to be more accurate for use in mid to late season biomass and tree crops, and was less likely to oversaturate compared to NDVI [28]. The BI was useful for improving the identification of bare soil areas [25], which were especially common in areas of high cattle traffic activity, such as water and supplement points. The CIr is commonly used to assess vegetation, as its combination of bands shows vegetation to appear red, giving different hues of the colour to show healthy or sparse areas of vegetation [22,23].

2.3. Live Weight and Live Weight Change Data

The WoW stations were placed at water points in the paddock and collected LW data for each animal when attending the water point throughout the trial. The WoW stations were erected at the entrance to water, and data processing was performed as described by González et al. [7]. Animals which accessed the water point during this period were automatically weighed and their electronic identification (EID) ear tag number (Allflex Australia Pty Ltd, Capalaba, QLD, Australia) was recorded, along with the date and time. The WoW data were stored on board the processor (indicator model WoW2; Tru-Test WOW, Pakuranga, New Zealand). Data were downloaded once per fortnight, depending on accessibility to the site by the research team. Once the data were collected, processing and filtering of data were performed, removing outliers and fitting penalised B-splines [26] for each animal, and then interpolating between measures to fill in days with no data.

2.4. Statistical Analysis and Linear Mixed-Effects Modelling

The VIs and WoW data were merged for each cow and date in RStudio [29]. There were 2 experimental years, with a wet and dry season for each. Year one was from the 24 April 2018 to the 8 April 2019, and year two from the 9 April 2019 to the 9 April 2020, coinciding with mustering. The initial linear mixed-effects regression model contained the fixed effect of experimental year, date, and their interaction, and animal ID as random effect with the dependent variables being LW and LWC. This model was not a predictive model, but was used to assess the effect of time and obtain mean daily values on the variables of interest. The date × year interaction was significant for all variables (p < 0.05) and, thus, the analysis was performed for each year separately. However, figures were presented with the two years together to show seasonal variation across the whole trial. The association between variables and equations to predict LWC from the predictor covariate such as VIs was assessed using linear mixed-effects regression, with cow EID and date as random effects. Significance was declared at p < 0.05.

2.5. Predictive Machine Learning Model

A predictive model of cattle LWC was created using a random forest machine learning model. Specifically, the ‘ranger’ function within the ranger package in R [30] was used, which is a fast implementation of a random forest model. Live weight change was the response variable, and the predictor variables were monthly rainfall, year, month, Julian day, and the suite of VIs. Rainfall data were measured at the closest Bureau of Meteorology weather station in Kununurra, 50 km from the trial site. Random forest models have an ability to deal with non-linear relationships and are well-suited to diverse and high-dimensional datasets [31,32]. The dataset used for the random forest modelling included only breeding cows with sufficient WoW data across the trial period. Cows which had less than 90 days of data recorded, or less than 10 observations on the WoW system, were not included. The dataset contained 65,000 rows of data over a 24-month period from 148 cows. This data subset with cows that had a more complete time-series of data was selected, as the aim of this modelling was to predict or nowcast the LWC of those animals for the subsequent month. The predictor variable importance was assessed using the ‘permutation’ approach in the ‘ranger’ function in R [33].
To test the quality of the models and validate the predictions, a leave-one-month-year-out approach was implemented in this study. This involved removing all data from a particular year and month, and then building a model with the remaining data. This model was then used to predict LWC of the removed data, and the predicted LWC was retained. This procedure was implemented for all years/months in the dataset (n = 24 months). The relationship between all of the observed live weight change data and independently predicted LWC data was then assessed using R2, Lin’s concordance correlation coefficient (LCCC) [34], root mean square error (RMSE), and bias.
The aim of creating this predictive LWC model was to test the ability to predict or nowcast the LWC of cattle. The robust validation implemented provided a realistic insight into whether LWC could be predicted in the future using machine learning in combination with readily available data such as satellite VIs, rainfall data, and dates.

3. Results

Animals were heavier in the second year of the trial, but had less daily LWC (p < 0.05) compared to year one (Table 3). The large range in the LW of over 750 kg (Table 3) was due to the presence of bulls, cows, and calves (data not shown). The yearly average of VIs was similar amongst years, although the maximum values of the mean study area were higher in year two compared to year one (Table 3). The VI with the largest coefficient of variation was reported for BI, followed by VREI and CIr, and the variability was greater in year 2 compared to year 1 (Table 3).
The LWC over both years of the trial reached the maximum values early in the wet season in January and their lowest values at the end of the dry season in October and November (p < 0.001; Figure 2). However, the LW reached its maximum before the start of the dry season in July and its minimum at the end of the dry season in November (p < 0.001; Figure 2).
The temporal changes in VIs showed an increase during the wet season (Figure 3). The VIs peaked in the middle of the wet season in February and March, coinciding with the greatest LWC, except for BI, which showed a decline with the onset of the wet season, as bare soil areas began to grow pasture with the onset of the rain (Figure 3). In contrast, the lowest values for all VIs, except for BI, reached their lowest point in the dry season, coinciding with the lowest LWC of the year (Figure 3). It was clear that the LWC of animals mirrored that of changes in VI, despite slight lags in the LWC after increases in VI, signifying the importance of VIs in the changes to animal LWC through the years.
There were significant relationships between all VIs and the LWC of cattle (p < 0.001; Table 4). Regression coefficients for all VIs except BI showed positive values (p < 0.001). Each unit increase in VIs resulted in an LWC increase by up to 33 kg in VIs such as VREI (Table 4). Similarly, a value of zero for all VIs except BI corresponded to the weight loss of the animals reflected in a negative intercept (Table 4).
Machine learning random forest modelling results predicted the growth of breeding cows using leave-one-month-out cross-validation from VIs, rainfall, day, month, and year of the trial period, with an LCCC of 0.62 when testing with LOMOCV (Table 5). The density plot of the observed vs. predicted LWC showed that many points were poorly predicted; however, a higher density of points fell along the 1:1 line (Figure 4).
The assessment of the model variable importance suggested that CIr was the most important predictor of the LWC in the random forest model (Figure 5). This was followed by the BI index and day of the year. Rainfall and VREI shared similar importance as predictors, followed by the month of the year. Less important predictors of the LWC were NDRE and NDVI, suggesting benefits of certain VIs compared to others in the ability to predict the LWC in extensive grazing conditions.

4. Discussion

The present study looked at the relationship between several VIs derived from Sentinel-2 and the LWC of cattle with linear mixed-effects models in a commercial breeding herd in northern Australia. It also used machine learning to create a predictive model of the LWC in breeding cows from a suite of VIs along with date and rainfall. To the author’s best knowledge, this was the first study to show the relationships between these variables in extensive commercial beef breeding operations using remote sensing technologies.
On the one hand, this knowledge could be used to predict the growth rate of cattle from satellite imagery, which is of importance to manage livestock production systems, including nutrition and fertility [35,36]. On the other hand, the fact that this information can be collected in near-real time and remotely with the set of technologies used in the present study, supports the hypothesis that the approach can be used to help in making more informed and timely decisions regarding animal management. Such decisions could involve the time to start and finish feed supplementation and the amount to be provided, the weaning time and age of calves, and paddock changes where possible [8]. For example, if the weight loss could be predicted by the VI the month before, early management interventions could include the onset of supplementary feed prior to weight loss, or the weaning of calves earlier than planned in a breeding system. Similarly, stocking rates could be modified based on predictive modelling, with potential for productivity gains as well as improved animal welfare. These results could also have substantial impact on rotational grazing systems, where predictions of the LWC and pasture availability are of high importance. This could also increase the productivity of breeding herds by determining threshold values beyond which negative impacts on production and animal welfare could occur, such as excessive weight loss and the survival of animals. However, further research is needed to determine these thresholds under multiple environmental conditions and management practices.
The strong relationships between VIs and animal performance demonstrate the importance of vegetation quality and quantity in extensive production systems. Despite the broad range of studies in which machine learning has been utilised for beef [37,38,39] and dairy cattle [40], to date, there are no studies which incorporate the use of VIs to explain the LWC of animals under commercial conditions of northern Australia. In addition to the VIs presented, other VIs were also used in the preliminary analysis, but not included in the paper due to their low correlation with the LWC. These included the Canopy Chlorophyl Content Index (CCCI) [41], Modified Soil-Adjusted VI (MSAVI) [42], Normalized Difference Moisture Index (NDMI) [43], and Modified Chlorophyll Absorption Ratio Index (MCARI) [44]. Furthermore, CIr seemed to be the most suitable VI to predict animal performance due to its greatest importance in the random forest model.
The machine learning predictive modelling dataset only included breeding cows, with bulls and calves removed from the data. This was because the objective of the predictive model was to predict the growth for breeding cows in a given month, so sufficient live weight data were necessary to ensure this model was a robust prediction of the LWC. However, results from linear mixed-effects models included the whole herd data with bulls and calves. The aim of the linear mixed-effects models was to establish whether there were relationships at the herd level with the VI. Since there were significant relationships with VIs and herd data, the presence of only cows with sufficient live weight data would only enhance the strength of these relationships with the VI. It is important to point out that the association between VIs and the LWC could also have been affected by other factors affecting energy and nutrient demands of the animals, including the lactation status, weaning, and feed supplementation, amongst others.
It was clear that cows responded with an increasing LWC immediately after weaning in October and November, and before rainfall and pastures started to green up, suggesting that the lactation status may be more important in affecting the LWC than the pasture biomass and availability and, indirectly, the VI. Interestingly, VIs were higher in the second wet season of the trial, which experienced a higher rainfall, even though cow live weight gains and the LW were lower than the first year. There could be several reasons for this, and one possible explanation is the lack of wet season phosphorus supplementation in year 2. Phosphorus supplementation during the wet season is critical to maintain the growth rate and fertility of cattle in northern Australia due to deficiency in soils and vegetation [45,46]; therefore, demonstrating that there may be other factors influencing cow LWC.
Satellite imagery enabling predictions of vegetation quantity and quality are not new [47], and their roles in agriculture have been proven useful, for example, for estimating crop health and productivity [17,41,48], and the forage quantity of grasslands alike [49,50]. However, results of the present trial relating VIs with important profit drivers in the cattle industry, such as animal performance, added considerable value to the information that satellite data can provide. The predictive modelling was applied under these conditions specific to this region and at this time. It is important to note that the independent variables of the predictive model and the parameters presented in the present study may not be applicable to other regions or operations. For example, regions with a larger variation in VIs such as more arid areas may show stronger associations with the LWC. Measuring the relative importance of other variables in each production system would also be important to quantify in future, to enable more robust predictive modelling in future research.
Traditionally, the productivity of extensive beef breeding operations in northern Australia has been measured through weaning or branding percentages (number of calves branded or weaned from every 100 cows exposed to a bull). It has previously been noted that new technologies could enhance the productivity and profitability in beef breeding herds of northern Australia [8,51]. This study offers an approach which utilises technology linking cattle production with potential indicators such as VIs and, hence, could also be linked to reproduction and calf loss [6]. However, further research on the use of this information to predict, monitor, and improve the reproductive performance and cow and calf survival is needed. Importantly, the present study emphasizes the broad range of factors which may contribute to calf loss and the overall productivity of a breeding herd. With further research, the predictive information that VI has on the LWC of animals in extensive grazing systems could provide an alternative method to predict the risk of calf and cow loss and reproductive efficiency.
Improvements to this study may include a longer-term assessment of the relationship that VIs and, hence, pasture quality and availability have on breeder productivity such as the weaning rate and cow growth rate over multiple years and locations. This statement was supported by the findings that the regression coefficients linking the VI and performance were different between years and amongst the different VIs calculated. Larger datasets could mean that the industry may be able to use VIs as a tool to estimate productivity at any point in time in relation to the stocking rate. Drawbacks of the present study included the use of only one paddock with one group of animals and one location over two years. Rainfall data were taken from approximately 50 km away from the trial paddock. Although this was a significant distance, this was the most complete rainfall dataset available for this research. This was also an accurate reflection of the many remote cattle stations and the difficulties in obtaining accurate site-specific rainfall data to conduct modelling and an analysis such as that implemented in this study. Future research should consider including more site-specific rainfall data.
Further studies showing the impacts of cow LW and LWC on calf survival and pregnancy rates may also help to consolidate the importance of pasture availability in beef cattle management under extensive commercial conditions. In addition, future research may also include the quantification of the LWC with and without dry season supplementation with urea-based supplements, and wet season phosphorus supplementation. The interaction between feed supplementation and VIs may help to refine supplementation strategies and quantify the impact on the LWC and LW of cows under extensive grazing conditions.

5. Conclusions

The present study supported the hypotheses that the remote monitoring of tropical rangelands with satellite imagery can be used to calculate VIs, which are associated with the LWC and LW of cows in commercial grazing conditions. Machine learning models allow the prediction of the LWC in breeding cows from satellite VI, date, and rainfall data. The combination of satellite imagery and automatic remote weighing of animals offers a tool to monitor and manage animals in near-real time under extensive grazing conditions and a further investment and development of these tools is encouraged. Longer term data collection in different locations could help in developing robust tools and models with applications in diverse conditions. However, the present study provides a framework integrating multiple technologies for the remote monitoring of animals in extensive conditions. A further investigation into the suitability of predictive modelling in various regions to nowcast and forecast cattle live weight changes and other productivity indicators from satellite-derived vegetation data could provide tools to improve land and animal management.

Author Contributions

Conceptualization, L.A.G.; Data curation, C.P., P.F. and L.A.G.; Formal analysis, C.P., P.F. and L.A.G.; Funding acquisition, L.A.G.; Investigation, C.P.; Methodology, L.A.G.; Project administration, L.A.G.; Resources, C.P., P.F. and L.A.G.; Software, P.F. and L.A.G.; Supervision, P.F. and L.A.G.; Validation, C.P., P.F. and L.A.G.; Visualization, C.P.; Writing—original draft, C.P.; Writing—review and editing, P.F. and L.A.G. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by Meat and Livestock Australia Donor Company, the Commonwealth Government of Australia, and The University of Sydney (project number P.PSH.0817).

Data Availability Statement

Data available on request from L.A.G. due to restrictions.

Acknowledgments

We would like to gratefully acknowledge funding from the Meat and Livestock Australia Donor Company, the Commonwealth Government of Australia, and The University of Sydney. Their support was pivotal in the success of completing this study. We would also like to thank Dave and Ellie, managers of the station where the study took place, who far surpassed our hopes in their enthusiasm, commitment, and long-term dedication to this work; they are a credit to the livestock industry. Without their help and passion, this study would not have been possible.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript.

References

  1. McLean, R.W.; McCown, R.L.; Little, D.A.; Winter, W.H.; Dance, R.A. Persistence and growth of lotononis bainesii-digitaria decumbens an analysis of cattle liveweight changes on tropical grass pasture during the dry and early wet seasons in northern australia. J. Agric. Sci. 1983, 101, 17–24. [Google Scholar] [CrossRef] [Scilit]
  2. Cowley, T.; Oxley, T.; MacDonald, N.; Cameron, A.G.; Conradie, P.; Collier, C.; Norwood, D. The 2010 Pastoral Industry Survey. Northern Territory Wide; Northern Territory Government: Darwin, Australia, 2015.
  3. Durante, M.; Oesterheld, M.; Piñeiro, G.; Vassallo, M.M. Estimating forage quantity and quality under different stress and senescent biomass conditions via spectral reflectance. Int. J. Remote Sens. 2014, 35, 2963–2981. [Google Scholar] [CrossRef] [Scilit]
  4. Wachendorf, M.; Fricke, T.; Möckel, T. Remote sensing as a tool to assess botanical composition, structure, quantity and quality of temperate grasslands. Grass Forage Sci. 2018, 73, 1–14. [Google Scholar] [CrossRef] [Scilit]
  5. Holroyd, R.G.; Entwistle, K.W.; Shepherd, R.K. Effects on reproduction of estrous cycle variations, rectal temperatures and liveweights in mated Brahman cross heifers. Theriogenology 1993, 40, 453–464. [Google Scholar] [CrossRef] [Scilit]
  6. Mayer, D.G.; McKeon, G.M.; Moore, A.D. Prediction of mortality and conception rates of beef breeding cattle in northern Australia. Anim. Prod. Sci. 2012, 52, 329–337. [Google Scholar] [CrossRef] [Scilit]
  7. González, L.A.; Charmley, E.; Henry, B.K. Modelling methane emissions from remotely collected liveweight data and faecal near-infrared spectroscopy in beef cattle. Anim. Prod. Sci. 2014, 54, 1980–1987. [Google Scholar] [CrossRef] [Scilit]
  8. González, L.A.; Kyriazakis, I.; Tedeschi, L.O. Review: Precision nutrition of ruminants: Approaches, challenges and potential gains. Animal 2018, 12, 246–261. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  9. Ausseil, A.; Dymond, J.; Dynes, R.; Shepherd, J.; DeVantier, B.; Sutherland, A. Estimating pasture quality using Landsat ETM+: Application for the greenhouse gas inventory of New Zealand. In Proceedings of the 34th International Society for Photogrammetry and Remote Sensing, Sydney, Australia, 10–15 April 2011; pp. 1–4. [Google Scholar]
  10. Flynn, E.S.; Dougherty, C.T.; Wendroth, O. Assessment of pasture biomass with the normalized difference vegetation index from active ground-based sensors. Agron. J. 2008, 100, 114–121. [Google Scholar] [CrossRef] [Scilit]
  11. Ali, I.; Cawkwell, F.; Dwyer, E.; Barrett, B.; Green, S. Satellite remote sensing of grasslands: From observation to management. J. Plant. Ecol. 2016, 9, 649–671. [Google Scholar] [CrossRef] [Scilit]
  12. Gredell, D.A.; Schroeder, A.R.; Belk, K.E.; Broeckling, C.D.; Heuberger, A.L.; Kim, S.Y.; King, D.A.; Shackelford, S.D.; Sharp, J.L.; Wheeler, T.L.; et al. Comparison of Machine Learning Algorithms for Predictive Modeling of Beef Attributes Using Rapid Evaporative Ionization Mass Spectrometry (REIMS) Data. Sci. Rep. 2019, 9, 1–9. [Google Scholar] [CrossRef] [Scilit]
  13. Miller, G.A.; Mitchell, M.; Barker, Z.E.; Giebel, K.; Codling, E.A.; Amory, J.R.; Michie, C.; Davison, C.; Tachtatzis, C.; Andonovic, I.; et al. Using animal-mounted sensor technology and machine learning to predict time-to-calving in beef and dairy cows. Animal 2020, 14, 1304–1312. [Google Scholar] [CrossRef] [Scilit]
  14. Bureau of Meteorology Daily Maximum Temperature. Available online: http://www.bom.gov.au/jsp/ncc/cdio/weatherData/av?p_nccObsCode=122&p_display_type=dailyDataFile&p_startYear=2018&p_c=-864745&p_stn_num=002056 (accessed on 9 April 2019).
  15. Pearson, C.; Lush, L.; González, L.A. Intravaginal devices and gnss collars with satellite communication to detect calving events in extensive beef production in northern australia. Remote Sens. 2020, 12, 3963. [Google Scholar] [CrossRef] [Scilit]
  16. Gorelick, N.; Hancher, M.; Dixon, M.; Ilyushchenko, S.; Thau, D.; Moore, R. Google Earth Engine: Planetary-scale geospatial analysis for everyone. Remote Sens. Environ. 2017, 202, 18–27. [Google Scholar] [CrossRef] [Scilit]
  17. Filippi, P.; Whelan, B.M.; Vervoort, R.W.; Bishop, T.F.A. Mid-season empirical cotton yield forecasts at fine resolutions using large yield mapping datasets and diverse spatial covariates. Agric. Syst. 2020, 184, 102894. [Google Scholar] [CrossRef] [Scilit]
  18. Bannari, A.; Morin, D.; Bonn, F.; Huete, A.R. A review of vegetation indices. Remote Sens. Rev. 1995, 13, 95–120. [Google Scholar] [CrossRef] [Scilit]
  19. Nagler, P.L.; Scott, R.L.; Westenburg, C.; Cleverly, J.R.; Glenn, E.P.; Huete, A.R. Evapotranspiration on western U.S. rivers estimated using the Enhanced Vegetation Index from MODIS and data from eddy covariance and Bowen ratio flux towers. Remote Sens. Environ. 2005, 97, 337–351. [Google Scholar] [CrossRef] [Scilit]
  20. Colwell, J.E. Vegetation canopy reflectance. Remote Sens. Environ. 1974, 3, 175–183. [Google Scholar] [CrossRef] [Scilit]
  21. Ahmad, S.; Chandra Pandey, A.; Kumar, A.; Parida, B.R.; Lele, N.V.; Bhattacharya, B.K. Chlorophyll deficiency (chlorosis) detection based on spectral shift and yellowness index using hyperspectral AVIRIS-NG data in Sholayar reserve forest, Kerala. Remote Sens. Appl. Soc. Environ. 2020, 19, 100369. [Google Scholar] [CrossRef] [Scilit]
  22. Sripada, R.P.; Heiniger, R.W.; White, J.G.; Meijer, A.D. Aerial color infrared photography for determining early in-season nitrogen requirements in corn. Agron. J. 2006, 98, 968–977. [Google Scholar] [CrossRef] [Scilit]
  23. Ihse, M. Colour infrared aerial photography as a tool for vegetation mapping and change detection in environmental studies of Nordic ecosystems: A review. Nor. Geogr. Tidsskr. 2007, 61, 170–191. [Google Scholar] [CrossRef] [Scilit]
  24. Vogelmann, J.E.; Rock, B.N.; Moss, D.M. Red edge spectral measurements from sugar maple leaves. Int. J. Remote Sens. 1993, 14, 1563–1575. [Google Scholar] [CrossRef] [Scilit]
  25. Zhao, H.; Chen, X. Use of normalized difference bareness index in quickly mapping bare areas from TM/ETM+. Int. Geosci. Remote Sens. Symp. 2005, 3, 1666–1668. [Google Scholar] [CrossRef] [Scilit]
  26. Eilers, P.H.C.; Marx, B.D.; Ellers, P.H.C. Flexible Smoothing with B-splines and Penalties. Stat. Sci. 1996, 11, 89–102. [Google Scholar] [CrossRef] [Scilit]
  27. Liu, H.Q.; Huete, A. Feedback based modification of the NDVI to minimize canopy background and atmospheric noise. IEEE Trans. Geosci. Remote Sens. 1995, 33, 457–465. [Google Scholar] [CrossRef] [Scilit]
  28. Lu, J.; Miao, Y.; Shi, W.; Li, J.; Yuan, F. Evaluating different approaches to non-destructive nitrogen status diagnosis of rice using portable RapidSCAN active canopy sensor. Sci. Rep. 2017, 7, 1–10. [Google Scholar] [CrossRef] [Scilit]
  29. R Core Team R: A Language and Environment for Statistical Computing. Available online: http://www.r-project.org/ (accessed on 29 April 2020).
  30. Wright, M.N.; Ziegler, A. Ranger: A fast implementation of random forests for high dimensional data in C++ and R. J. Stat. Softw. 2017, 77, 1–17. [Google Scholar] [CrossRef] [Scilit]
  31. Genuer, R.; Poggi, J.-M. Random Forests with R; Springer: Cham, Switzerland, 2020; ISBN 9783030564841. [Google Scholar]
  32. Li, J.; Alvarez, B.; Siwabessy, J.; Tran, M.; Huang, Z.; Przeslawski, R.; Radke, L.; Howard, F.; Nichol, S. Application of random forest and generalised linear model and their hybrid methods with geostatistical techniques to count data: Predicting sponge species richness. Environ. Model. Softw. 2017, 97, 112–129. [Google Scholar] [CrossRef] [Scilit]
  33. Nembrini, S. On the behaviour of permutation-based variable importance measures in random forest clustering. J. Chemom. 2019, 33, 1–5. [Google Scholar] [CrossRef] [Scilit]
  34. Lin, L.I. A concordance correlation coefficient to evaluate reproducibility. Biomatrics 1989, 45, 255–268. [Google Scholar] [CrossRef] [Scilit]
  35. D’Occhio, M.J.; Baruselli, P.S.; Campanile, G. Influence of nutrition, body condition, and metabolic status on reproduction in female beef cattle: A review. Theriogenology 2019, 125, 277–284. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  36. Burns, B.M.; Fordyce, G.; Holroyd, R.G. A review of factors that impact on the capacity of beef cattle females to conceive, maintain a pregnancy and wean a calf-Implications for reproductive efficiency in northern Australia. Anim. Reprod. Sci. 2010, 122, 1–22. [Google Scholar] [CrossRef] [Scilit]
  37. Shahinfar, S.; Al-Mamun, H.A.; Park, B.; Kim, S.; Gondro, C. Prediction of marbling score and carcass traits in Korean Hanwoo beef cattle using machine learning methods and synthetic minority oversampling technique. Meat Sci. 2020, 161, 107997. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  38. Higaki, S.; Miura, R.; Suda, T.; Andersson, L.M.; Okada, H.; Zhang, Y.; Itoh, T.; Miwakeichi, F.; Yoshioka, K. Estrous detection by continuous measurements of vaginal temperature and conductivity with supervised machine learning in cattle. Theriogenology 2019, 123, 90–99. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  39. Chen, W.; Alexandre, P.A.; Ribeiro, G.; Fukumasu, H.; Sun, W.; Reverter, A.; Li, Y. Identification of Predictor Genes for Feed Efficiency in Beef Cattle by Applying Machine Learning Methods to Multi-Tissue Transcriptome Data. Front. Genet. 2021, 12, 103. [Google Scholar] [CrossRef] [Scilit]
  40. Shahinfar, S.; Page, D.; Guenther, J.; Cabrera, V.; Fricke, P.; Weigel, K. Prediction of insemination outcomes in Holstein dairy cattle using alternative machine learning algorithms. J. Dairy Sci. 2014, 97, 731–742. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  41. Li, F.; Miao, Y.; Feng, G.; Yuan, F.; Yue, S.; Gao, X.; Liu, Y.; Liu, B.; Ustin, S.L.; Chen, X. Improving estimation of summer maize nitrogen status with red edge-based spectral vegetation indices. Field Crop. Res. 2014, 157, 111–123. [Google Scholar] [CrossRef] [Scilit]
  42. Guo, B.; Yang, F.; Fan, Y.; Han, B.; Chen, S.; Yang, W. Dynamic monitoring of soil salinization in Yellow River Delta utilizing MSAVI–SI feature space models with Landsat images. Environ. Earth Sci. 2019, 78, 1–10. [Google Scholar] [CrossRef] [Scilit]
  43. Morresi, D.; Vitali, A.; Urbinati, C.; Garbarino, M. Forest spectral recovery and regeneration dynamics in stand-replacing wildfires of central Apennines derived from Landsat time series. Remote Sens. 2019, 11, 308. [Google Scholar] [CrossRef] [Scilit]
  44. Eitel, J.U.H.; Long, D.S.; Gessler, P.E.; Hunt, E.R. Combined spectral index to improve ground-based estimates of nitrogen status in dryland wheat. Agron. J. 2008, 100, 1694–1702. [Google Scholar] [CrossRef] [Scilit]
  45. Bowen, M.K.; Chudleigh, F.; Dixon, R.M.; Sullivan, M.T.; Schatz, T.; Oxley, T. The economics of phosphorus supplementation of beef cattle grazing northern Australian rangelands. Anim. Prod. Sci. 2020, 60, 683–693. [Google Scholar] [CrossRef] [Scilit]
  46. Coates, D.B.; Dixon, R.M.; Mayer, R.J. Between-year variation in the effects of phosphorus deficiency in breeder cows grazing tropical pastures in northern Australia. Trop. Grassl.-Forrajes Trop. 2019, 7, 223–233. [Google Scholar] [CrossRef] [Scilit]
  47. Kogan, F.; Gitelson, A.; Zakarin, E.; Spivak, L.; Lebed, L. AVHRR-Based Spectral Vegetation Index for Quantitative Assessment of Vegetation State and Productivity: Calibration and Validation. Photogramm. Eng. Remote Sens. 2003, 69, 899–906. [Google Scholar] [CrossRef] [Scilit]
  48. Filippi, P.; Jones, E.J.; Wimalathunge, N.S.; Somarathna, P.D.S.N.; Pozza, L.E.; Ugbaje, S.U.; Jephcott, T.G.; Paterson, S.E.; Whelan, B.M.; Bishop, T.F.A. An approach to forecast grain crop yield using multi-layered, multi-farm data sets and machine learning. Precis. Agric. 2019, 20, 1015–1029. [Google Scholar] [CrossRef] [Scilit]
  49. Chen, P.; Liu, H.; Wang, Z.; Mao, D.; Liang, C.; Wen, L.; Li, Z.; Zhang, J.; Liu, D.; Zhuo, Y.; et al. Vegetation Dynamic Assessment by NDVI and Field Observations for Sustainability of China’s Wulagai River Basin. Int. J. Environ. Res. Public Health 2021, 18, 2528. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  50. Dingaan, M.N.V.; Tsubo, M. Improved assessment of pasture availability in semi-arid grassland of South Africa. Environ. Monit. Assess. 2019, 12, 1–12. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  51. Ash, A.; Hunt, L.; McDonald, C.; Scanlan, J.; Bell, L.; Cowley, R.; Watson, I.; McIvor, J.; MacLeod, N. Boosting the productivity and profitability of northern Australian beef enterprises: Exploring innovation options using simulation modelling and systems analysis. Agric. Syst. 2015, 139, 50–65. [Google Scholar] [CrossRef] [Scilit]
Remotesensing 13 04132 g001
Figure 1. Satellite image of the trial paddock with water points displayed. Trial animals were able to access the dam on the northern boundary in Year 1, but not in Year 2.
Figure 1. Satellite image of the trial paddock with water points displayed. Trial animals were able to access the dam on the northern boundary in Year 1, but not in Year 2.
Remotesensing 13 04132 g001
Remotesensing 13 04132 g002
Figure 2. Live weight and live weight change measured using remote automatic weighing systems of cattle grazing in extensive conditions and over the two-year trial period.
Figure 2. Live weight and live weight change measured using remote automatic weighing systems of cattle grazing in extensive conditions and over the two-year trial period.
Remotesensing 13 04132 g002
Remotesensing 13 04132 g003
Figure 3. Temporal trends in vegetation indices using Sentinel-2 satellite imagery from a paddock grazed by cattle and daily growth rate of cattle (LWC) over a two-year period.
Figure 3. Temporal trends in vegetation indices using Sentinel-2 satellite imagery from a paddock grazed by cattle and daily growth rate of cattle (LWC) over a two-year period.
Remotesensing 13 04132 g003
Remotesensing 13 04132 g004
Figure 4. Observed and predicted live weight changes of breeding cows from satellite imagery, rainfall and day of the year in extensive conditions over the 2-year trial period.
Figure 4. Observed and predicted live weight changes of breeding cows from satellite imagery, rainfall and day of the year in extensive conditions over the 2-year trial period.
Remotesensing 13 04132 g004
Remotesensing 13 04132 g005
Figure 5. Variable importance of a random forest model predicting the live weight change of breeder cows under commercial conditions in northern Australia over a 2-year trial period. NDVI-Normalised Difference Vegetation Index; NDRE- Normalised Difference Red Edge; EVI- Enhanced Vegetation Index; CIr- Coloured Infrared; VREI- Vogelmann Red Edge Index; BI- Bare Soil Index; Rainfall- monthly cumulative rainfall data near trial site; Month- the numeric month of the trial from 1 to 24; Day- the Julian day of the trial for each year; Year- the numeric year of the trial,1 or 2.
Figure 5. Variable importance of a random forest model predicting the live weight change of breeder cows under commercial conditions in northern Australia over a 2-year trial period. NDVI-Normalised Difference Vegetation Index; NDRE- Normalised Difference Red Edge; EVI- Enhanced Vegetation Index; CIr- Coloured Infrared; VREI- Vogelmann Red Edge Index; BI- Bare Soil Index; Rainfall- monthly cumulative rainfall data near trial site; Month- the numeric month of the trial from 1 to 24; Day- the Julian day of the trial for each year; Year- the numeric year of the trial,1 or 2.
Remotesensing 13 04132 g005
Table 1. Nutritional information of dry season supplements for breeding animals under extensive conditions in northern Australia.
Table 1. Nutritional information of dry season supplements for breeding animals under extensive conditions in northern Australia.
Ingredient % as FedDry Season Urea—2018Dry Season Urea—2019
Urea equivalent36.2132.86
Phosphorus3.333.09
Crude protein2.884.25
Equivalent CP103.9294.31
Total CP106.8098.56
Calcium4.374.85
Sulphur3.323.92
Fibre0.550.95
Magnesium0.310.18
Potassium0.060.17
Nitrogen0.4515.27
Chloride16.7714.86
Sodium10.579.37
Dry matter76.5295.07
Metabolizable Energy, MJ /kg DM0.801.57
Table 2. Vegetation indices included in the study, and associated bands used to calculate the vegetation indices, and description.
Table 2. Vegetation indices included in the study, and associated bands used to calculate the vegetation indices, and description.
VIRedGreenBlueNIRRed EdgeDescription
NDVIX X Chlorophyll-sensitive
Identifying canopy structure
NDRE XXGood to estimate late season biomass and tree crops
Does not oversaturate images
EVIX XX Reduces soil and atmospheric noise
Sensitive to changes in high biomass
Detects variations in plant structure
CIrXX X Assesses soil moisture
Assesses soil composition
VREI XChlorophyll-sensitive
Sensitive to small changes in vegetation
BIXXX Identifies areas of bare soil
Good for detection of degradation and drought-affected areas
Table 3. Summary statistics for beef cattle performance and vegetation indices from Sentinel-2 satellite imagery measured across the two-year period.
Table 3. Summary statistics for beef cattle performance and vegetation indices from Sentinel-2 satellite imagery measured across the two-year period.
VariableMeanStd DevMinimumMaximumCV
Year 1Live weight change (kg/day)0.281.15−9.078.14414.2
Live Weight (kg)380.29115.6736.26797.5330.46
NDVI0.170.040.120.2421.5
NDRE0.130.030.100.1821.2
EVI0.140.030.110.1918.6
CIr0.360.110.240.5830.9
VREI0.030.010.020.0528.9
BI0.060.030.020.1043.8
Year 2Growth rate (kg/day)0.071.00−9.189.081440.2
Live Weight (kg)404.7798.0344.12835.0424.2
NDVI0.160.050.100.2832.4
NDRE0.120.040.080.2033.0
EVI0.130.050.080.2539.4
CIr0.350.140.220.6640.5
VREI0.040.020.020.0746.2
BI0.070.06−0.040.1386.6
Table 4. Linear mixed-effect regression models of vegetation indices and live weight change of grazing cattle calculated from Sentinel-2 satellite imagery in both experimental years.
Table 4. Linear mixed-effect regression models of vegetation indices and live weight change of grazing cattle calculated from Sentinel-2 satellite imagery in both experimental years.
YearVegetation IndexInterceptRegression Coefficient
Estimate ± SEp-ValueEstimate ± SEp-Value
Year 1NDVI−0.34 ± 0.06<0.0014.44 ± 0.10<0.001
NDRE−0.96 ± 0.06<0.00110.82 ± 0.13<0.001
EVI−0.77 ± 0.06<0.0018.54 ± 0.13<0.001
CIr−0.84 ± 0.06<0.0013.36 ± 0.03<0.001
VREI−0.72 ± 0.06<0.00133.19 ± 0.36<0.001
BI1.41 ± 0.06<0.001−16.46 ± 0.12<0.001
Year 2NDVI−0.84 ± 0.07<0.0016.61 ± 0.05<0.001
NDRE−0.94 ± 0.07<0.0019.79 ± 0.07<0.001
EVI−0.66 ± 0.07<0.0016.71 ± 0.05<0.001
CIr−0.75 ± 0.07<0.0012.76 ± 0.02<0.001
VREI−0.57 ± 0.07<0.00122.23 ± 0.17<0.001
BI0.71 ± 0.07<0.001−7.56 ± 0.05<0.001
Table 5. Validated model statistics for machine learning model predicting live weight change of breeding cows from vegetation indices, date, and rainfall under extensive conditions.
Table 5. Validated model statistics for machine learning model predicting live weight change of breeding cows from vegetation indices, date, and rainfall under extensive conditions.
R2* LCCC* RMSE (kg)Bias
0.440.620.67−0.02
* LCCC—Lin’s concordance correlation coefficient; RMSE—root mean squared error.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Pearson, C.; Filippi, P.; González, L.A. The Relationship between Satellite-Derived Vegetation Indices and Live Weight Changes of Beef Cattle in Extensive Grazing Conditions. Remote Sens. 2021, 13, 4132. https://doi.org/10.3390/rs13204132

AMA Style

Pearson C, Filippi P, González LA. The Relationship between Satellite-Derived Vegetation Indices and Live Weight Changes of Beef Cattle in Extensive Grazing Conditions. Remote Sensing. 2021; 13(20):4132. https://doi.org/10.3390/rs13204132

Chicago/Turabian Style

Pearson, Christie, Patrick Filippi, and Luciano A. González. 2021. "The Relationship between Satellite-Derived Vegetation Indices and Live Weight Changes of Beef Cattle in Extensive Grazing Conditions" Remote Sensing 13, no. 20: 4132. https://doi.org/10.3390/rs13204132

APA Style

Pearson, C., Filippi, P., & González, L. A. (2021). The Relationship between Satellite-Derived Vegetation Indices and Live Weight Changes of Beef Cattle in Extensive Grazing Conditions. Remote Sensing, 13(20), 4132. https://doi.org/10.3390/rs13204132

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop