Assessing Performance of Vegetation Indices to Estimate Nitrogen Nutrition Index in Pepper

: Vegetation indices (VIs) can be useful tools to evaluate crop nitrogen (N) status. To be e ﬀ ective, VIs measurements must be related to crop N status. The nitrogen nutrition index (NNI) is a widely accepted parameter of crop N status. The present work evaluates the performance of several VIs to estimate NNI in sweet pepper ( Capsicum annuum ). The performance of VIs to estimate NNI was evaluated using parameters of linear regression analysis conducted for calibration and validation. Three di ﬀ erent sweet pepper crops were grown with combined irrigation and fertigation, in Almer í a, Spain. In each crop, ﬁve di ﬀ erent N concentrations in the nutrient solution were frequently applied by drip irrigation. Proximal crop reﬂectance was measured with Crop Circle ACS470 and GreenSeeker handheld sensors, approximately every ten days, throughout the crops. The relative performance of VIs di ﬀ ered between phenological stages. Relationships of VIs with NNI were strongest in the early fruit growth and ﬂowering stages, and less strong in the vegetative and harvest stages. The green band-based VIs, GNDVI, and GVI, provided the best results for estimating crop NNI in sweet pepper, for individual phenological stages. GNDVI had the best performance in the vegetative, ﬂowering, and harvest stages, and GVI had the best performance in the early fruit growth stage. Some of the VIs evaluated are promising tools to estimate crop N status in sweet pepper and have the potential to contribute to improving crop N management of sweet pepper crops.


Introduction
Vegetable crops production is characterized by nitrogen (N) losses and the associated environmental problems [1][2][3]. The most common environmental problems include ground and surface water contamination, eutrophication of surface water, and nitrous oxide (N 2 O) emission [4,5]. These problems are often a consequence of the high use of N fertilizer as a way to ensure optimal growth and production [6], which generally exceeds the demand of the crops [3,7,8]. Knowing crop N requirements and matching N supply to crop demand are requirements to reduce N contamination of water bodies by intensive vegetable production [3,9,10]. Various tools are available for monitoring crop N status [3,11]. A traditional tool is leaf nutrient analysis, which requires laborious and time-consuming laboratory work, and which generally cannot characterize the temporal and spatial variability of N status [12,13]. These are major drawbacks, because knowledge of temporal and spatial variability of crop N status appreciably assists the matching of N supply to crop N requirements [14]. Optical sensors are devices that provide rapid, effective, and nondestructive assessment of crop N status, in the field [3,15]. They enable frequent assessment throughout a crop, and assessment of spatial variability.
Amongst the proximal optical sensors, canopy reflectance sensors have two very positive features in that they can measure large areas of a crop and have on-the-go measurement capability [16].
Crop reflectance measurements can be used to assess N status of field crops [11]. These measurements are based on the differential reflection of wavelengths of radiation [3], which are absorbed and reflected by the crop in different proportions, depending on crop N status [15]. Generally, the light wavelengths used are in red, green, and near-infrared ranges [11]. More recently, the red-edge has been proposed to overcome the reported saturation of the red band [17,18]. Using reflectance data of different wavelengths, vegetation indices are calculated, which commonly combine reflectance data from 2-3 wavelengths [19].
Measurements of crop reflectance can be made with proximal sensors positioned relatively close to the canopy, from several centimeters to a few meters away [15]. Due to the field of view of the sensors, each individual measurement can integrate a large area of crop canopy [3,20]. Depending on the sensor, continuous measurements can be made as the sensor passes along the crop canopy ("on-the-go" measurement), thereby integrating large surface areas of crop canopy [15].
In order to use vegetation indices, calculated from canopy reflectance measurement, as a proxy of crop N status, calibration is required. A commonly-used approach is to determine the relationship between values of a given vegetation index and a measure of crop N status, such as the nitrogen nutrition index (NNI) [13,21]. NNI is calculated by dividing the actual crop N content by the critical crop N content [22,23], the latter being the lowest crop N content necessary for nonlimiting growth. Values of NNI equal to 1 indicate optimal N nutrition [24], and any deviation from 1 indicates excess N (i.e., NNI>1) or deficient N (i.e., NNI<1) crop status.
Numerous studies have reported that vegetation indices, obtained with canopy reflectance sensors, are strongly related to crop biomass and yield [11,25,26]. Appreciably, fewer studies have assessed the capability of vegetation indices, measured with proximal reflectance sensors, to assess crop N status [27]. Most studies have been conducted in cereal crops such as wheat [12,27,28] and rice [29,30]; very few with vegetable crops such as sweet pepper. To use vegetation indices as estimators of crop N status, it is necessary to derive a regression equation between the measured vegetation index (independent variable) and crop NNI (dependent variable) [31]. This procedure requires firstly fitting a regression equation between the vegetation index and crop NNI with a calibration dataset [27], and secondly, it requires validation of this regression equation with an independent, validation dataset.
Environmental problems associated with the high use of N fertilizer in vegetable production systems have been reported for diverse regions [5], such as southeastern (SE) Spain [32], SE United States [6], and China [7,33]. Greenhouse production systems are major sources of vegetables [34]. Within greenhouse-based vegetable production systems, sweet pepper is one of the most important vegetable crop [35]. In SE Spain, approximately 40,000 ha [36] of highly-concentrated greenhouses are used for intensive vegetable production; 30,000 ha are located in the Almería province. This system is characterized by high rates of N fertilizer and an excessive N supply [2,37] that are associated with nitrate contamination of underlying aquifers [32]. There is increasingly strong pressure to improve crop N management to reduce aquifer contamination from this vegetable production system. In Almeria, sweet pepper is one of the most important crops; each year, it is grown on 8000 ha [38].
Given the pressure to improve N management in greenhouse-based vegetable production [2,7] and that sweet pepper is a major crop, information is required of tools and sensors that inform of the N status of sweet pepper crops grown in greenhouses. Such tools will provide vital information of the adequacy of ongoing N management, enabling optimal N fertilizer use and ensuring less environmentally harmful N losses [3].
In the present work, eight vegetation indices, calculated from canopy reflectance measurements obtained with two different proximal sensors, were evaluated to estimate crop N status of sweet pepper. Firstly, calibration regression equations of each vegetation index to crop NNI were fitted. Secondly, these regression equations were subsequently validated using a different dataset. Thirdly, using the validated equations between vegetation indices and crop NNI, sufficiency values were derived for each vegetation index for optimal N nutrition, for the major phenological stages of sweet pepper crops.

Site and Experimental Design
Three experiments with sweet pepper (Capsicum annuum cv. Melchor) were carried out in a plastic greenhouse in Almería, southeast Spain. The greenhouses were located in the experimental station of the University of Almería (36 • 51 51" N, 2 • 16 56" W, 92 m altitude). The first crop was grown in 2014-2015, the second in 2016-2017, and the third in 2017-2018 (Table 1). All crops spanned a summer-winter cycle. Further details of the greenhouse are in Padilla et al. 2014 [39] and 2017 [21]. The crops were grown in an artificial layered soil, known as "enarenado", typical of commercial greenhouse crops in SE Spain [2]. The crops were established by transplanting 35-day old seedlings, in twin rows (0.8 m between twin rows and 1.2 m between twin rows) and 0.5 m distance between plants within each line, with a plant density of 2 plants m −2 . Each experimental plot measured 6 by 6 m, giving a total of 72 plants per replicate plot. There were three twin rows of plants, 6 m in length in each plot. The middle twin row was used for canopy reflectance measurements.
Water and fertilizers were applied combined through fertigation, by using an above-ground drip irrigation system. Each plant was planted close to an emitter. The fertigation was applied every two-three days, depending on crop demand. The three experiments consisted of a fully randomized block design, with five N treatments and four replicates per treatment. The N treatments were applied by fertigation by using different nutrient solutions with increasing N concentration. All other macro and micronutrients were applied in the nutrient solution to ensure they were not limiting. The treatments were: Very deficient N (N1), deficient N (N2), conventional N (N3), excessive N (N4), and very excessive N (N5) ( Table 1). N was applied mostly (90%) as nitrate (NO 3 − ), the rest as ammonium (NH 4 + ). The crop was physically supported which is typical for pepper production in this system.
Crop management followed local practices.

Canopy Reflectance with Optical Sensors
Two active proximal reflectance sensors were used to measure canopy reflectance information throughout the three crops. In the first crop, reflectance measurements were made weekly and in the second and third crops every two weeks. The sensors used were the GreenSeeker handheld sensor (Trimble Navigation Limited, Sunnyvale, CA, USA) and the Crop Circle ACS-470 (Holland Scientific Inc., Lincoln, NE, USA). The measurements were made by positioning both sensors vertically and parallel to the crop rows, so that the upper limit of the field of view was at the height of the most recently fully expanded leaf [15].
The GreenSeeker includes two light sources, visible (660 nm-red light) and near-infrared (NIR) (780 nm). This sensor measures the fraction of emitted lights reflected from the crop to calculate the vegetation index NDVI which is explained in Table 2. The GreenSeeker handheld sensor was positioned at 60 cm horizontal distance to the foliage; the field of view was an oval with a height of ≈ 25 cm. The measuring mode was the individual measurement ("one-shot"). For each date of measurement, eight marked plants were measured per replicate plot, and the mean value was determined. The Crop Circle ACS-470 used filters at 550 nm (green), 670 nm (red), 760 nm (near infrared; NIR), and 730 nm (red edge). The sensor was positioned at a 45 cm horizontal distance. The field of view was a rectangle of ≈ 26 (vertical) × 5 (horizontal) cm. The measurements were made in two separate passes. Each pass consisted of a 4 m transect in each line of plants in the middle twin row of each plot. In the first pass, green, red, and NIR filters were used; in the second pass, red edge, red, and NIR filters were used. Measurements were collected at a frequency of 10 readings per second. On-the-go measurements were made by walking at approximately 1.5 km h −1 . In total, 200 individual measurements were collected per plot. Data were stored in a portable GeoSCOUT GLS-400 data logger (Holland Scientific, Inc.). The vegetation indices shown in Table 2 were calculated from reflectance values of individual wavelengths.

Crop Sampling and NNI Determination
In each of the crops, periodical aboveground biomass samplings (approximately every 14 days) were made to determine dry matter (DM). For each replicate plot in each sampling, two complete plants were selected and removed. The dry weights of different components of the plants (stem, leaf, and fruit) were recorded by oven-drying until constant weight at 65 • C. In each replicate plot, the fruit production and pruned material were recorded throughout the crop in eight marked plants. Subsamples of dry material were ground prior to analysis of N content (%N) in a Dumas-type elemental analyzer (Rapid N, Elementar, Analysensysteme GmbH, Hanau, Germany). The amount of N was calculated by multiplying the %N by the dry matter mass of the corresponding component.
The NNI was calculated using the critical N curve derived for greenhouse-grown sweet pepper crop: Critical N = 4.71·DM −0.22 (Alejandra Rodríguez, University of Almeria, unpublished data). The NNI was calculated by dividing the N content measured in the crop by the critical N content. The NNI value for each reflectance measurement day was by interpolating DM and crop N content values between two consecutive biomass samplings [46].

Data Analysis
Data of reflectance measurements and NNI were grouped and analyzed for phenological stage. Four main phenological stages were considered for sweet pepper, according to de Souza et al. [47], as: (1) Vegetative, (2) flowering, (3) early fruit growth, and (4) harvest. The definition of the phenological stages is in de Souza et al. [47]. Within each phenological stage, several canopy reflectance measurements and biomass samplings were conducted. To integrate data of the various measurements within each phenological stage, integrated NNI and vegetation indices values were calculated, according to Lemaire and Gastal [24] and Padilla et al. [21], as: where D was the duration of the phenological stage, V was the value of NNI or vegetation index for each day of measurement, and ds was the duration between two successive measurements [47]. Predictive regression functions were evaluated to estimate NNI of sweet pepper for each of the eight vegetation indices assessed in the current work. For each phenological stage of the three crops; data of integrated vegetation indices and the corresponding integrated NNI were pooled. This created a single pooled data set of 60 data points for each vegetation index in each phenological stage considering the three crops together. The 60 data points, for each phenological stage were randomly separated into two groups: Forty data points (2/3 of total data) for the calibration dataset, and 20 data points (1/3 of total data) for the validation dataset. With the calibration dataset, simple linear regression analyses were conducted with the integrated vegetation index as the independent variable (x variable) and NNIi as the dependent variable (y variable). The software CurveExpert Professional®2.2.0 software (Daniel G. Hyams, MS, USA) was used. Validation of the equations that related each integrated vegetation index with NNIi, for each phenological stage, was then conducted with the validation dataset. Validation consisted of calculating the predicted NNIi from the calibration equation for each combination of vegetation index and phenological stage. Predicted NNIi values were then compared with the NNIi values of the validation dataset. Linear regression analysis was made between observed NNIi (independent variable) and predicted NNIi (dependent variable) and the root mean square error (RMSE) of the NNI estimation was determined. The RMSE was calculated as: where n is the number of samples, Ei is the estimated value of the relationship, and Oi is the observed value [48]. The performance of the different vegetation indices was evaluated according to Xin-feng et al., 2013 [49]; this procedure considers both the calibration and validation results. Coefficient of determination (R 2 ) and RMSE values of the linear regression of the calibration dataset, and the R 2 and RMSE values, absolute values of slope-1, and absolute values of intercept of the linear regression of the validation dataset were used [49]. Slope-1 is the absolute value of the slope after subtracting one from the slope of the linear regression. The use of this parameter effectively normalizes slope values and enabled ranking of all integrated vegetation indices from lowest to highest values.
The performance of each vegetation index was calculated (i) by sorting R 2 in decreasing order and RMSE in ascending order for the calibration and validation datasets separately, and (ii) the sorting of absolute slope-1 and absolute intercept values in ascending order for the validation dataset [49]. The best performing vegetation index was that which had the lowest sum of these six factors [49]. Additionally, the performance of the validation regression equation of the different vegetation indices, in each phenological stage, was assessed by comparing the relative error (RE) between observed and estimated NNIi values. The relative error was calculated as: where Oi is the average of observed values. Sufficiency values of each vegetation index, for each phenological stage, were calculated from the regression equations of the calibration datasets. The equations of calibration for each phenological stage were solved for NNI = 1, according to Lemaire et al. [23].

Phenological Relationships Between Integrated Vegetation Indices and Integrated NNI (NNIi), for Calibration Dataset
For the calibration data, the relationships between most of the integrated vegetation indices and NNIi, in each phenological stage, were highly significant (Table 3, Figure 1). In the vegetative stage, the coefficients of determination (R 2 ) of these relationships were generally low; averaged across all vegetation indices, the R 2 of the vegetative stage was 0.45 ± 0.05. In the vegetative stage, the R 2 ranged from 0.19 to 0.50 for most of the vegetation indices, except the GNDVIi which had a R 2 value of 0.63 (Table 3). In the flowering stage, the coefficients of determination were slightly higher than in the vegetative stage, with an average R 2 value for all vegetation indices of 0.52 ± 0.03 and a range from 0.38 to 0.65 (Table 3). The highest R 2 values were obtained in phenological stage corresponding to early fruit growth, where the average R 2 value across all vegetation indices was 0.71 ± 0.04, with a range from 0.52 to 0.84 (Table 3). The harvest stage had the lowest R 2 values of all phenological stages for all vegetation indices considered together, the average was 0.27 ± 0.02, with a range from 0.19 to 0.42 (Table 3).
Comparing the performance of different vegetation indices to estimate NNI throughout the crops, the integrated vegetation indices that were based on the green band (GNDVI and GVI) had higher and more consistent R 2 values in the first three phenological stages, which were the vegetative, flowering, and early fruit growth stages. The R 2 values for GNDVIi were 0.63, 0.65, and 0.62 for the vegetative, flowering, and early fruit growth stage, respectively. For GVIi, the R 2 values were 0.56, 0.60, and 0.63, respectively, for the same phenological stages ( Table 3). The R 2 values of RVIi, CIi, and CCIi vegetation indices were low (R 2 < 0.50) and very similar in the vegetative and flowering stages but increased in the early fruit growth stage (Table 3). For the rest of the integrated vegetation indices evaluated, the R 2 values increased from the vegetative to early fruit growth stages, being lowest in the harvest stage (Table 3). Sufficiency values of each integrated vegetation index, for each phenological stage, were calculated from the regression equations of the calibration datasets. The equations of calibration for each phenological stage were solved for NNI = 1. Table 3. Equations, coefficient of determination (R 2 ), and root mean square error (RMSE) of linear regressions between integrated vegetation indices (x variable) and integrated nitrogen nutrition index (NNIi, y variable) at different phenological stages, for calibration data (n = 40). Significance of regressions are indicated with asterisks close to R 2 values. ***, p < 0.001; **, p < 0.01). Abbreviations for vegetation indices are in Table 2 Table 3. Abbreviations for vegetation indices are in Table 2.

Validation of the Phenological Relationships Between Vegetation Indices and NNIi
Validation of the relationships established with the calibration dataset was made with an independent and different dataset. For all of the vegetation indices analyzed, the vegetative stage had the worst validation results. In this stage, the average R 2 and RMSE values for all indices were 0.46 ± 0.05 and 0.123 ± 0.006, respectively ( Table 4). The validation results, for all vegetation indices, improved in the flowering and early fruit growth phenological stages. In these stages, the average R 2 and RMSE values for all indices were 0.63 ± 0.04 and 0.127 ± 0.006, for the flowering stage, and were 0.87 ± 0.02 and 0.120 ± 0.008, for the fruit growth stage, respectively (Table 4). In the harvest stage, validation results were intermediate (Table 4), with average R 2 and RMSE values for all indices of 0.59 ± 0.09 and 0.125 ± 0.002, respectively. Table 4. Results of validation analysis for each vegetation index at different phenological stages. Equations, coefficient of determination (R 2 ), and root mean square error (RMSE) of linear regression between observed NNIi values (x variable) and predicted NNIi values (y variable), for validation dataset (n = 20). Significance of regressions are indicated with asterisks close to R 2 values. ***, p < 0.001; **, p < 0.01; ns: Not significant. Abbreviations for vegetation indices are in Table 2 Generally, for the vegetative stage, the slope of linear regression between observed and predicted NNIi values was appreciably different to one for all of the vegetation indices evaluated; the average slope value for all indices was 0.513 ± 0.057 (Table 4). The exception was the GNDVIi that had a slope of 0.806. Compared to the 1:1 line, there was a tendency for all vegetation indices except GNDVI to overestimate NNI values for NNI values < 0.9, and a tendency to underestimate NNI, for NNI values >0.9 (Figure 2, green circles). In the flowering stage, the slopes of the regression between observed and predicted NNIi values were closer to one for all of the vegetation indices evaluated (average value of 0.756± 0.049), and particularly so for NDVI (0.946) and GNDVI (0.905) ( Table 4). Compared to the 1:1 line, all of the vegetation indices except for NDVI and GNDVI tended to overestimate NNI values at NNI values <1, and underestimate NNI at NNI values >1 (Figure 2, red triangles). In the early fruit growth stage, slopes of linear regression between observed and predicted NNIi values were slightly above 1 for all of the vegetation indices evaluated (average value of 1.282 ± 0.036) ( Table 4). Compared to the 1:1 line, all vegetation indices underestimated NNI values at the whole range of NNI observed in the early fruit growth stage (Figure 2, blue squares). In the harvest stage, the slopes of the regression between observed and predicted NNIi values were close to 0.5 for all vegetation indices evaluated (average value of 0.433± 0.018) ( Table 4). Compared to the 1:1 line, all vegetation indices overestimated NNI values, at NNI values < 0.9, and underestimate NNI, at NNI values > 0.9 during the harvest stage ( Figure 2, grey diamonds).  Table 4. Abbreviations for vegetation indices are in Table 2.
The relative error (RE) of the validation analysis for all vegetation indices evaluated in each phenological stage are presented in Figure 3. For the vegetative, flowering, and early fruit growth stages, RE ranged from 8% to 17%, with average RE values, for all indices, of 12.9% ± 0.59%, 13.2% ± 0.66% and 12.4% ± 0.78%, respectively. In the harvest stage, the RE ranged from 12% to 15%, with an average value of 13.5% ± 0.21% for all indices. The vegetation indices GVI and GNDVI had consistently lower RE values in most of the phenological stages (average RE values across phenological stages of 10.7% ± 1.03% and 11.3% ± 0.81%, respectively), followed by the RVI index (averaged RE values across phenological stages of 12.4% ± 0.40%). The GVI had the lowest RE in the vegetative, flowering, and early fruit growth stages (Figure 3). In contrast, the RENDVI index was the vegetation index with highest RE values throughout the four phenological stages (average value of 14.5% ± 1.09%).  Table 2.

Performance of Vegetation Indices
The classification of vegetation indices based on R 2 and RMSE of linear regression analysis of the calibration and validation datasets, and on the slope and intercept values of linear regressions of the validation dataset, showed that six (NDVI, RVI, RENDVI, CI, CCCI, and MTCI) of the nine vegetation indices evaluated had their best performance in the early fruit growth stage (Table 5). For NDVI, RVI, CI, CCCI, and MTCI, the flowering stage, was the phenological stage in which the second best results were obtained, for these indices, which were only slightly inferior to those in the early fruit growth stage. The RENDVI index was the exception, where the second best performance was in the vegetative stage (Table 5). For the three remaining vegetation indices (NDVI GS measured with GreenSeeker, GNDVI, and GVI), the best performance was in the flowering stage, followed by the early fruit growth stage. The worst performance for most of the vegetation indices occurred in the harvest stage, followed by the vegetative stage (Table 5). Table 5. Ranking of best performing phenological stage for each vegetation index. Performance was evaluated using R 2 and RMSE of linear regression of calibration and validation datasets, and slope and intercept of linear regression of validation dataset. Numbers in brackets show the performance of each phenological stage. The best performance is the one which has the lowest value. Performance  NDVI GS  NDVI  GNDVI  RVI  GVI  RENDVI  CI  CCCI  MTCI   1st Flowering (10) Early fruit growth (10) Flowering (9) Early fruit growth (8) Flowering (10) Early fruit growth (10) Early fruit growth (11) Early fruit growth (6) Early fruit growth (7) Early fruit growth (10) 2nd

Best
Early fruit growth (14) Flowering (12) Early fruit growth (14) Flowering (14) Vegetative (17) Vegetative (11) Flowering (13) Flowering (16) Flowering (14) 3rd Harvest (16) Vegetative (18) Vegetative (15) Vegetative (15) Harvest (23) Flowering (19) Vegetative (19) Harvest (18) Harvest (17) 4th Vegetative (20) Harvest (20) Harvest (22) Harvest (23) Harvest (20) Harvest (20) Vegetative (20) Vegetative (22) For each phenological stage, the performance of the vegetation indices was compared to one another in Table 6. In the vegetative, flowering, and harvest stages, the best performing index was GNDVI. In early fruit growth stage, the best performing index was GVI. The performance of various indices, in this ranking was not constant in the different stages (Table 6). For example, in the vegetative stage, the second and third best performing vegetation indices were GVI and RENDVI, respectively, but RENDVI was one of the worst performing indices in the other three stages. Similar results were obtained for MTCI, which was the second-best performing index in the early fruit growth stage but was amongst the last positions in the other three stages. Overall, considering the four stages together, the best performing vegetation index was GNDVI, followed by GVI. The vegetation indices that performed worse were CCCI and MTCI (Table 6). Table 6. Ranking of best performing indices for each phenological stage. Performance was evaluated using R 2 and RMSE of linear regression of calibration and validation datasets, and slope and intercept of linear regression of validation dataset. Numbers in brackets show the performance of each index. The best performance index is the one which has the lowest value.

Sufficiency Values of Vegetation Indices
Sufficiency values of each vegetation index for each phenological stage were derived from the calibration equations. Figure 4 shows the dynamics of sufficiency values of each vegetation index throughout the four phenological stages. The sufficiency values of the best performing vegetation indices ranged 0.64-0.76 for GNDVI, and 4.93-7.36 for GVI. The largest difference between sufficiency values, for most of the vegetation indices evaluated, was between vegetative and flowering stages. On average, the relative increase in sufficiency values from the vegetative to the flowering stage was approximately 10% for NDVI, measured both with Crop Circle and GreenSeeker sensors, and for GNDVI, RENDVI, and CI. There were much larger relative increases for RVI and GVI, which were 34% and 25% higher in the flowering stage compared to the vegetative stage. In contrast, the smallest differences in sufficiency values between these two stages were for the CCCI and MTCI indices, with values of approximately 5%. Generally, the sufficiency values for the early fruit growth stage were similar to those for the flowering and harvest stages; except for GVI, CCCI, and MTCI which were on average 13% higher in the early fruit growth stage compared to the flowering stage.

Discussion
In sweet pepper, the R 2 and RMSE of linear regressions between vegetation indices and crop NNI were variable between phenological stages throughout the crop and between the eight vegetation indices evaluated. The best performance of vegetation indices for estimation of crop NNIi, in terms of R 2 and RMSE values, was in the early fruit growth stage. Using these criteria, the worst performance for estimating NNIi was in the harvest stage, followed by the vegetative stage, for most of the vegetation indices evaluated. Similar variability of performance of vegetation indices, throughout a crop, was reported by Hatfield and Prueger [50], in maize and soybean. In that research, the relative performance of different vegetation indices for estimating leaf chlorophyll content differed as the growing season progressed. Similarly, Yu et al. [29] found in rice that some red edge-band based vegetation indices had better performance to estimate plant N concentration after the heading stage. In wheat, performance of vegetation indices varied across growth stages, with better results after flowering was reported by Li et al. [51].
According to the analysis of R 2 and RMSE values of linear relationships between vegetation indices and NNIi, for the calibration dataset, vegetation indices based on reflectance of the green band (i.e., GNDVI and GVI) had consistently higher R 2 values and lower RMSE values throughout most of the phenological stages of the crop (average R 2 value of 0.62 in the three first phenological stages). These results indicate that vegetation indices based on reflectance of the green band estimated crop NNIi with more accuracy than the rest of vegetation indices evaluated. In rice, Cao et al. [30] also found that the best indices to estimate NNI, in the different parts of the crop cycle, were mainly green band based indices. In the present study, there was an exception in the early fruit growth stage, where vegetation indices based on reflectance of red edge band (i.e., RENDVI, CI, CCCI, and MTCI) had higher R 2 values (average R 2 value across these four indices of 0.83) than green band based vegetation indices such as GNDVI and GVI. These results are consistent with Yu et al. [29], who found that the red edge based vegetation indices were more sensitive to plant N concentration particularly after the heading stage in rice, whereas green based vegetation indices were more sensitive to plant N concentration in the rest of stages.
Vegetation indices based on reflectance in the green band and in the red edge band are very sensitive to leaf and crop greenness [41,52,53]. They have been preferred over red band based reflectance indices as indicators of crop N status [11,21,25] because of higher sensitivity, particularly of the red edge band at high chlorophyll levels contents [43]. In the present study, vegetation indices based on reflectance in the green band were more sensitive to estimate crop N status than vegetation indices based on the red edge band in all phenological stages, except for the early fruit growth stage when green pepper fruits developed and enlarged. It is possible that the abundance of green tissues in this stage, formed by green pepper fruits and leaves, caused some degree of saturation of the green band but not of the red edge band.
To validate the calibration linear regression equations (derived from the calibration data set) that estimated crop NNIi values from integrated vegetation index measurements, the same calibration linear regression equations were used to estimate NNIi values from the validation data set for each vegetation index. The NNIi values estimated, with this procedure, were compared with the integrated measured NNI values, by linear regression analysis. For most of the vegetation indices evaluated, there was a deficient validation of the regression equations in the vegetative and harvest stages, and more successful validation in the flowering and early fruit growth stages. This interpretation is based on the slope of linear regression between observed and predicted NNIi values, and on the calculated relative error.
In the vegetative and harvest stages, the slopes of linear regression diverged appreciably from one and the relative errors were very different to 0% which would represent perfect validation of regression equations [54][55][56]. In contrast, in the flowering and early fruit growth stages, all indices had slopes and relative errors closer to 1 and 0%, respectively. The reason for this poor calibration in the vegetative and harvest stages may be associated to characteristics of the crop canopy in these two stages. In the vegetative stage, the plants are small and have low foliage density, which could affect reflectance measurements by the inclusion of background noise [11,15]. Johansen and Tømmervik [57] and Wang et al. [58] reported a lack of precision with NDVI until the plant canopy achieved adequate coverage. In the harvest stage, mottling and discoloration of older leaves, because of crop age, can affect reflectance measurements, as has been reported for cucumber [46].
Validation results (slopes of linear regression between observed and predicted NNIs values, and relative errors) were consistent with the evaluation conducted taking into account results of regression of both calibration and validation datasets [49]. There was a clear tendency for better performance of most vegetation indices in the early fruit growth stage, followed by the flowering stage. The performance of most vegetation indices was worse in the vegetative and harvest stages, most likely due to the characteristics of the crop canopy in these two stages, insufficient foliage density in the vegetative stage, and aging foliage in the harvest stage, as discussed previously.
Analyzing the performance of each vegetation index to estimate NNI within each phenological stage, GNDVI was the best index in three of four phenological stages (vegetative, flowering, and harvest), and GVI was the best performing vegetation index in the other stage (early fruit growth). This is in agreement with Padilla et al. [21] and de Souza et al., [59] who reported these two indices (i.e., GNDVI and GVI) to be more strongly related to NNI in cucumber. Likewise, green vegetation indices of processing tomato were more strongly related to leaf N content than red vegetation indices [60]. Very similar results were also obtained in broccoli [61].
The RENDVI and CCCI were the worst performing indices. The poor performance of CCCI index, in the present study, in the vegetative stage is inconsistent with the results obtained in maize by Li et al. [62]. Li et al. [62] reported that CCCI successfully excluded the effect of soil reflectance when crop cover was low. The different results with CCCI in our study and that of Li et al. [62] may be due to the measurement procedures and the structure of the different crops. In the current study, measurements were made from the side of the crop, and in maize from above. Moreover, the pepper crops in the current study were vertically supported. It is possible that CCCI was influenced by the small areas of empty background, between adjacent pepper plants, that were exposed by vertically supporting the plants.
Comparing the sufficiency values obtained, for maximum dry matter production between the different phenological stages, it was possible to derive a unique sufficiency value for the complete crop cycle for CCCI of 0.61 and for MTCI of 1.57 because the relative differences between phenological stages were small (Figure 4). For the other indices, such as RVI and GVI, the relative difference between the sufficiency value of vegetative and flowering stages were too large (around 30%) to be able to derive a unique sufficiency value for the complete crop cycle. For the rest of the indices evaluated (NDVIGS, NDVI, GNDVI, RENDVI, and CI), the relative difference between vegetative and flowering stages was approximately 10%. In the current work, it was considered not possible to calculate a unique sufficiency value for the whole crop cycle for sweet pepper for the NDVI, GNDVI, RVI, GVI, and RENDVI indices. However, in cucumber, it was possible to calculate a unique sufficiency value for the entire crop cycle because of the relative constancy of sufficiency values throughout the cycle [21]. Overall, the sufficiency values derived for sweet pepper were higher than those derived for cucumber [21], for equivalent indices. This difference may be due to the relatively high chlorophyll content and greenness of sweet pepper crops compared to other vegetable and cereal species [63].

Conclusions
The present work evaluated the capacity of different vegetation indices to estimate crop NNI in the vegetative, flowering, early fruit growth, and harvest phenological stages of sweet pepper. There were differences in the performance of the indices within individual phenological stages and between stages. The best performance of all indices was in the early fruit growth and the flowering stages. The best performing indices to assess crop N in sweet pepper were the green band based indices GNDVI and GVI which had the best results for all phenological stages.