In-Flight Retrieval of SCIAMACHY Instrument Spectral Response Function

Abstract: The instrument Spectral Response Function (ISRF) has a strong impact on spectral calibration and the atmospheric trace gases retrievals. An accurate knowledge or a fine characterization of the ISRF shape and its FWHM (Full width at half maximum) as well as its temporal behavior is therefore crucial. Designing a strategy for the characterization of the ISRF both on ground and in-flight is critical for future missions, such as the spectral imagers in the Copernicus program. We developed an algorithm to retrieve the instrument ISRF in-flight. Our method uses solar measurements taken in-flight by the instrument to fit a parameterized ISRF from on ground based calibration, and then retrieves the shape and FWHM of the actual in-flight ISRF. With such a strategy, one would be able to derive and monitor the ISRF during the commissioning and operation of spectrometer imager missions. We applied our method to retrieve the SCIAMACHY instrument ISRF in its different channels. We compared the retrieved ones with the on ground estimated ones. Besides some peculiarities found in SCIAMACHY channel 8, the ISRF results in other channels were relatively consistent and stable over time in most cases.


Introduction
One of the key components that is needed to perform spectral calibration and to proces Level 1 and Level 2 data is the instrument spectral response function (ISRF) which is sometimes referred to as a slit function.Additionally, the lower the accuracy of the ISRF is, the higher the error on the retrieval is, i.e., larger systematic errors of the retrieved quantities.As a matter of fact, one of the origins of the instrument introduced errors is insufficient knowledge of the ISRF during the mission.It is one of the most important parameters for level 2 retrieval.It is additionally crucial to probe any variations over time or during different observing cases (e.g., [1]).Some studies have in fact investigated the ISRF impact on data retrievals and calibration for different instruments in-flight, like GOME-2 (e.g., [2,3]).Furthermore, another study [4] reported that the retrieved in-flight ISRF in the UV improved the level 1 data in the period of 2003-2011.Requirements for achieving high accuracies of retrieved ISRF(s) in-flight and for probing its stability over time are becoming common for the next generation instruments.We developed a method using mathematical functions and algorithms for the retrieval of the instrument ISRF in-flight.The motivation was to calculate it with the best accuracy possible on the fly.We applied our method for the retrieval of SCIAMACHY (SCanning Imaging Absorption spectroMeter for Atmospheric CHartographY) instrument's slit functions in-flight.We developed a model that used SCIAMACHY solar measurements to obtain its ISRF on a daily basis over its lifetime.
SCIAMACHY is a scanning nadir and limb spectrometer covering the wavelength range from 212 nm to 2386 nm in eight channels (Table 1).It is a joint project of Germany, the Netherlands and Belgium and was launched in February 2002 on the ENVISAT platform.Designed for a lifetime of five years, it performed measurements until April 2012, when the contact to the satellite was lost.The project is now in phase F. SCIAMACHY was designed to measure column densities and vertical profiles of trace gas species in the mesosphere, in the stratosphere and in the troposphere [5].It can detect O 3 , H 2 CO, SO 2 , BrO, OClO, NO 2 , H 2 O, CO, CO 2 , CH 4 , N 2 O, O 2 , and can provide information about aerosols and clouds.In addition to the spectrally resolved measurements of the radiance reflected from the Earth's atmosphere, the polarization of the incoming light is measured with seven broadband sensors.

Background
We developed and ran the SCIAMACHY operational processor to retrieve both Level 1 and Level 2 data [6].We extracted solar measurements (Figure 1) from the processed Level 1b data and used it as an input for our ISRF retrieval algorithm.Ground based calibration measurement campaign OPTEC5 [7] provided an estimation of the ISRF shape in each of the SCIAMACHY science channels [8].Furthermore, from the commissioning estimations, various mathematical functions were recommended (Table 2).We used both of these estimations as starting inputs for our simulations.Like our presented analysis, several studies had already tested various parameter functions to constrain the ISRF shape of different instruments like OMI [9] and OCO-2 [10].

Fitting the ISRF
We modeled the solar measurements by convolving highly resolved reference spectra [11,12] of the sun with a simulated ISRF function.The convolution is expressed like the following: where S is the solar spectrum.After the convolution, the resulting spectra are pixel sampled to match SCIAMACHY wavelength axis in order to fit the modeled S f it against a SCIAMACHY solar measurement S(λ), while the ISRF is the free component to be estimated.We retained the ISRF function that provided the best fit of the solar measurements (Equation ( 1)) in each channel.Some regularization is usually necessary to solve such convolution type integrals, in our case the regularization is applied by assuming a specific parameter function (Table 2) for the ISRF in each case.For the fitting, we used the χ 2 minimization method to retrieve the in-flight ISRF from each individual measurement (orbit) separately.This method is also sometimes used for the retrieval of the ISRF [13].We calculated the χ 2 of a fitted solar measurement from Equation (1), like: where N is the number of pixels per measurement in each tested orbit.The individual measurement per pixel is defined by S(λ i ) and its corresponding error by σ i .The ISRF free-parameters (in Equations ( 3)-( 7)) are defined by x M , where M is the number of parameters per function.For each set of parameters x M , we normalized the calculated ISRF function, i.e., equality to the unit of its integral, before simulating the measurement S f it .Since the number of free parameters is small, we used the brute force search method, as it was more appropriate in our case and more reliable since it excludes local minima that might affect our results.We imposed a few constraints on each free parameter x by letting them vary between pre-defined values only, in order to be compatible with the physical and technical specifications of the existing estimated ISRFs of SCIAMCHY.Overall, the functions (Equations (3), (4), ( 6), (7)) width were initially constrained within the range 0.01× FWHM and 5× FWHM (Table 1) depending on the channel, while their other free parameters were constrained based on assumptions from our knowledge of the instrument, as described below.We then searched the M-dimension parameter space grid with the highest reasonable resolution.These conditions allowed us to find the global minimum in this space.In each point of the grid, we computed the χ 2 (Equation ( 2)).The best fit parameters occur at the grid point of the lowest χ 2 , and thus yielding the best fit ISRF function.We additionally deduced the corresponding confidence level of each fitted measurement.
The ISRF is parameterized by a mathematical function that we modeled for each tested channel.Thus in each channel, we selected the mathematical function that provided the best fit of all fitted solar lines (at various wavelengths) simultaneously.We chose the parameter functions based on the recommended ones from ground based and commissioning estimations (Table 2).Accordingly, we tested in channels 1 to 6 a Gaussian (Equation ( 3)) and a Hyperbolic function (Equation ( 4)).The Gaussian is defined in two wavelength regions like the following: ( where λ 0 is the central wavelength of the ISRF.C 1 was left as a free parameter.On the other hand, and despite being a free parameter, C 2 depended on the value of C 1 .In fact, C 2 was constrained within 0.98 ×C 1 and 1.02 ×C 1 range for each C 1 value.The relation between these two parameters allowed the tracking of any asymmetry of the simulated ISRF function, by up to 2%.The hyperbolic function is defined like where the FWHM is determined by, but not equal to c.Following the existing estimations and recommendations in channel 8, we tested both a Voigt (Equation ( 5)) and a compound hyperbolic function (Equation (7)) where G(λ) is a Gaussian function (Equation (3)) and L(λ) is a Lorentzian function as defined below (Equation ( 6)): where the FWHM of the Lorentzian function is 2 × γ.We defined the compound hyperbolic function like: In the simulation of each solar spectrum, we normalized the ISRF.We consequently fixed the value of a = 1 in Equations ( 3)-( 5).We similarly fixed a 1 = 1 in Equation ( 7), while a 2 was left as a free parameter, but yet constrained within 0-10.To take into account any shifting impact on the ISRF, we left λ 0 [nm] as a free parameter, and constrained it within −5% and 5% of the pixel size in each channel.This allowed us to retrieve the ISRF in-flight whether it was slightly shifted or not.

SCIAMACHY Solar Measurements
We used all SCIAMACHY available daily solar measurements from Level 1b data taken over the entire lifetime of the mission, but excluded decontamination and anomaly orbits.
In channel 7, the measurements are contaminated by straylight from a light leak.The stray light prevents any retrieval of atmospheric parameters from this channel.Therefore we excluded this channel from our study.
We wanted to deduce the ISRF in each channel and additionally investigate its FWHM variation across the channel.In order to test that, we tried to retrieve the ISRF at different wavelengths within the same channel.We therefore needed to investigate as many solar lines or spectral structures as possible at various wavelengths in each channel.The selection of the lines was however challenging due to a few constraints.We needed strong lines from the theoretical spectra that should also be spectrally resolved by SCIAMACHY.Additionally, the measured line should cover enough pixels within the line's spectral range (a few nanometers) to avoid an over-parameterization of the model and also be resolved to fit a relatively complex curve.These are key points in order to perform a reliable and statistically meaningful fitting.
We identified a few solar lines per channel at different wavelengths across the channel from the reference spectra.However, for the fitting we only used the ones that were well resolved by SCIAMACHY (Table 3).Furthermore, channels 6 and 8 contained a large number of dead and bad pixels [14] which reduced significantly the number of usable pixels for the fitting.This has been detected early in the mission and it increased over time.Channel 8 in particular lacked the necessary number of pixels covering our identified solar lines across the entire channel.This was an additional limiting factor in this study.
Table 3. Identified solar lines for the fitting in different SCIAMACHY channels.The lines that were not resolved were not fitted.Column 3 indicates whether a line had acceptable fits for our analysis. (1)Channel 8 line was an exception as this was the only one we could identify, which we used for testing.

Line ID λ Range (nm) Resolved Acceptable Fit
Channel 1  (1)   In Table 3, we present the identified solar lines and indicate whether they were resolved by SCIAMACHY.Ultimately for each channel we used and presented the results that were obtained with spectrally resolved available solar lines.Channel 8 had one line only which we used despite it being only partially resolved.This allowed us to test our algorithm in this channel and probe the relative variation of its ISRF over time.

Results
For comparison, Figure 2 shows SCIAMACHY pixel masks per channel over the entire lifetime of the mission.We only show channels 1, 2, 6, and 8 for comparison between the UV/VIS and the SWIR bands.In channels 1 and 2, most pixels are good, as highlighted in green, over most of the mission lifetime.The vertical stripes that occur every several months correspond to the decontamination and the anomaly orbits, which we excluded in our work.On the other hand, in the infrared channels 6 and 8, a large number of pixels were flagged as bad/dead over an important part of the mission lifetime.A pixel is flagged as bad based on 11 criteria derived from different in-flight measurements.Typically a bad flag means the pixels had no signal (disconnected), were too noisy, or had excessively large leakage current.The reason for bad pixels is a substrate/light-detecting layer mismatch and induced damages because of proton impact [14].We fitted each daily measurement (orbit) of the resolved lines in Table 3.In each channel, we retained the parameter function (Equations ( 3)-( 5) and ( 7) depending on the channel) that overall (over the mission lifetime) provided the best fits of all available lines concurrently.The deduced fits of the measured solar lines that had <50% confidence levels over most of the mission lifetime were not used for our analysis (Table 4), e.g., line 2 in channel 1 (Table 3) was not analyzed in this study.In order to investigate the stability of the ISRF over time, we calculated the FWHM, or width at the half of their maximum values, of the retrieved ISRFs between 2002 and 2012.We could therefore probe its variation on a daily basis.
Figures 3-9 show examples of the corresponding SCIAMACHY measurements in orbit 8884 including their modeled counterparts, as well as their relative fitting residuals.The theoretical highly resolved corresponding solar lines are shown to verify the consistency between the measured and the theoretical spectra.The corresponding deduced ISRFs are shown in Figure 10.3-9, in SCIAMACHY channels.The corresponding function shape are shown in the y-axis label (see also Table 4).

Figure 10.
Examples of retrieved ISRF functions from our solar line fittings, in Figures 3-9, in SCIAMACHY channels.The corresponding function shape are shown in the y-axis label (see also Table 4).
Column 2 of Table 4 shows the retained mathematical functions per channel from our results.Table 4 (column 4) presents the time averaged FWHM of the ISRF(s) over the mission lifetime, in all channels and for each fitted spectral line.The temporal variation of the FWHM(s) are shown in Figures 11-18.4).4).

The ISRF in the UV/VIS Channels
The corresponding  show stable FWHMs in time, within the error bars, except in channel 1.In this channel, the FWHM increases not linearly beyond the error bars with the time.Its value increases from 0.23 nm to reach 0.256 nm in 2008, where it stays until mid 2009, and then drops down to 0.252 nm and stays there until the end of the mission in 2012.Furthermore, its average value over the life time of the mission is 0.258 ± 0.005 nm.The other UV/VIS channels (channels 2-5) do however not show such a variation or trend.They are pretty stable over time within the error bars.Furthermore, the observed sharp decrease of the optical throughput in channels 1 & 2 [7] does not provide an obvious explanation as to why the FWHM varies over time in channel 1, since we did not see a similar trend in channel 2. This perhaps would still need a further investigation.
UV/VIS channels have an averaged FWHMs that are consistent within a few percents with the on ground estimated ones (Table 1), except channel 4 where it is a little over 10% higher.In our simulation, we found that channel 4's ISRF is shifted all the time by at least 1 20 of a pixel, which is about 0.01 nm.The signal to noise ratios in all the UV/VIS channels were very similar as well as the ISRF resolution, or the average number of pixels per FWHM.Therefore, these two properties do not appear to be a reason for the temporal variation in Channel 1 and the particularity of Channel 4.However, we only had one line per channel in both and could therefore not make a final statement about their behavior or deviation.Further investigations might be needed to consolidate these results.This could include fitting SCIAMACHY non-solar spectra measurements at these wavelengths.
In channels 2, 3 and 5, we had a few measured lines per channel, and thus a few retrieved ISRF(s).The retrieved FWHMs in channel 2 increased on average from one region around ∼330 nm to another ∼390 nm by up to 5%.However, this does not indicate a trend within the channel.In any case, more lines in this channel, or at least between 330 and 390 nm, would have been necessary to better analyze any possible variation within the channel.Channel 5 results show a slight trend where the average FWHM increases by 2% as a function of the wavelength beyond the error bars.Since we only covered the region from 850 to 885 nm, which is only ∼10% of the channel spectral range, we cannot confirm a possible trend within this channel.Furthermore, in channel 3 the averaged FWHM of the measured lines were consistent with each other within the error bars.

The ISRF in the NIR Channels
As shown in Figure 2, channel 6 and 8 pixels were mostly flagged as bad/dead with a very little amount of usable or available pixels or wavelengths, highlighted in green in Figure 2. Additionally, the signal to noise ratios in these channels were significantly lower than in the UV/VIS channels.We still identified a few solar lines that we tried in this analysis (Table 3).Although SCIAMACHY did not resolve the line in channel 8 (Figure 9), we fitted the measurement to basically test our ISRF retrieval method in this channel and to probe the relative temporal variation of its FWHM.
Channel 6 FWHM are on average relatively stable in the long term (Figure 16).There were no trends detected like in channel 1.However, line 1 of channel 6 showed a seasonal-like variation beyond the error bars.We could distinguish eight distinct temporal periods between 2003 and 2012.In each, the FWHM varied by 0.6 nm, nevertheless the average (Table 4) was overall constant and very consistent with the ground based estimated value (Table 1).A comparison to the periodical variation of the temperature did not show any correlation [6].However, in both cases the periods duration are close to a year.Line 2 at longer wavelength in the same channel and close to the edge of channel 6+ is a lot more constant within the error bars (Figure 16), yet its value diverges from the on ground one.The difference of the two lines results in a same channel made their interpretation more intriguing, and thus may require further investigation.
Channel 8 (Figure 17) shows a stable, within the error bars, FWHM up until December 2008, nevertheless it is over 20% higher than the on ground value.Afterwards, between late 2008 and mid 2009, the FWHM becomes very noisy before jumping into a larger value that is almost twice the on ground known FWHM, where it stays until the end of the mission.As a matter of fact, the sudden variations of the FWHM started before the decontamination on 19-December 2008/3-January-2009.Besides, channel 7 & 8 were peculiar cases as the detectors were operated at the lowest temperatures.The throughput was in the beginning hampered by the growth of an ice layer in the optical light path over time.In fact, immediately after the detectors were cooled to operating temperatures, the ice layer began accumulating-and not uniformly over the light detecting area [16]-and the average throughput decreased rapidly (Figure 18), and thus the detector temperature increased.This can be seen in the behavior of the corresponding temperature curve (Figure 19).The ISRF does also depend on the structure (microscopic and macroscopic) of the ice layer or its smoothness.The ice does not only absorb light but also scatters photons, which ultimately fall on different parts of the detector.This generally leads to a widening of the ISRF and an increase of the intensity in the outer wings.This generated a scattering of the light and likely the widening of the ISRF of channel 8. Due to the particularity of channel 8 including the fact that the fitted line was only partially resolved does not yield an absolute result regarding the ISRF in this channel.

Conclusions
We developed a method using mathematical algorithms to recover the ISRF function (sometimes referred to as a slit function) of space spectrometers and imagers in-flight.Our method was used to retrieve the ISRF(s) of SCIAMACHY instrument in seven channels 1-6 and 8, by using its daily solar measurements independently of each other and over its entire lifetime.By fitting these measurements with an algorithm using the brute force search method, we retrieved daily in-flight ISRFs in each tested channel.We lastly compared our results to the on ground defined ones and probed the ISRF's width behavior over time.
The FWHM of the retrieved ISRF were consistent within a few percents (up to about 10%) with their pre-defined values from ground based estimation in channels 1-6.They were stable over time in the UV/VIS channels 2-5.Channel 1 showed however a long term temporal trend revealing a non linear increase of the FWHM value by 10% in the first seven years of the mission.In the SWIR channel 6, the deduced ISRF's FWHM were on average stable over the entire lifetime of the mission.Nevertheless, despite being stable on average, the ISRF from one of the spectral regions in channel 6 showed a seasonal-like variation of its FWHM by 40%, which is hard to interpret as this was not observed in the second retrieved ISRF's within the same channel.The SWIR Channel 8's ISRF FWHM values were on the other hand very different than the ground based estimation, and it additionally showed a sudden change of its value around year 2009.This was due to the known peculiarity of this SWIR channel mainly because of non-uniform ice accumulation on its detector.
In this study we did not see any particular variation or trend of the FWHM as a function of wavelengths across the channels that had a few solar lines.Channel 5 showed a slight increase by up to 2% of the FWHM as a function of the wavelength.However, we could only analyze a very small spectral region of this channel.Thus, one cannot settle with these results nor fully confirm them, as there were not enough measured and also resolved solar lines per channel for the purpose of this analysis.
Ultimately this approach and algorithms retrieved SCIAMACHY's in-flight ISRF using its solar measurements.It also showed that the signal-to-noise ratio was not a limiting factor in the retrieval.However, the spectral resolution has been a key factor to perform and endorse the retrieval results.Designing a strategy for the characterization of the ISRF both on ground and in-flight is in any case important for spectral imager missions and their data processing; our algorithm could eventually be tested for that purpose.

1 )Figure 1 .
Figure 1.Solar spectrum measured by SCIAMACHY on 2 August 2002.The different colors represent its different channels from 1 to 8.

Figure 2 .
Figure 2. The figures display the distribution of SCIAMACHY 1024-pixel per channel for channels 1 & 2 (top) and channels 6 & 8 (bottom) over the mission lifetime (2002-2012).Good pixels are shown in green and the flagged bad/dead ones in red.The figures are shown for comparison between the UV/VIS channels 1 & 2 and the SWIR channels 6 & 8.

Figure 3 .
Figure 3. Example of a fitted solar line in Channel 1 (left).This is Line 1 from Table3and is measured in orbit 8884.SCIAMACHY measurement is shown in red, our model in green, and the theoretical very highly resolved one in black.The corresponding retrieved ISRF is a Hyperbolic function with a FWHM 0.24 nm.The fitting residuals (right) corresponding rms 0.023.

Figure 4 .Figure 5 .
Figure 4. Similar to Figure 3 but for Channel 2 and Line 3. The corresponding retrieved ISRF is a Hyperbolic function with a FWHM 0.26 nm.The residuals (bottom graph) rms 0.019.

Figure 6 .
Figure 6.Similar to Figure 3 but for Channel 4 and Line 1.The corresponding retrieved ISRF is a Hyperbolic function with a FWHM 0.43 nm.The residuals (right) rms 0.012.This particular line revealed an exceptional shift of the ISRF compared to other channels.

Figure 7 .
Figure 7. Similar to Figure 3 but for Channel 5 and Line 1.The corresponding retrieved ISRF is a Hyperbolic function with a FWHM 0.57 nm.The residuals (bottom graph) rms 0.005.

Figure 8 .
Figure 8. Similar to Figure 3 but for Channel 6 and Line 1.The theoretical spectrum (top) black curve was lowered here on purpose, it would otherwise hide the model and measurement curves.The corresponding retrieved ISRF is a Hyperbolic function with a FWHM 1.47 nm.The residuals (bottom graph) rms 0.01.

Figure 9 .Figure 10 .
Figure 9. Similar to Figure 3 but for Channel 8 and Line 1.The corresponding retrieved ISRF is a Compound Hyperbolic function with a FWHM 0.31 nm.The residuals (right) rms 0.02.Version February 20, 2018 submitted to Remote Sens. 15 of 23

Figure 11 .
Figure11.The ISRF's FWHM, in nm, in Channel 1 as a function of time over the entire lifetime of SCIAMACHY.The marks (black asterisk) are the averaged FWHM of the retrieved ISRF, with their corresponding error bars (blue).The dashed red line identifies the estimated on ground one (Table1).

Figure 12 .
Figure 12.Similar to Figure 11 but for Channel 2. The three plots correspond to the three spectral lines from Table4.
Figure 12.Similar to Figure 11 but for Channel 2. The three plots correspond to the three spectral lines from Table4.

Figure 13 .
Figure 13.Similar to Figure 11 but for Channel 3. The four plots correspond to the four spectral lines from Table 4.

Figure 14 .
Figure 14.Similar to Figure 11 but for the solar line in Channel 4 (Table4).

Figure 15 .
Figure 15.Similar to Figure 11 but for Channel 5.The three plots correspond to the three spectral lines from Table 4.
Figure 15.Similar to Figure 11 but for Channel 5.The three plots correspond to the three spectral lines from Table 4.

Figure 16 .
Figure 16.Similar to Figure 11 but for Channel 6.The two plots correspond to the two spectral lines from Table4.The black line in the left plot is a smoothed curve of the FWHM.

Figure 17 .
Figure 17.Similar to Figure 11 but for the line of Channel 8 (Table4).

Figure 18 .
Figure 18.Degradation of SCIAMACHY throughput relative to August 2002 in Channel 7 & 8 in the calibration light path as a function of orbit [15].

Figure 19 .
Figure 19.SCIAMACHY temperature variation over time of channel 8.It shows a small period of noisy variations at the beginning of the mission and then increases over time.After the decontamination from 19 December 2008 (orbit 35574) until 3 January 2009 (orbit 35783), the average temperature decreased.

Table 1 .
The SCIAMACHY instrument channels.The spectral resolution in column 3 is also the FWHM (Full width at half maximum) of the corresponding ISRF Instrument Spectral Response Function).

Table 2 .
Recommended analytical functions for the ISRF function in seven channels of the SCIAMACHY instrument.Channel 7 was not included in the table as it was not analyzed in this study.The second column's functions were retrieved from on ground measurements during the OPTEC5 campaign.The third column's were obtained during the commissioning phase.
Table 3 and is measured in orbit 8884.SCIAMACHY measurement is shown in red, our model in green, and the theoretical very highly resolved one in black.The corresponding retrieved ISRF is a Hyperbolic function with a FWHM 0.24 nm.The fitting residuals (right) corresponding rms 0.023.

Table 4 .
The ISRF fitted function per channel and the averaged FWHM over the mission lifetime per measurement based on this study's results.Channel 8 results are shown in parenthesis because of their peculiarity and also as the measured line was not fully resolved.