A Foveon Sensor/Green-Pass Filter Technique for Direct Exposure of Traditional False Color Images

Abstract

A direct exposure technique to simulate the vegetation red-magenta hues associated with the traditional Kodak Ektachrome Professional Infrared EIR film is proposed. The method uses the Foveon sensor in full spectrum mode as found in Sigma cameras in combination with green-pass filters to reproduce the red-magenta hues of healthy vegetation without resorting to multi-exposures or channel swaps as is the current norm in single sensor cameras. An array of commercially available green-pass filters are analyzed via spectrophotmetric, densitometric and colorimetric measurements. A calibration technique using color-compensating filters is presented such that any green-pass filter will transmit green wavelengths to a set norm and ensure repeatability. A color model for the Foveon/green-pass filter is developed using quantum efficiencies to explain the color effects observed. Color similarities in near-infrared, blue and black and differences in red and green compared with Kodak Ektachrome Infrared film are discussed. Aerial photography examples are discussed, and the technique is proposed as a more direct means of obtaining traditional false color infrared imagery for pictorial and scientific applications when using Foveon type commercially available digital cameras.

1. Introduction

Today’s consumer-level digital color infrared imagery is quite different from the standard that was established by the Kodak Ektachrome professional infrared EIR film [1,2]. The traditional false color rendition associated with the film, particularly the red-magenta vegetation is generally replaced by a silvery looking vegetated landscape when exposing straight from the camera [3]. In pictorial photography, a near-infrared (NIR) pass filter, such as a Hoya R72, is generally used although camera infrared conversion firms do offer other filter possibilities. Still, the reddish magenta vegetation characteristics that the film exhibited and for which it was used extensively in scientific applications, particularly in remote sensing, have not been readily reproduced in pictorial photography, at least not without a degree of image post-processing [4]. If simply for continuity, a direct digital photographic exposure simulating a now defunct product would be welcomed in pictorial photography and may have some direct applications in more scientific endeavors, such as UAV remote sensing.

Kodak initially designed the color infrared film with the goal of camouflage detection [5]. The question of what color combination would maximize the tonal contrast of human-made objects amongst a vegetated background was foremost with the Kodak’s scientists and engineers at the time [5]. Rendering vegetation red because of its NIR reflective properties via an ingenious dye-layer combination in the film emulsion provided the maximum hue contrast, thus the precedent for depicting NIR vegetation as red was set with this initial work. With Kodak discontinuing this film and its aerial equivalent in 2009, research has been active in devising methods to reproduce these hues for the general consumer and scientific applications, particularly for relatively small scale unmanned aerial vehicle (UAV) operations [6,7,8,9,10,11,12]. The traditional NIR effect is reproduced by sophisticated (and expensive) aerial survey cameras, such as the Leica ADS series, the Z/I DMC series, the Ultracam series, to name three, but they use multiple sensors to re-create the traditional film rendition by imaging and registering the infrared, red and green images in separate channels. Most consumer level cameras are restricted to a single sensor using the Bayer color filter array for color reproduction and are further hampered in recording near-infrared by the infrared cutoff filter (hot mirror) placed on the sensor to block wavelengths greater than 700 nm. The hot mirror can be removed from consumer-level cameras to allow the cameras’ access to the sensor’s infrared sensitivity and a number of firms provide this modification service. These firms range from companies like Drones Imaging [13] and MosaicMill [14] that provide a specialized full camera conversion-remote sensing solution to firms such as LifePixel [15], Maxmax [16] and Kolari Vision [17] that will provide a range of NIR-capable cameras to the general consumer. Once the cameras have been modified the user has a number of methods to reproduce the traditional false color effect such as post-processing channel swaps using an infrablue (BG3) filter and separate (dual camera) NIR and RGB image registrations to reproduce the traditional red NIR film product hue [18,19].

Experimentation with the Sigma brand of cameras which use the Foveon sensor [6,20,21,22,23,24] has shown that the traditional Kodak color infrared (CIR) effect may, at least in the NIR portion of the spectrum, be possible from direct exposure without channel swapping or multiple exposures/image registrations. For many of the models that Sigma produces, there is the added advantage of being able to remove (and replace) the infrared cutoff filter within seconds allowing the sensor to register light in full spectrum mode, from about 400 nm to 1100 nm. The uniqueness of the Foveon sensor technology which, in general terms, mimics color film by registering three separate images in the blue, green, and red-NIR layers, allows for the general reproduction of the red NIR hues associated with the now defunct Kodak Professional Ektachrome Infrared EIR film and its aerial equivalent Kodak Aerochrome III Infrared 1443 film. Specifically, the effect is achieved by using a generic green-pass filter while sensing in full spectrum mode (infrared cutoff filter removed). Although green-pass filters have been one of many filters used in infrared filter effects [25], their transmission variability has not been fully investigated, particularly for achieving the Kodak CIR red NIR hue effect.

2. Background

The Foveon sensor/green-pass filter combination achieves the CIR effect through an additive process whereby the infrared object is registered in the red-NIR layer of the sensor with the green-pass filter absorbing much of the blue and red wavelengths while variably transmitting (depending on the green-pass filter) the green and infrared wavelengths. The sensor is quite sensitive to the mitigating effects of the green-pass filter; with too much green in the image a cyanish-green image is recorded, with too little green the infrared (red/magenta hues) overwhelms the photograph. The green transmittance needs to be at the ‘appropriate’ level (Figure 1). The question can therefore be stated as follows; what is the proper level of green-pass filter transmittance for producing the Kodak CIR red NIR effect and how do we calibrate to that level for repeatability?

The initial investigations into using a green-pass filter to reproduce Kodak CIR hues, although promising, were quite unpredictable; not all green-pass filters were creating the intended infrared red vegetation exposure effect demonstrated in Figure 1c. However, most of these green-pass filters were designated as ‘X1’ filters. For example, the filter used in Figure 1a is a Hoya 77 mm X1 while Figure 1c is a Hoya 49 mm X1, Figure 1b is actually a visually lighter green filter designated as a Hoya 77 mm X0. In monochrome film photography green-pass filters were typically used in portraiture to make skin tones appear more natural [26]. The spectral characteristics of these filters tended to vary to some degree, particularly among manufacturers, and that variability was acknowledged by photographers [27] but because the intended application was not critical the issues focussed mostly on exposure compensation and not spectral compensation. Since the green-pass filter was never intended to be used in color photography, small variations in green transmission were likely deemed to be inconsequential. This variability is, however, critical in applying green-pass filtration to achieve the Kodak CIR effect. A systematic sample evaluation of commercially available green-pass filters was required to quantify and calibrate for the variation.

3. Materials and Methods

3.1. Camera/Sensor

A Sigma SD1 Merrill camera fitted with a Sigma 18–50 mm f/2.8 EX DC telephoto lens was used to assess the photographic performance of its internal Foveon sensor with the green filters. The lens was also fitted with a UV filter as is standard practice in photography. This filter blocked all wavelengths shorter than 390 nm from being registered by the sensor; however, the UV filter had little effect here since shorter wavelength radiation was largely absorbed by the green-pass filters as will be noted later. In addition, the sensitivity of Foveon sensor is at or beyond the extreme limit of its working range at such short wavelengths.

The Sigma camera has a number of preset white balance settings, including a customizable setting. Through a process of trial and error the 4100 K preset color temperature setting was found to provide the best image results at solar noon. There was no apparent change with images taken a couple of hours prior to solar noon and up to four hours after solar noon (July, latitude 53°N). No imagery was taken at lower sun angles closer to sunrise or sunset. The 4100 K color temperature effect is attained by digital filtering in the camera and is the equivalent of filtering daylight (5500 K) through an 85C filter; the mired (micro-reciprocal-degrees) shift value is 62 for this conversion [28]. The 4100 K setting tends to emphasize the image signal in the red-NIR layer over the other two layers, particularly the blue, by a factor of about three. Although these settings can be modified in post-processing of raw-format imagery, it was important to find the best white balance setting at time of exposure so that our stated goal of ‘direct exposure’ with minimal or no post-processing could be achieved.

As noted, the internal near-infrared blocking filter was removed from the camera to enable full spectrum sensitivity; this ‘pre-filter’ also functions as a color optimizer in order to shape the spectral sensitivity curves of the sensor to the three types of human cones [19]. However, since the purpose of this experiment was to generate near-infrared images in false colors, the colorimetric standards that the Sigma camera is designed to achieve are irrelevant. The removal of the pre-filter allows the Foveon sensor spectral sensitivity characteristics to be used unaltered.

Although a data compilation process similar to that reported in Berra et al. [29] was devised for acquiring the spectral curves of the Foveon sensor, the data provided by Alternate Vision Corp. (Figure 2) was used in the analysis because of its cross reference to different Foveon sensor versions. The information is essentially for the Foveon F13 sensor but has been modified slightly in the blue layer [30,31] so as to correspond to the Foveon F20 version; this is the version found in the Sigma SD1 Merrill camera model. The minor blue layer differences in the Foveon sensor versions may ultimately be inconsequential here because all the green filters tested absorb strongly in the blue.

3.2. Green-Pass Filters

Thirteen ‘green’ standard commercially available filters were randomly purchased from photographic retail outlets to assess the green-pass variability across different filter brands (

Table 1. The thirteen ‘green’ filters randomly selected for comparison.

Filter Description	Abbreviation	Material	Format
Hoyarex Green 044	H044	Polycarbonate	Square 75 mm
Hoya 49 mm X1	H49X1	Glass	Circular 49 mm
C-N.P.S 49 mm X1	C49X1	Glass	Circular 49 mm
Tiffen Green 1 49 mm	T49H1	Glass	Circular 49 mm
Kodak Wratten 58	W58	Gel	Square 75 mm
Crown G 62 mm	C62X1	Glass	Circular 62 mm
Berkeley 67 mm P01	B67X1	Glass	Circular 67 mm
Izumar 58 m X1	I58X1	Glass	Circular 58 mm
NDG 72 mm X1	N72X1	Glass	Circular 72 mm
Sing 77 mm X1	S77X1	Glass	Circular 77 mm
Hoya 77 mm X1	H77X1	Glass	Circular 77 mm
Vivitar 55 mm No. 11 X1	V55X1	Glass	Circular 55 mm
Hoya 77 mm G (X0)	H77X0	Glass	Circular 77 mm

). Nine of the filters are designated as ‘X1’ which are typical green filters used in photography; the X1 designation is generally understood to be associated with the Kodak Wratten series #11 filter. The remaining four filters are also ‘green’ filters but have different designations. Most filters are glass except the Kodak Wratten #58 and Hoyarex 044 which are a gel and polycarbonate, respectively. The filters vary in diameter from 49 mm to 77 mm, and were generally held in the light path of the lens (77 mm diameter) during photographic testing.

3.3. Green-Pass Filter Spectrophotometry

The spectral characteristics of the green-pass filters were measured to provide a comprehensive transmission data set in the 400 nm to 1030 nm spectral range. Of particular interest was the performance of the filters in the near-infrared region since densitometry and colorimetry did not provide data in this area. The transmission spectra were measured by simply placing the filters in the air gap between two aligned optical transmission fibers separated by less than 1 cm. The “input fiber” was coupled to an LS-1-Cal reference light source (Ocean Optics) and the output fiber was attached to a USB2000 spectrometer from the same manufacturer. This method can accommodate all the various filter (and retainer) sizes and shapes. First, a reference spectrum was taken with no filter present, and then the filters were inserted and the corresponding transmission curves were calculated according to T = 100%∙(S_λ – D_λ)/(R_λ – D_λ) where S_λ is the signal intensity with the filter present, R_λ is the intensity of the reference light source with no filter present, and D_λ represents the dark spectrum.

One filter (the H77X0) was observed to exhibit slightly more than 100% transmission in the NIR. This effect likely occurs because of refraction in the filter, which concentrates a slightly greater fraction of rays into the collection fiber than without the filter present. The ideal solution would be to use an equivalent “reference substrate” but this was deemed impractical, as the filters had various different thicknesses and possibly different indices of refraction. Thus, the overall transmission curves should be taken as accurate to within ~5% (i.e., all of them will be slightly too high depending on the filter thickness).

The thirteen filters evaluated exhibited both general commonalities and differences (Figure 3). As ‘green-pass’ filters they all showed maximum transmission in the green from about 520 nm to 570 nm, varying mostly in degree of transmission. They also strongly absorbed in the blue region with most dropping below 5% transmission at 400 nm. The differences amongst the filters were most apparent in the red-NIR portion of their spectrum. Two distinct categories of transmissivity were discerned, one group (termed group 1) is characterized by a sharp increase near 700 nm and high NIR transmission. The second group of filters (group 2) showed relatively low red transmission and low-to-moderate transmission in the NIR.

Relating these curves to the filters and images initially shown in Figure 1, it is apparent that the high NIR transmission of the Hoya 77 mm X0 (H77X0) filter overwhelms the image with an abundance of NIR radiation associated with the red-NIR sensor layer (Figure 1b). Alternatively, the relatively high transmission of the Hoya 77 mm X1 (H77X1) filter in the green overwhelms the image with mostly a cyanish-green color. These two filters represent the extreme cases of the filters tested. The Hoya 49 mm X1 (H49X1) filter, which replicates the Kodak CIR effect ‘straight out of the camera’ (Figure 1c) is located between the extreme cases with its curve closer in shape to the H77X1 filter. The H49X1 filter was established as the reference filter as it required no compensation to produce the Kodak CIR-like red vegetation hue effect.

3.4. Green-Pass Filter Densitometry and Colorimetry

Densitometry and colorimetry provide us with practical methods for assessing and modifying the optical density and hue characteristics of materials. Densitometry further enables us to associate the visible values observed in the spectrophotometry with the color compensation filtering system generally used in photography. For example, the green transmission value for the W58 filter peaks at about 66% at 520 nm (Figure 3). This corresponds to an optical density of 0.18 (D = −log₁₀ T); the densitometric measured value is 0.16 which is within the error tolerance (±0.02) of the densitometer.

The calibration process that follows is based on this measurement system. An X-Rite 310 color transmission densitometer was used to obtain Status A densitometry (cyan, magenta and yellow) transmission densities for each of the green-pass filters tested. An X-Rite eXact spectrophotometer (this spectrophotometer did not have NIR sensitivity) was also used to obtain the colorimetric values (

Table 2. Densitometric (C, M, Y) and colorimetric (CIE L*a*b) measurements of some commercially available green-pass filters (various diameters) obtained for the study. The filters and their measurements are intended to show the variation in the actual color properties of these ‘green-pass’ filters. The CIE L*a*b* color patch provides a visual impression of the filters’ green hues.

¹ The CIE L*a*b* color space equivalents were generated in Adobe Photoshop CS3.

, Figure 4).

The colorimetric CIE L*a*b* color space values were derived for comparison and qualitative visual color description. Although a colorimetric approach was deemed unsuitable because the measurement purpose was not to relate the resultant color to an observer's perception but to the Foveon sensor’s sensitivity (the CIE standard is also not applicable for near-IR wavelengths), the outlier filter points (Figure 4) do correspond with the group 1 filters (high NIR transmission) in Figure 3. Alternatively, the group 2 filters (low NIR transmission) are generally vertically aligned between CIE a* −64 and −69 indicating variation mostly in their ‘yellowness’. Metamerism, however, is evident in the differences in perceptual color and color transmission. For example, the S77X1 green-pass filter is colorimetrically close to the H77X1 and the B67X1 filters (Figure 4) but it is densitometrically quite different from them (Table 2). In effect, it is the quantity and quality of the green-pass filter’s transmitted light and not our color impression of it that produces the CIR effect [32].

The test photographs included a Kodak Color Separation strip (also called ‘Color Control’) that provided the color patches for a hue comparison between the photographed color renditions and the strip's descriptive primary and secondary colors; its white patch was also used for white balance measurement. As already noted, a neutral white balance was generally achieved with the camera white balance set at the “Fluorescent” setting or 4100 K. Included in the photographed scene with the Kodak color patch strip were green trees to provide an indication of the amount of near-infrared radiation being registered by the sensor.

Since the green transmission, and the consequent infrared effect, varied with the different filters, Kodak Color Compensating (CC) filters [28] were used to attenuate the amount of green light reaching the sensor [33]. CC filters enable density control by adding or subtracting color density in increments of 0.025, 0.05, 0.10, 0.20, 0.30, 0.40, and 0.50 units (Roscoe is another supplier of calibrated CC filters); these density units are multiplied by 100 to designate the CC filter. For example, if we wanted to absorb 0.20 density units of green layer transmission we would use a 20 magenta CC filter. Therefore, regardless of the green-pass filter used, the NIR red hue rendition for vegetation was repeatable once the optical density of the green-pass filter was measured and compensated for. These green-pass filters also varied in amounts of yellow and cyan in their transmission profiles; this component was corrected by appropriate blue and red CC filtration.

3.5. Calibration Procedure

Establishing a reference is critical in the calibration process. Through a trial and error exposure process, the filter that best reproduces the Kodak CIR effect straight out of the camera was the Hoya 49 mm X1 filter (Figure 5).

The red/magenta hues of the trees coupled with a close to neutral white balance on the Kodak Color Control strip makes this the best filter in achieving the desired end result. The densitometric values for this green-pass filter (Table 2) are cyan (C) = 0.95, magenta (M) = 0.22, and yellow (Y) = 1.09, and the colorimetric L*, a*, and b* values are 62.2, −64.1 and 52.2, respectively; it is considered a yellowish green-pass filter. The density values provide an objective measure of how much red, green and blue “dye” this filter contains, while the colorimetric values give a visual impression of its “greenness” in CIE L*a*b* color space. Proceeding to zero (subtract) the neutral density for each color, (0.22 density units are common to each color), we obtain C = 0.73, M = 0.0 and Y = 0.87. Since cyan and yellow produce green, the filter will have a density of 0.73 green and a small component of 0.14 yellow over and above the green density (0.87 − 0.73 = 0.14).

Table 3. Calculated base and final green densitometric values with derived color compensation calibration adjustments for all the green-pass filter brands. CC adjustments are relative to the reference filter (Hoya 49 mm X1). The letters in the color compensation column are associated with the standard notation and description of color compensation filters i.e., M: magenta, B: blue, G: green, R: red. Heading letters in the other two columns (Y and –C) refer to excess yellow (denoted by positive values) and cyan (denoted by negative values), respectively. The CC values for the Kodak Wratten 58 are considered beyond practical adjustment.

Filter Description	Abbreviation	Green Base	Green Final	Y or –C Difference	Zero Green	Zero Y or –C	Color Compensation Adjustments
Hoyarex Green 044	H044	1.04	0.42	0.39	−0.31	0.25	30 G + 25 B
Hoya 49mm X1	H49X1	0.95	0.73	0.14	0.0	0.0	None
C-N.P.S 49 mm X1	C49X1	1.05	0.79	0.46	0.06	0.32	5 M + 30 B
Tiffen Green 1 49 mm	T49H1	0.92	0.72	0.65	−0.01	0.51	0 M + 50 B
Kodak Wratten 58	W58	1.92	1.76	2.4	1.03	2.26	100 M + 225 B
Crown G 62 mm	C62X1	1.09	0.84	0.16	0.11	0.02	10 M
Berkeley 67 mm P01	B67X1	1.38	1.06	−0.06	0.33	−0.20	30 M + 20 R
Izumar 58 mm X1	I58X1	1.19	0.9	0.07	0.17	−0.07	15 M + 5 R
NDG 72 mm X1	N72X1	1.05	0.76	0.45	0.03	0.31	5 M + 30 B
Sing 77 mm X1	S77X1	2.57	1.94	0.3	1.21	0.16	120 M + 15 B
Hoya 77 mm X1	H77X1	1.69	1.29	0.07	0.56	−0.07	55 M + 5 B
Vivitar 55 mm No. 11 X1	V55X1	0.39	0.28	0.94	−0.45	0.80	45 G + 80 B
Hoya 77 mm G (X0)	H77X0	0.26	0.19	0.24	−0.54	0.10	55 G + 10 B

shows the comparative results of this effective green calculation along with the residual yellow or cyan cast for each of the filters assessed in this study. Since we are setting the Hoya 49 mm X1 filter as the reference, it requires no color compensation adjustment.

Using other brands of green-pass filter to obtain the Kodak CIR effects resulted in highly variable results; most often the desired effects are either muted or, as was noted earlier, not reproduced at all (Figure 1a,b)). This indicates a high sensitivity to slight variations in green (and infrared) densities. Figure 1a was obtained with a Hoya 77 mm green X1 filter (H77X1). To calibrate the green transmission characteristics of this filter and match it to the reference filter (H49X1, Figure 1c) it needs to be attenuated. The process is as follows:

• Obtain transmission status A densitometry readings for cyan = 1.69, magenta = 0.40, and yellow = 1.76. Visually, this filter can be described by the colorimetric values L* = 67.9, a* = −68.9 and b* = 63.5, representing a slightly darker green-pass filter than the reference.
• Subtract the neutral density from each color i.e., the lowest common density value (M = 0.40), cyan = 1.29, magenta = 0.0, yellow = 1.36.
• Determine the base green density (cyan and yellow produce green) by taking the lowest of the cyan and yellow densities; 1.29 is the base green density for this filter and it has a slight yellow cast of 0.07 density (1.36 − 1.29 = 0.07).
• Determine the difference in the green transmission of this filter to that of the reference H49X1 green-pass filter (1.29 − 0.73 = 0.56). This filter is “greener” by 0.56 density units.
• Use magenta color compensating filters to absorb the extra green (50 M + 5 M = 55 M filter pack).
• Use blue color compensating filters to absorb the very slight 0.07 yellow (5 B).

The RGB values for the reference image and calibrated images are comparatively close. The white patch RGB values are 211, 248, 247 on the calibrated image (Figure 6) and 205, 249, 224 on the reference image (Figure 5). The discrepancy in the blue value is likely due to the higher yellow cast in the reference image. The RGB values for the all-important vegetation component (same location on images) are 170, 20, 66 for the calibrated image and 177, 20, 73 for the reference image.

Working through one final example with the Izumar 58 mm X1 green filter we obtain density values of C = 1.19, M = 0.29, and Y = 1.26 and colorimetric values are L* = 66.7, a* = −66.5 and b* = 56; again, a relatively darker green-pass filter than the reference. Zeroing the neutral density by subtracting the lowest common value (0.29) from each of the colors results in C = 0.90 and Y = 0.97 (magenta is 0.0). This filter will have a green density of 0.90 with a slight cyan cast of 0.07 density units (Table 3). The main difference between the reference filter and this one is that it is a deeper green by about 0.90 − 0.73 = 0.17 density units which is also equivalent to about a half-stop in exposure value. We proceed with the calibration procedure by compensating the additional green with a 15 M and 5 R color compensating filter (this is a 0.15 density magenta filter and 0.05 red density filter). This image’s RGB values for the white patch and vegetation are 192, 244, 221 and 172, 22, 68, respectively (Figure 7) and are comparatively close to the reference; white patch = 205, 249, 224, vegetation = 177, 20, 73.

These working examples show that in most of the products tested both the green and most often yellow, require adjustment. As noted earlier with the colorimetric descriptors, the X1 filters are visually a yellowish-green, varying in actual density by manufacturer. Figure 8 represents a graphical summary of the degree of color compensation required, by magenta if the filter is too green, by green if the filter is not green enough, by blue if the filter is too yellow and by red if the filter is too cyan.

This working method for recreating consistent near-infrared red/magenta hues with Sigma/Foveon cameras is structured as a one-time input process. Once the green filter transmission (densitometry) is known and calibrated (with CC filters), and the camera white balance is set to 4100 K, then no more post-processing of the images is needed to achieve the NIR hues. Output may be in raw format but JPEG or TIFF images work equally well since it is assumed that very little hue adjustment of the images is required; Figure 5, Figure 6 and Figure 7 are, in fact, JPEG images straight from the camera.

4. Analysis and Discussion

4.1. Kodak CIR Color Renditions

The Kodak CIR signature was the red rendition of highly NIR reflective vegetation; the Foveon/green-pass filter process simulates this false color in the same manner of the original film product with the appropriate CC filter adjustments. But what of other colors, how are they rendered in relation to the CIR film’s false colors?

The Kodak CIR film reproduced colors as per the model in Figure 9; blue was rendered black, green would be blue, red would be green, NIR would be red and yellow would be rendered cyanish-white depending on the luminosity. Film examples are difficult to replicate today (the original film is no longer available) but an example from personal archives (Figure 10) provides some indication of how the film product transformed normal colors. The work by Eyton [34] is also a good source for comparison. The color renditions in Figure 10 are as predicted by the model except for the blacks and blues, both these colors should be black according to the Figure 9 but they are rendered a reddish-purple by the film. This is likely caused by the NIR reflective properties of the dye in the fabrics that are worn by the people in the images.

4.2. A Foveon/Green-Pass Filter Model

A Foveon/green-pass filter model similar Kodak CIR film can be constructed using the green-pass transmission curves, the color temperature modifier, and the Foveon QE sensor curves. recalling the filters used in Figure 1 where the images produced were either too cyan-green (H77X1 filter), too reddish-orange (H77X0), or ‘just right’ (H49X1), we can use the sensor quantum efficiency and filter information to model the color effects observed in the images. The CIR emulation process using the green-pass filters and the application of a color temperature filter [35,36] modifies the effective QE of the whole system (as compared to those of the sensor alone, as shown in Figure 2). By multiplying the sensor QE values by the fractional transmission through the green pass and color temperature correction filters we obtain the “effective” system QE values (Figure 11).

We next estimate the ratio of incident photons entering the R, G, and B bands, assuming a flat light source. These bands have different effective QE values and spectral widths. A simple estimate can be obtained by summing the area under the curves in Figure 11 in the wavelength ranges from 420–490 nm (blue), 520–575 nm (green), 600–670 nm (red), and 600–1030 nm (red-NIR) for each individual layer of the Foveon sensor, as shown in

Table 4. Ratios representing the relative number of detectable photons for each Foveon layer at the specified wavelength intervals. The resultant colors are the hues one would obtain if these ratios were transformed to 8-bit RGB hues. The red band data is actually a subset of the red-NIR band since the Foveon sensor records red and NIR wavelengths in the same layer.

Filter	Spectral Intervals (nm)	Blue Foveon Layer	Green Foveon Layer	Red-NIR Foveon Layer	Resultant Color
H77X1	420–490	1.0	0.7	0.2
H77X1	500–575	7.3	9.1	6.1
H77X1	600–675	0.2	0.3	0.4
H77X1	600–1050	0.8	1.4	2.8
H77X0	420–490	5.3	3.4	0.6
H77X0	500–575	15.3	19.4	13.6
H77X0	600–675	7.5	13.0	19.7
H77X0	600–1050	28	53.6	120.1
H49X1	420–490	2.8	1.9	0.4
H49X1	500–575	10.6	13.2	9.0
H49X1	600–675	1.1	1.9	2.8
H49X1	600–1050	4.0	7.3	14.6

. These numbers simply represent the ratio of incident photons detected in each wavelength band, assuming a flat light source, for each sensor layer. They can be straightforwardly converted into an RGB color ratio, as shown by the resulting color patches in Table 4.

The outcome of this calculation shows that most of the image signal associated with the H77X1 filter will occur in the 500–575 nm wavelength range (the green band), hence, a cyanish-green predominant hue will be observed (Table 4). The system response with the H77X0 filter is strong in the 600–1030 wavelength interval (orange-red hue) while the H49X1 appears balanced between these two extremes. These derived results are representative of the hues we see in the image sequence displayed in Figure 1.

The H49X1 filter (reference) displays other color anomalies associated with the Foveon/green-pass filter process that can be observed on the Kodak color separation strip photographed in Figure 5. For example, a red reflecting object assumes a muted response in the Foveon/green-pass model, yet it is vibrant and pronounced on the color strip. Similarly, the green patch represents a ‘purer’ green than the model’s cyanish-green prediction. The blue and black patches are rendered very close to the model colors: they are dark blue and black, respectively. Blue and black fabrics, however, tend to more closely emulate the CIR film with black fabrics rendered a dark red while blue fabrics appear reddish violet (Figure 12).

The primary colors on the Kodak color control strip would have been imaged quite differently with the CIR film; blue would be black, green would be blue, and red would be green (Figure 9). It is evident that although the Foveon/green pass filter process substitutes well for CIR film in the NIR, it assumes a more RGB-like profile in the visible spectrum (blue excepted). In this sense, the Foveon CMOS/green-pass filter combination behaves more like an RGB/CIR film hybrid which introduces some unique imaging possibilities.

4.3. Color Compensation Filter Effects

The densitometric measurement process provided a means to calibrate the variable transmission profiles of all the green-pass filters to the reference H49X1 filter via CC filters. We now introduce these CC filters in the effective QE calculation and calibration of the H77X1 and H77X0 filters. Figure 13 shows the spectral transmission curves for each 50CC filter in which we see that the blue and green filters are absorptive in the NIR while the red, yellow, cyan, and magenta are generally transparent in the NIR. This property of the CC filters allows for a dual attenuation of the green and the red-NIR wavelengths which is important for calibrating the green-pass filter transmission characteristics.

Repeating the process used to sum the effective QE values and deriving the colors in the same way as before, we include the transmission effects of the color compensating filters (in addition to the green filter + color temperature filter combinations in the previous section). The densitometric calculation for calibrating the H77X1 and the H77X0 filters resulted in 55 M-5 B and 55 G-10 B, respectively (Table 3). Adding these filters’ transmission characteristics to the H77X1 and H77X0 filters allows us to derive color profiles that now resemble the H49X1 reference filter (

Table 5. Comparative color results of non-adjusted and CC adjusted H77X1 and H77X0 green-pass filters used in Figure 1. They are compared to the reference H49X1 filter.

Green-Pass Filter	Spectral Wavelength Interval	Non-Adjusted Colors 1	CC Filter 2 Adjusted Colors	H49X1 Reference Colors
H77X1	420–490
H77X1	500–575
H77X1	600–675
H77X1	600–1050
H77X0	420–490
H77X0	500–575
H77X0	600–675
H77X0	600–1050

¹ Included from Table 4 for comparison; ² The CC filter adjustments are 55 M-5 B and 55 G-10 B for the H77X1 and H77X0, respectively (CC values from Table 3).

). These two filters are, in effect, now calibrated.

4.4. Remote Sensing Applications

One of the most important uses of Kodak CIR film was in remote sensing. The Kodak Aerochrome III Infrared 1443 film and its predecessors were standards in camouflage detection, plant disease assessment, tree species differentiation, hydrological delineation etc. [37]. As noted in the introduction, many firms are now providing modified digital cameras with the NIR filter removed for remote sensing purposes, usually for uses with UAVs. The typical process for this work is to photograph with a red (Wratten #25) or infrablue (BG3) filter and perform a NNR (red/blue) or NBN (blue/green) channel swap in post processing [38], multi-sensor systems will generally replace the red channel with the NIR image data and insert the red and green image data into the green and blue channels, respectively (NRG); the process often leads to deriving normalized difference vegetation index (NDVI) values [39,40]. Sigma cameras with the Foveon sensor have been investigated for this purpose but most have been used in dual or multi-sensor camera mode (capturing both RGB and near IR) or in single camera mode with channel swapping [17,18].

Preliminary aerial field trials using the Foveon/green-pass filter technique were performed in September 2015 in an area of the mixed-wood Boreal forest of central Alberta, Canada. Low oblique images were obtained from a height of about 1500 m above ground level with the Sigma SD1 camera in full spectrum mode using the Berkeley 62 mm green X1 filter; a normal RGB camera (Sony DCS H3) was also used for comparison (Figure 14 and Figure 15).

The results simulate the image hues associated with an NGR channel sequence common to dual camera platforms, NDVI modified cameras, or even the high end multi-sensor photogrammetric camera systems. Figure 16 shows a comparison of a Foveon green-pass filter image with one taken with a Z/I DMC 230 camera. Since the Foveon/green-pass filter technique is depicting some of the landscape in RGB mode, some of the hue differences are more pronounced, and may possibly, enhance overall interpretability. This can readily be observed in the pronounced differences between the deciduous and coniferous forest species where the hardwoods are prominently reflecting in the NIR band while the softwood signatures are actually a dark green and not a subdued red as is typical in NRG-channel-configured sensor systems. It is important to re-state that very little post processing (other than some contrast enhancement for visual appeal) was done to the Sigma/Foveon images—the photos are JPEG format images, essentially straight out of the camera. These preliminary results suggest the need for further investigation because the process introduces a more data-rich technique that may be of benefit in forestry and in other remote sensing environmental applications. It also provides for a more direct exposure-to-application process without the intervening post-processing time and effort currently in use, which can translate to lower costs in image acquisition and processing.

5. Conclusions

The preliminary results of this study show that a direct exposure Kodak CIR like image with red/magenta NIR reflecting vegetation is possible with today’s digital camera technology, specifically with the use of the Foveon sensor and green-pass filters. The calibration procedure, although relatively simple, does need some special instrumentation such as a densitometer but these instruments are quite common (graphics labs) and need to be used only once to obtain the base density data for the filter. Although Kodak CC filters were used in this study, similar filters are also commonly available from companies, such as Roscoe or Lee. For pictorial use, the Foveon/green-pass filter technique provides enough false color similarity to the Kodak Ektachrome Infrared EIR film to serve as a versatile substitute. For scientific applications more testing is needed with this technique in the specific field of inquiry and refine its application, particularly since the technique may in fact improve upon the original the film product.

The key point is to achieve an ‘appropriate’ level of green filtration to induce the Kodak CIR effect and calibrate to that level. The appropriate level probably depends on the application but once the reference is chosen the process follows as outlined. Pictorially, the technique and reference image presented seems to adequately simulate the general character of the Kodak CIR film. For applications in remote sensing, forensics, medical, document verification, etc., researchers could employ this technique, with adjustments, to determine the ideal green-pass filter density conditions for those areas in which the Kodak CIR film excelled. Certainly, one obvious advantage of this process is stability and repeatability; the CIR film was prone to variability in its NIR rendition because of storage conditions, age, and chemical processing. These variables are effectively eliminated in the digital CIR world.

Finally, some mention should be made that Sigma cameras have a relatively small commercial market niche (as of 2016) and therefore the technique presented is restricted to this camera user group. Custom camera applications using the Foveon sensor are possible with firms such as Alternate Vision Corp. but they would be specialized adaptations. In this regard, Bayer based systems should also be investigated to assess whether a direct exposure Kodak CIR effect can also be reproduced following a variation of the principles discussed. There are the challenges of the numerous makes and models commercially available, the variability of the Bayer color filter arrays [29,31], and that most of these cameras require hardware conversion (physical removal of the hot mirror from the sensor), nevertheless the investigations could lead to providing a traditional CIR-like continuity to a much larger photographer base.