Changing from an entrance-aperture diameter of 20 mm for viewing Earth scenes to 0.5 mm for viewing the Sun provides a geometric attenuation level of 10-3.2. Optical-system complexities disfavor the use of larger apertures, while diffraction-loss uncertainties start to preclude the use of significantly smaller ones to obtain greater attenuation ratios via this method.

The HySICS apertures are diamond-turned nickel-coated aluminum, providing a very sharp aperture edge with nearly negligible scatter. The six installed apertures have entrance diameters of 20, 10, and 0.5 mm, with two each of the largest and smallest. All are calibrated by NIST/Gaithersburg for geometric area using a non-contact optical technique to achieve the desired attenuation uncertainties, with the limiting factor being the 0.06 to 0.08 % (1σ) relative area uncertainties of the solar-viewing 0.5 mm diameter apertures.

Shorter integration times are used for solar viewing than Earth-scene measurements. These are enabled by the FPA electronics, reproducible detector linearity, and an electronic global shutter to avoid spatial smear during image integration.

The FPA's controlling electronics demonstrate < 14 ns timing stability and linearity to < 10-6 over the integration time range from 16.8 µs to 34.4 ms, providing 10-3.3 solar-attenuation capability. The FPA response itself, unsurprisingly, has higher non-linearities but nevertheless demonstrates sufficient linearity stability to allow corrections for operation over the large applied intensity range.

Spectral filters capable of roughly 10-1 attenuations can be calibrated on-orbit via lunar observations. Greater filter-based attenuations are precluded by low lunar-radiance levels that would limit the accuracies of these on-orbit calibrations. On-orbit spectral filter calibrations using the Sun are also possible because of the large integration-time range achievable with the FPA. These solar-based calibrations benefit from the use of the same small aperture (and thus the same optical path) used in operations when acquiring solar observations with the filters.

The three ionically colored Schott glass filters in the HySICS were polished to 0.1 nm RMS surface roughness and λ/4 flatness to reduce induced scatter and distortion. The balloon-flight filter selection includes

NG4 (a neutral-density filter with ∼ 0.1 transmission),

The Wallops Arc Second Pointer (WASP) is a two-axis altitude-azimuth gimbal-based pointing system designed to achieve nearly arc-second accuracy levels for balloon payloads (Stuchlik, 2015a, b). This system was provided courtesy of HySICS co-investigators D. Stuchlik's and J. Lanzi's team at NASA's Wallops Flight Facility (WFF). With the HySICS center-of-mass aligned within the WASP gimbal-axes to ±25 µm, the system is able to point the instrument accurately at the ground, Sun, and Moon and track each of these objects while acquiring the needed measurements and calibrations.

The WASP generally provided < 10 arcsec of pointing accuracy during Flight 2, meeting the HySICS pointing requirements. The most critical pointing requirements are driven by scanning the Sun or the Moon lengthwise along the HySICS slit to obtain FPA flat fields by positioning the same portion of the Sun or Moon on each pixel in the FPA's spatial direction. Pointing knowledge and after-the-fact corrections are not sufficient for this flat-fielding calibration method; real-time pointing accuracy is needed. The WASP system achieved approximately 8 arcsec (1σ) pointing deviations across the ±6∘ range about disk center for along-slit solar scans, acquiring the needed flat-field calibrations. Accuracies of 2 arcsec (1σ) across the ±1.5∘ range for cross-slit solar-scans were achieved, with these scans intended to acquire solar-irradiance measurements by spatially integrating sequential images across the solar disk via post-flight ground-based data processing. The WASP provided 0.7 arcsec (1σ) along-slit stability and 2.0 arcsec cross-slit stability when staring at the Sun. While the WASP is generally capable of yet more accurate pointing, that provided during flight was sufficient for the HySICS's purposes.

The WASP was also able to inertially track the Moon using an on-board ephemeris. This new pointing-system capability enabled flat-fielding calibrations using the Moon while operating with the same 20 mm aperture (and thus optical paths) and integration-time parameters as used for Earth-scene observations.

The balloon gondola is a rectangular-frame structure that houses the entire payload, consisting of the HySICS instrument, the WASP, 27 lead-acid batteries to supply power, all telemetry and tracking equipment, thermal enclosures, several crush pads for landing, and ballast. The net mass of the payload and gondola is 2300 kg (5000 lb), including 540 kg (1200 lb) of ballast.

The gondola was designed and built at the University of Colorado's Laboratory for Atmospheric and Space Physics (LASP), using a combination of 80–20 aluminum and square aluminum tubing. The structure is 3 m in height and contained within a 4.3 m diameter region when the crush pads are installed on all but the top of the gondola's six rectangular sides. During flight, the entire structure is suspended by the azimuthal rotator that provides coarse pointing.

The WASP and HySICS are centrally located in the gondola such that the HySICS can view nadir for observing the Earth and greater elevation angles for solar and lunar measurements. Once expanded at altitude, the overhead Helium-filled balloon restricts viewing to elevation angles < 60∘. A remotely controlled caging mechanism locks the WASP to the gondola frame for launch and parachute-descent landing.

These cross-track scans, with the ground track and speed determined by the balloon velocity from the aloft winds, provide samples of the desired data from an eventual flight instrument. During Flight 2, four ground scans were acquired. The two in the morning included a mix of the New Mexico high desert with broken clouds, while the two in the afternoon were predominantly of high, thin clouds. Three-dimensional data cubes of these scans were created in ground processing after all radiometric calibrations were applied.

Several scans of the Earth limb were also obtained on this flight. These scans provide spatial–spectral information through the vertical extent of the Earth's atmosphere. The Earth limb itself was largely occulted by the tops of bright cumulus clouds at the near-horizontal look-angle for these scans. Some such scans also included the Moon as it was setting.

Along-slit scans enable flat-fielding of the FPA by placing the same portion of the Sun on each spatial element of the array. Cross-slit scans build up an entire data cube of the Sun, enabling the spatially integrated solar irradiance to be determined and allowing SI-traceability to SSI (provided on an absolute scale by other measurements or models), as detailed in Sect. 4.4. Since demonstrating the solar cross-calibration method was the primary purpose of these flights, solar scans dominated the flight observation time. Near local noon the Sun's elevation was greater than 60∘, so solar observations could not be acquired due to glint or occultation by the large overhead balloon. At these times, either lunar or ground scenes were acquired instead.

Similar to those done with the Sun, along-slit scans enable flat-fielding of the FPA by placing the same portion of the Moon on each spatial element of the array. The lunar scans can be done with the larger Earth-viewing aperture, potentially providing a more appropriate flat field to be applied to ground scans than those obtained from solar scans. Additionally, spectral-filter transmission is calibrated during flight by quick successive measurements with each filter in and out of the optical path while tracking a fixed position of the Moon.

Internal-instrument calibrations and diagnostics helped track instrument functionality, stability, and performance in flight. Spectral calibrations were made intermittently throughout the flights by briefly illuminating the instrument's Hg / Ar pen-ray lamp. Pointing stability was quantified by attempting to maintain the instrument slit at a fixed position on the edge of the lunar limb for an extended period. At this position, lunar intensity is very sensitive to cross-slit variations in pointing, providing a diagnostic of pointing stability. An along-slit scan at this lunar position quantified the instrument's alignment relative to the WASP's elevation (altitude) direction.

Non-responsive pixels and badly fluctuating pixels, defined as those with a measurement-to-measurement standard deviation of more than 5σ greater than the sensor-wide average standard deviation, are filled using an average of all properly operating neighboring pixels. The HySICS FPA had 732 pixels needing such corrections. These are sufficiently few that they do not greatly influence subsequent statistics based on full-FPA data using their corrected values.

Read noise for the Teledyne sensor is determined using a traditional photon-transfer measurement (Janesick, 2001) of a constant radiant-power source provided by blackbody radiation from a uniform, warm, temperature-stabilized target. This target is measured at various exposure levels by varying the integration time from 33.6 µs to 34.4 ms. A corresponding dark image, acquired during a prior measurement of a 77 K target to eliminate blackbody radiation, is subtracted from each exposure in the photon-transfer measurement. The measured noise on each pixel, given by the standard deviation of 50 repeated measurements, is dominated by read noise at the shortest integration times and by shot noise at the longest integration times. Although a true zero integration time cannot be achieved, the noise versus signal level for each pixel is curve-fit to an expected photon-transfer curve to extrapolate to its true read noise. The sensor-wide average read noise is 8.3 DN.

With a gain of ∼ 12 e- DN-1 (see Sect. 4.1.5), read-noise uncertainties are thus based on random fluctuations around 100 e-. Since these are of similar amplitude across the array and are independent of incident signal, read noise causes a higher relative uncertainty at low signal levels, such as the extreme portions of the spectral range where the solar signal and the detector response are both low. Higher signal levels, such as can be achieved from brighter scenes or longer integration times, reduce the effects of read-noise uncertainties. By acquiring all solar calibrations at both short and long integration times, read-noise in select wavelength ranges is greatly improved. Similarly, since read noise is a random statistical fluctuation, acquiring repeated images of the same scene reduces the effects of read noise as the reciprocal square root of the number of images. Such integration-time variations and multiple-image acquisitions are not possible when viewing the ground during flight, since balloon-track motion between frames causes either a different ground scene (for static nadir-viewing) or a different look-angle of the same ground scene (if actively tracking) to be measured by non-simultaneous successive frames; however, multiple-image acquisitions are implemented for HySICS calibrations using static sources such as the Sun and Moon. Thus, read noise mainly contributes to the ground-measurement uncertainties at shorter wavelengths.

Both dark-noise and thermal-background signals from the surrounding instrument scale with integration time and are dependent on instrument or FPA temperature. Thermistors monitor the FPA and several of the nearby instrument-components. Laboratory characterizations of the dark signal enable corrections for both internal-FPA and background-thermal effects.

The HySICS FPA's inherent dark signal is sufficiently low that it is difficult to detect in the presence of any background light. A cold target placed in front of the imager while keeping the sensor housing at -25 ∘C reduced such background signals but did not completely eliminate them sufficiently. Dark current, which increases with FPA operating-temperature, was therefore measured at elevated operating temperatures of 165 K and warmer. These measurements were extrapolated to the FPA's nominal 150 K operating-temperature, yielding a dark current of 350 e- s-1 and resulting uncertainties of 0.29 DN for the longest HySICS integration times (34.4 ms) used. These inherent dark-signal uncertainties are well below the quantization limit of the device.

Background signals were also corrected during flight. Following all data acquisitions, the blanked aperture wheel position blocked incoming light for 100 exposures. These consist only of dark current, instrument-thermal-background contributions, and imager-fixed-pattern noise. They are acquired at the same integration time and nearly the same temperatures as the data frames themselves. The average of these dark exposures is subtracted from the data frames, thereby removing background offsets with the exception of possible thermal offsets caused by temperature differences between when the data and the dark measurements where acquired. These temperature dependencies are in turn corrected via in-flight thermal-background measurements using portions of the array viewing dark space during solar and lunar scans. From multiple such scans, FPA sensitivities to instrument thermal effects are determined as a function of surrounding instrument-component temperatures. All raw HySICS data images are thus corrected for thermal background based on the instrument temperatures at the actual time of data acquisition, using the instrument-temperature dependencies determined from these dark-space observations.

Although these thermal-background signals are largest at the longer-wavelength portion of the FPA's sensitivity, they influence the entire array uniformly, since the FPA has no long-wave rejection filter over the portions used only for shorter-wavelength readout, making the above corrections necessary for all portions of the spectrum. While the dark current is very small and contributes nearly insignificantly to the net HySICS uncertainties, the thermal-background signal contributes to shot noise (described in Sect. 4.2.1).

Deviations from linearity are determined individually for each FPA pixel in laboratory testing using varying levels of incident-light intensity and integration times. If temporally stable, non-linearities can be corrected once characterized. These corrections are applied to the images after the bad-pixel, dark, and thermal-background corrections.

Sensor linearity is measured in two steps: (1) The electronically determined integration time is measured directly using timing pulses from the sensor's field-programmable gate array's digital output signal and (2) the response of the FPA itself is measured using a stable light source while varying the now-known electronically controlled integration time. The former verifies the timing of the controlling electronics, which are, as expected for oscillator-based signals, very linear and stable. The latter step includes the effects of FPA pixel-well or amplifier-signal saturation and is a function of the net signal on each pixel. To characterize these non-linearities, the sensor is illuminated by a stable FEL lamp while the electronically controlled integration time is varied and the resulting signal levels are measured. A linear curve-fit is used to determine the expected signal level on each pixel, and deviations from that fit with signal level are considered non-linearities in that pixel's response. The curve fit uses only the most linear portion of the data at less than 50 % of the FPA's full well. Repetition of this measurement using various FEL-lamp intensities ensures that the deviation from linearity has an FPA signal-level dependence rather than an integration-time dependence.

The resulting non-linearities and uncertainties are detailed in Sect. 4.3.2, where the non-linearity corrections, uncertainties, and intensity range and the resulting dominant determinants of the overall instrument attenuation uncertainty based on the integration-time method are discussed.

Sensor gain, or the conversion [e- DN-1] from FPA DNs to electrons [e-] and thus photons, is determined from a photon-transfer measurement in laboratory testing on a pixel-by-pixel basis using statistics of each pixel's variations at different intensity-exposure levels. This was done using the same experimental setup as the read-noise measurement described in Sect. 4.1.2. In the larger-signal regime, where pixel noise is dominated by shot noise, sensor gain is defined as the ratio of signal level to pixel-noise variance. The previously determined read-noise variance is subtracted from the measured pixel-noise variance so that the residual noise is that due solely to shot noise. For each pixel, the signal level and pixel noise are determined using 50 exposures repeated at ten different signal levels. The experiment is repeated 100 times to determine the average gain and to reduce the shot-noise measurement uncertainty. The sensor-wide average pixel gain is 12.01 e- DN-1, with an average uncertainty per pixel of 0.12 e- DN-1, or < 0.003 % uncertainty in the shot-noise calculation at a signal level of 15 % (10 000 DN) of full scale. This pixel-dependent correction is applied to each pixel in the array but is an insignificant contributor to the net uncertainties.

Flat-fielding the HySICS sensor requires a full-system calibration, since it is affected by the collective efficiencies of all upstream optics. This calibration therefore needs to be performed separately for the smaller solar-viewing aperture and the larger Earth-viewing aperture, as light passing through the two apertures interacts with different portions of the downstream optical elements in the instrument. These differences are accounted for via the flat-field calibrations and are corrected in post-processing of the data.

Although different apertures are used for the two scans, the flat-fielding procedure for both is to use a stable light source that can be swept across every pixel on the sensor, enabling a measurement of the relative response, or gain, of each. In space, the only available sufficiently stable light sources are the Sun and the Moon, which are used for the small- and large-aperture flat-field calibrations respectively. In both cases, a slice near the center of the solar or lunar disk is scanned in the along-slit direction from one edge of the imager to the other, while images are continuously captured at the instrument's nominal 14 Hz cadence used for ground-scene measurements. The flat-field calibration scans ±6.5∘ from the boresight, going 1.5∘ outside the HySICS's FOV in both directions to ensure full spatial coverage. At a scan rate of 5.88 arcmin s-1, the along-slit flat-field scan requires 102 s to complete, during which time both sources are considered stable even during high rates of change in lunar phase. (Diffraction from the small solar-viewing aperture spatially blurs the Sun so that fine detail due to solar oscillations or granulation, which vary on 5 to 10 min timescales, are not observable and sensitivity to pointing errors is small. The lunar flat-field scan, however, is more sensitive to pointing and slit-alignment errors because of large intensity-variations across the lunar crescent and – unlike viewing the Sun through the small aperture – there is very little diffraction to spatially blur the image.) The resulting data are a series of images containing the solar or lunar spectrum stretched across the full sensor in the spectral direction and gradually moving through the sensor's entire spatial direction with each successive image. This technique essentially sweeps an identical spectrum across the spatial direction of the array, by which the spatial-direction flat-fielding is accomplished. (The spectral-direction flat-fielding is done at a later data-processing stage when the HySICS measurements are calibrated to the independently known SSI.) Since both sources extend over multiple spatial pixels, the pixel-to-pixel signal comparison can be repeated multiple times and utilized in measurement averaging as well as providing a basis for uncertainty estimations. A total of 31 spatial positions across the solar disk are applied to the flat-field correction, while only 9 positions across the narrow lunar crescent during Flight 2 are used. Uncertainties for select wavelengths are summarized in Table 2.

As with read noise, since the flat-field calibrations are acquired using static sources, they can benefit from multi-acquisition scans to reduce random uncertainties and at different integration times to improve signal in spectral regions having lower sensitivity such as the visible. These approaches, described in more detail in Sect. 4.2.6, were not performed for the flat-field calibrations of either the Sun or the Moon during Flight 2 and, as a result, the flight-acquired flat-field uncertainties dominate all others at the shorter wavelengths where instrument sensitivity is low. For flat-field calibrations using the Sun, cross-slit scans of which did benefit from multi-acquisition scans at different integration times and thus have low uncertainties for most other parameters, the flat-field uncertainties dominate at the shorter wavelengths and are comparable to diffraction at the longer wavelengths, so would greatly benefit from multi-acquisition scans. Lunar flat-field calibrations were marginal because of the high lunar phase during the time of the flight, giving low lunar signal and small spatial extent. These along-slit lunar scans are not only low in signal, but very sensitive to pointing, particularly since the large aperture used for lunar flat-field calibrations does not spatially blur the lunar image due to diffraction as the smaller aperture does to the solar image. Where the acquired flat-field uncertainties exceed the array's intrinsic 3.3 % pixel-to-pixel variations, such as in the shorter-wavelength portion of the visible, they were clipped at this intrinsic value.

Multi-acquisition flat-field calibrations at different integration times for both the Sun and the Moon and a lower lunar phase-angle would greatly improve the uncertainties demonstrated by Flight 2. Nevertheless, in spectral regions having high signal, the flat-field uncertainties acquired during this flight are < 0.2 %, demonstrating the flat-field calibration method capabilities and showing promise of achieving desired lower uncertainties across a broader spectral range with the suggested multi-acquisition approach.

Flat-field corrections are applied to ground scenes and cross-slit solar scans, and thus these uncertainties directly affect those data. Measurements that rely purely on relative measurements, such as calibrations of aperture ratio (Sect. 4.3.1) and filter transmission (Sect. 4.3.3), are not affected by these flat-field uncertainties.

Shot noise arises from photon-counting statistics and varies as the reciprocal square root of the signal, including that from any thermal background. As with read noise, it is reduced via multiple-image acquisitions for calibrations of static sources, namely the Sun and the Moon, but cannot be similarly reduced for single-acquisition images of the ground. Shot noise is the dominant source of uncertainty for ground scenes across the majority of the spectrum.

HySICS directly measures outgoing Earth-reflected shortwave radiances and incoming solar radiances. By spatially integrating radiances from the entire solar disk, which are acquired from cross-slit scans of the Sun, the HySICS measurements are calibrated to the independently known incoming SSI. To provide an accurate spatial integration, HySICS data analysis needs to correct for radiative losses, such as due to stray light and diffraction, that may cause differences in the amount of light reaching the FPA when viewing the Sun as opposed to ground scenes. Losses from diffraction are higher for the solar-viewing configuration than for ground viewing because of the smaller aperture used for solar observations. (Figure 2 illustrates the noticeably larger diffraction that must be accounted for when using the 0.5 mm solar aperture compared to the 20 mm Earth-viewing aperture. At 1000 nm, the diffraction limit from each is ∼ 8 and 0.2 arcmin, respectively.) Spatial scans sweeping across and then well away from the Sun provide scatter and diffraction characterizations. These, in addition to lab measurements of the same, enable corrections to facilitate accurate determinations of the net SSI based on cross-slit scans of the Sun.

Diffraction can be modeled well with a NIST-quoted uncertainty of ∼ 1.8 % for simple circular-aperture geometries (Shirley et al., 2002). The effects of scatter, however, are very instrument-specific and can be more difficult to model in advance to sufficient levels of accuracy. Lab measurements using the setup shown in Fig. 3 helped determine these contributions by characterizing the HySICS system's diffraction and scatter properties. This experiment occults the light coming directly through the aperture, but captures most of the light scattered or diffracted from it, and then re-images that light onto a separate FPA. Sample results are shown in Fig. 4 and match the expected angular dependence due to diffraction alone, indicating that diffraction, as opposed to scatter, is the dominant source of this indirect light for the as-built HySICS.

By modeling the diffracted light and verifying the model with lab measurements, the expected light losses are accounted for when spatially integrating the solar disk to obtain a value that can be correctly calibrated to the independently known SSI. A 1.8 % uncertainty on this correction is allocated as per NIST diffraction-estimate uncertainties. Because diffraction scales with wavelength, these corrections begin to dominate the solar-calibration uncertainties at the longer wavelengths but never greatly exceed the contributions from read and shot noise. The HySICS's small solar-viewing aperture was chosen such that uncertainties due to diffraction may be the limiting uncertainty at the longest wavelengths but would not dominate across the spectrum, effectively balancing desirable greater-attenuation capabilities afforded by smaller apertures with the increased uncertainties expected from them. This balance established the HySICS attenuation levels achievable via aperture ratios to ∼ 10-3.

Since radiometric uncertainties are dependent on the product of the instrument's spectral accuracy and the derivative of the measured spectrum, knowledge of, and corrections for, the spectral scale (or wavelength position) are characterized and applied.

Intermittent measurements using the HySICS's internal pen-ray lamp throughout the flight allow spectral calibrations based on this narrow-band source to verify spectral-scale accuracy or correct for possible wavelength-position fluctuations due to thermal or mechanical changes. Independent control of the three TECs regulating optical-bench temperature reduced thermal gradients during Flight 2 and thus reduced variations in the spectral scale. The spectral scale when at altitude shifted by only 3 nm, with variations across all wavelengths being maintained to < 1 nm across the spectral range.

The spectral corrections were interpolated to the times of observations. Of particular importance are the corrections at the times of solar calibrations, as the Sun has more abrupt spectral variations than ground scenes, and uncertainties in the spectral scale near the edge of a large spectral variation can give a correspondingly large radiometric uncertainty at wavelengths near spectral lines. Since the HySICS uses an FPA that spectrally bins the incident light from the spectrometer into 3 nm regions defined by the size of the FPA pixels, small potential spectral shifts in the incident light coupled with large spectrally dependent changes in signal near the sharp edges of these pixel-defined spectral bins can affect radiometric uncertainties.

Both the effects of this pixel-delineated spectral binning and those from thermal or mechanical instrument distortions are included in estimates of the HySICS's wavelength-position uncertainties, which are plotted as a function of wavelength in Fig. 6 for solar observations and in Fig. 7 for ground-scene measurements. Because of the in-flight spectral-calibration corrections via the internal pen-ray lamp, wavelength-position uncertainties are rarely the dominant contributor to the net radiometric uncertainties, although they do increase at the shortest wavelengths, where the Sun has more spectral-absorption lines, as well as near 820 nm, where the Sun has several absorption lines and the HySICS has low sensitivity.

For a spaceflight instrument regularly acquiring Earth observations, the spectral scale determined by the pen-ray calibrations could be validated by select Earth-atmospheric spectral lines under certain viewing conditions to help distinguish them from surrounding spatial or spectral features, such as by observing these lines from uniform bright background clouds or dark oceans or viewing them near the Earth limb by off-pointing from nadir. Oxygen molecules provide some such possible spectral lines, with one HySICS balloon flight even showing an O2 line in emission against darker space in an Earth-limb scene. Although spectral-scale corrections are a small source of uncertainty and predominantly affect solar calibrations rather than Earth observations, such Earth-atmospheric spectral-line observations could help verify the instrument's in-flight spectral scale during normal observations.

The FPA has a background-level offset that varies linearly with the measured signal. This offset is detectable by observing the extreme-most ultraviolet spectral column of the sensor, which, at 320 nm, is below the reflectivity cutoff for the instrument mirrors and, therefore, is effectively a dark column. All pixel values in this column should remain nearly constant, showing mainly dark-current and fixed-pattern noise. Instead, they consistently decrease by up to 120 DN when other portions of the array are observing extremely bright signals. This background-level decrease is also observed in all dark pixels during a solar scan, including columns neighboring the dark column as well as regions of the sensor viewing dark space up to 9.5∘ away from the Sun. Lab measurements of FEL and LED sources show similar effects.

This “brightness offset” of the background level, as measured on the dark column, is characterized using flight data. A matrix of background-level reduction versus sensor signal is generated from all large power-level transitions during the flight, such as when the solar disk moves out of the instrument FOV during a flat-field scan or when it comes into- or out-of-view during an irradiance scan. The amount of background-level reduction is linear with the amount of light detected by the sensor, regardless of its spectral distribution or spatial location, with the background level changing by -2.3 × 10-7 DNs per DN of signal. Because there is some dependence on the integration time of the sensor, this slope was determined for all integration times used during flight. The slope has an uncertainty of 14 % based on the standard deviation from multiple background-level characterization measurements. However, being as the brightness offset itself is a small correction, this does not directly translate into a similar-magnitude contribution to the overall radiometric measurement uncertainties.

The resulting brightness-offset corrections, which are dependent on the total signal on the sensor as well as the integration time, are applied to each image acquired. The relative uncertainties in this correction are greatest for measurements having low signals, so they predominantly affect Earth ground scenes, where they are generally the second-largest contributor to net uncertainties.

Accurate radiometric measurements of scenes having unknown polarization rely on the instrument having low polarization sensitivity (Lukashin et al., 2015). The HySICS was designed to reduce polarization sensitivity by orienting the optical plane of the 4MA perpendicularly to that of the spectrometer, such that reflection-induced diattenuation in the former is nearly offset by that in the latter. This was effective with the exception of the custom-ruled grating, the primary HySICS optical component that did not meet expected performance. Along with having low efficiency in the visible, polarization tests of this grating showed a much larger sensitivity than anticipated, with the net instrument-diattenuation results plotted in Fig. 5. Despite the orthogonal orientation of the 4MA to the Offner optics, this grating limits the instrument's desired low polarization sensitivity, particularly in the near infrared.

If measuring randomly polarized scenes, this internal-instrument polarization sensitivity has no effect on radiometric accuracy but, for scenes of unknown polarization amplitude and orientation, the radiometric uncertainties can potentially be as large as the instrument's diattenuation itself in the specific – albeit highly improbable – case of a 100 % polarized incident signal oriented along, or perpendicular to, the direction of the instrument's greatest polarization sensitivity.

Integrated-instrument uncertainties showing the effects described above are plotted in Fig. 6 and tabulated for select wavelengths in Table 3 for spatially integrated cross-slit observations of the Sun and in Fig. 7 and Table 4 for sample single-acquisition measurements of bright and dark ground scenes. These two figures indicate the uncertainties on the measurements Smeas_SSI(λ) in Eq. (2) and Smeas_obj(λ) in Eq. (1), respectively.

With the exception of the flat-field and diffraction uncertainties, the solar-scan uncertainties benefit from multiple-image acquisitions and a dual-scan approach. Multiple, repeated measurements of the same scene particularly reduce the effects of read and shot noise and from the brightness offset caused by the small vs. large apertures at the shorter wavelengths where the HySICS's response is lowest. Two back-to-back scans of the Sun, one using longer integration times to increase signals at wavelengths shorter than 850 nm and one with integration times better matched to the higher signals at longer wavelengths, followed by spectrally combining the scans in post-processing improves the signal in select portions of the spectrum. The improvements from multiple image acquisitions and the dual-scan approach are possible only because the Sun can be viewed repeatedly with the same instrument look-angles, so it provides a static in-flight calibration source. These techniques would also be applicable to reducing flat-field uncertainties but were not performed on Flight 2, so the solar-calibration results shown are dominated by the flat-field uncertainties.

Balloon-flight motions over the ground prevent applying these beneficial uncertainty-reduction techniques to ground scenes, so uncertainties must be based on single-image acquisitions. Despite the larger aperture and the longer integration times for ground scenes, the lower radiances of these single images have larger relative uncertainties than those from the Sun, since they do not benefit from multi-image or dual-scan techniques. Typical net uncertainties from representative bright (cloud-filled) and dark (desert- and vegetation-filled) ground scenes are plotted in Fig. 7; these are the net scene-dependent uncertainties in the measurement factor Smeas_obj(λ) from Eq. (1). The dominant uncertainties are from shot and read noise, with the brightness offsets with flat-field uncertainties dominating at shorter wavelengths. Note that while the dark scene has higher uncertainties across much of the spectral region, at the longer near-infrared (NIR) wavelengths, it has slightly greater overall signal and therefore lower uncertainties, since the darker ground scenes emit more infrared radiation than the brighter (in the visible) scenes from colder clouds, although these are still large due to the very low reflectance signals at these wavelengths.

The baselined 10-3.2 attenuation due to aperture-area ratios was demonstrated, with measured results plotted in Fig. 8. This figure also gives the corresponding uncertainties as a function of wavelength, while Table 5 details a breakdown of their contributing components. Although this aperture-ratio attenuation method relies on geometry and so should be nearly spectrally flat, at shorter wavelengths the method causes much more than the 10-3.2 attenuation expected from aperture-area ratios alone.

The small and large apertures respectively used for the Sun and Earth measurements illuminate different areal portions of the HySICS optical surfaces and thus have different throughput efficiencies that must be accounted for when transferring the solar-based radiometric scale to radiances from ground measurements. While most optical surfaces are sufficiently uniform or similarly illuminated to not be greatly affected by these different areal-illumination effects, the spectrometer grating is the dominant cause of current HySICS spectrally dependent efficiency variations between the two aperture-illumination regions.

To achieve the broad spectral range and high throughput efficiencies required with a single-spectrometer design, varying grating-blaze-angles are needed. The fabricated balloon-flight grating contains a sawtooth pattern of four discrete regions, with the blaze-angle varying monotonically across each. The small aperture used for solar measurements illuminates a boundary between two such regions to a much greater proportional degree than the larger aperture does, so it is more sensitive to symmetric alignment on this boundary. A slight misalignment on the edge of this “tooth” in the sawtooth grating pattern will preferentially favor the corresponding extreme-blaze-angle at the edge of the region in that misalignment direction, thus making the system more sensitive to either the shortest or longest wavelengths. The relative throughput for the small and large apertures was characterized in lab measurements with the results shown in Fig. 8. These effects are accounted for in HySICS's radiometric results as part of the aperture-ratio portion of the full attenuation-system correction, A(λ). The spectral dependence shows much lower efficiency in the visible and higher efficiency in the NIR, suggesting the small region of the grating illuminated by the solar-viewing aperture is biased toward the NIR-blaze edge of one grating region rather than equally split between the two it straddles.

These laboratory calibrations were performed using two light sources, with one peaking in the visible and the other in the NIR, to span the full spectral region. The intermediate visible-to-NIR spectral region had low intensity from both lamps. Combined with the strong increase in attenuation and resulting lower intensities at shorter wavelengths when using the small aperture, these low light-source intensities limited the relative uncertainties in this visible-to-NIR spectral region, resulting in the large uncertainty peak shown in Fig. 8. Further calibrations with a broader range of bright lamp sources, particularly near the visible-to-NIR transition, could improve the uncertainties shown. More significantly, reducing the large spectral dependence of this aperture-ratio correction by using smoothly varying but non-monotonic blaze-angles via a more expensive custom-made grating rather than the four-region sawtooth one used here, should reduce much of these aperture-ratio uncertainty issues in the first place and is planned for a future HySICS instrument.

The aperture-ratio attenuation technique is inherently nearly independent of wavelength. That HySICS demonstrated the technique to well less than the needed uncertainties through most of the NIR spectral region shows promise that this attenuation method would be equally applicable over the entire spectral range with a more uniformly blazed grating and further laboratory characterizations.

Correcting for non-linearities while varying the FPA's electronically controlled integration times was more successful than initially anticipated, achieving a demonstrated attenuation of > 10-3 with a 0.05 % uncertainty for generally used exposure levels and a maximum of 0.12 % uncertainty that could accommodate extremely bright Earth-scenes. As described in Sect. 4.1.4, these corrections rely on characterizations of the electrically controlled integration-timing signals and the FPA's resulting response to various saturation levels.

The electronic timing signals show deviations from linearity that are < 2 × 10-7 for integration times from 16.8 µs to 8.62 ms, spanning a range of 102.7 in integration times, and < 1.6 × 10-6 over the full range from 16.8 µs to 35.23 ms, spanning a 103.3 range. These timing-signal deviations from linearity are relatively insignificant.

The results from the characterizations of the FPA's response described in Sect. 4.1.4, whereby the FPA's signal levels are determined from multiple, repeated measurements of an input FEL-lamp source at different exposure times, are shown in Fig. 9. Since the electronic shutter has nearly negligible non-linearity across this range, these deviations from linearity that manifest mainly at greater exposure times (i.e. greater signal levels) are due to non-linearities in the detector response and/or readout-amplifier electronics. The average of the deviations plotted in Fig. 9 (lower graph) provides the applied non-linearity correction as a function of detector signal, and the standard deviations about this average give the corresponding uncertainties in the applied non-linearity correction. The corrections are measured to be < 0.1500 % for almost all pixels over an intensity range of > 103, and the reproducibility of each pixel's response is generally < 0.05 % for intensities up to 53 % (35 000 DN) of full scale, which accommodates the brightest Earth scenes viewed during Flight 2. This attenuation method can also accommodate higher-intensity scenes, allowing up to 75 % of the FPA's full scale to be utilized while maintaining uncertainties to < 0.12 %. The corresponding uncertainties are shown in Fig. 10 and tabulated in Table 6.

The greater-than-anticipated attenuation range and the lower-than-anticipated uncertainties due to this integration-time attenuation method allow flexibility in the attenuation amounts needed by the other two attenuation methods. The integration-time attenuation capabilities provided by this flight-capable FPA eliminate the need for attenuations via filters altogether, which reduces mass, cost, complexity, and power for a future flight instrument.

Spectral filters were calibrated during Flight 2 using both the Moon and the Sun. The filter attenuation method demonstrated the desired attenuation range of 10-0.9, but with higher than the anticipated 0.05 % uncertainty due to the low lunar-signal levels at the time of this flight. As explained in Sect. 4.1.6, the narrow lunar-crescent illuminated only a few pixels on the HySICS FPA, resulting in read-noise-limiting filter-calibration uncertainties at wavelengths less than 900 nm (see Fig. 11). Since the filter calibrations are acquired while viewing stationary sources, namely the Sun and the Moon, multiple images and different integration times could be used to reduce noise in low-signal portions of the spectrum, although only single integration times were used for the calibrations during Flight 2. This operational improvement would substantially reduce the uncertainties in the visible spectral region shown in Fig. 11. Nevertheless, the filter uncertainties, particularly in the near-infrared, are already lower than those from the other two attenuation methods (see Table 7), although this method does not provide nearly the attenuation range of either of those other two. Fortunately, the large attenuation range provided by the integration-time attenuation method likely makes this entire filter-attenuation system unnecessary.

Section 4.1 and 4.2 explain the intrinsic imaging-measurement uncertainties from Sun and Earth scenes, Smeas_SSI(λ) in Eq. (2) and Smeas_obj(λ) in Eq. (1), while Sect. 4.3 adds uncertainties from the HySICS's attenuation-system calibrations, A(λ), which relate the relative signals of the Earth scenes to those of the Sun via the applied attenuation amount. This Earth-to-Sun ratio effectively gives the (unit-less) reflectance of the Earth scene. The uncertainty in the ratio includes uncertainties from the cross-slit solar-disk scans (Fig. 6), the Earth images (Fig. 7), and the attenuation systems applied (Fig. 8 and Table 6, since the filter-attenuation system was not utilized for the results presented here). Being independent, these uncertainties can be root-sum-squared to give the net Earth-to-Sun ratio (reflectance) uncertainties shown in Fig. 12 for both bright and dark ground scenes.

The solar cross-calibration techniques achieved a radiometric uncertainty of nearly 0.3 % across a large spectral region longward of 1000 nm from a bright ground scene, demonstrating a ∼ 6 × improvement over current spacecraft uncertainties and the capability of the approach to achieve the desired radiometric accuracies. The net uncertainties shown in Fig. 12 are dominated by spectral regions where there is very little power from the reflected-Earth radiation or very low HySICS sensitivity, such as the 1800 to 2200 nm and shorter visible spectral regions, respectively, and by the increased uncertainties from the calibration of the aperture-ratio attenuations near the visible-to-NIR transition spectral region. (The latter can be improved with further calibrations and/or an improved instrument grating, as detailed in Sect. 4.3.1, and the former partially by improved flat-field calibrations, as explained in Sect. 4.1.6) At most NIR wavelengths, the uncertainties are dominated by read and shot noise in the ground scenes. The uncertainties shown are characteristic of individual pixels, and spatial or spectral binning could allow yet further reductions in uncertainties.

A globally averaged, all-sky estimate of Earth-reflected irradiance over the 8-year period from 2003 to 2010, based on results from observation-system simulation experiments generated using SCIAMACHY data (Y. S. Shea, personal communication, 2016), is plotted in Fig. 12 (gray) to indicate a typical, realistic reflected-solar (RS) spectrum observed by a spaceflight hyperspectral imager. Weighting the HySICS's net radiometric uncertainties by this estimated RS spectral-irradiance gives the resulting spectrally averaged radiometric uncertainties stated in the (black) figure text. These are higher than ultimately desired, which is largely caused by low instrument efficiencies and high flat-field uncertainties in the visible as well as increased aperture-ratio attenuation uncertainties near the visible-to-NIR transition. Improving these via the methods described in Sect. 4.3.1 and extending the multiple-image acquisition and a dual-scan approach to flat-field calibrations should reduce the weighted, Earth-reflected HySICS uncertainties for a future instrument by another ∼ 4 × improvement over the values demonstrated here.

The high-quality data from Flight 2 with all instrument-level and attenuation-method corrections applied and with a final calibration factor, C(λ) in Eq. (1), to provide an SI-traceable radiometric scale, enables the creation of three-dimensional spatial/spectral data cubes from ground scans that represent the end product of the HySICS solar cross-calibration technique. With sufficiently low uncertainties in the three factors in Eq. (1), this results in hyperspectral ground images with lower SI-traceable radiometric-accuracy uncertainties than existing flight instruments provide. The details of acquiring this final calibration factor are described in this sub-section.

Multiple images as the solar disk is scanned in the cross-slit direction are spatially integrated to give a net spectral solar irradiance with corrections to account for the spectrometer's NIST-calibrated slit width as well as image overlap during the cross-slit scan. This irradiance is corrected for the diffraction and scatter described in Sect. 4.2.2 as well as other instrument effects described above. At this stage, the spectral “irradiance” is in units of instrument data numbers (DNs) and has no traceability to normal physical units. Figure 13 shows the SSI determined from HySICS using a cross-slit scan of the solar disk during Flight 2.

By knowing what the actual SSI is, the HySICS instrument DNs can be converted to useful physical units and the overall instrument sensitivity can be determined. SSI values from Lean's NRLSSI2 model were applied to the HySICS Flight 2 data, since they were available prior to measurements from any on-orbit instrument. These values, which account for the solar activity state on that day, were adjusted from their as-provided 1-AU distance to the actual Earth–Sun distance on the date of the flight. They are plotted in Fig. 13 and provide the transfer to realistic physical units.

The ratio of this model-based “actual” SSI to the HySICS's measured irradiance in Fig. 13 gives the instrument sensitivity via a conversion from DNs to physical irradiance units via correction factor C(λ), as shown in Eq. (2). This conversion factor is plotted in Fig. 14. The low instrument sensitivity in the visible is mainly the result of low efficiencies of the FPA and the grating at these wavelengths.