Merging fluxgate and induction coil data to produce low-noise geomagnetic observatory data meeting the INTERMAGNET definitive 1 s data standard

For frequencies above 30 mHz the instrument intrinsic noise level of typical fluxgate magnetometers used at geomagnetic observatories usually masks ambient magnetic field variations on magnetically quiet days. This is especially true for stations located at middle and low latitudes, where variations are generally smaller than at high latitudes. INTERMAGNET has set a minimum quality standard for definitive 1 s data. Natural field variations referred to as pulsations (Pc-1, Pc-2, Pi-1) fall in this band. Usually their intensity is so small that they rarely surpass the instrumental noise of fluxgate magnetometers. Moreover, highquality magnetic field observations in the band 30 mHz– 0.5 Hz contain interesting information, e.g., for the study of ionospheric electron interactions with electromagnetic ion cyclotron plasma waves. We propose a method to improve 1 Hz observatory data by merging data from the proven and tested fluxgate magnetometers currently in use with induction coil magnetometers into a single data stream. We show how measurements of both instruments can be combined without information loss or phase distortion. The result is a time series of the magnetic field vector components, combining the benefits of both instruments: longterm stability (fluxgate) and low noise at high frequencies (induction coil). This new data stream fits perfectly into the data management procedures of INTERMAGNET and meets the requirements defined in the definitive 1 s data standard. We describe the applied algorithm and validate the result by comparing power spectra of the fluxgate magnetometer output with the merged signal. Daily spectrograms from the Niemegk observatory show that the resulting data series reveal information at frequencies above 30 mHz that cannot be seen in raw fluxgate data.

. Noise spectra of a magnetometer prototype and of measured natural geomagnetic variations (modified after Korepanov (2007)).
We added the high frequency noise level of the FGE magnetometer as measured at NGK (green line) and the high frequency noise of a typical induction coil magnetometer (red line). The blue dot shows the specification given in the "INTERMAGNET Definitive One-second Data Standard" (Turbitt et al., 2014). Instrument noise at very long periods from days to years are equivalent to instrument long term stability. This is the most important quality feature of an observatory fluxgate magnetometer. As described in the following, merging data of both instruments does not in any way effect the long term stability.

Combining fluxgate and induction coil magnetometer
We take advantage of the fact that the signal of an induction coil magnetometer is proportional to the time derivative of the 5 magnetic field component along the axis of the coil. In the following we consider exemplary the X-component. For each sample time t 0 we look at values X F G (t i ) registered by the fluxgate magnetometer at the times t i varying from t 0 − T s N to t 0 + T s N , T s being the sample width. Additionally we need recordings of the induction coil also at the times t i . We get as result of our algorithm the value B X (t 0 ) as a more precise value of the magnetic field at the time t 0 .
Integrating induction coil data provides a curve which is comparable to fluxgate data. The integration constant B X (t 0 ) is 10 the field at the start time t 0 of the integration. Due to the low noise content of an induction coil this curve is smooth compared to fluxgate data (blue line in fig. 2). Our approach is, to fit the former like a spline curve to the fluxgate data ( fig. 2 , right).
This effectively eliminates instrument noise of the fluxgate by averaging. However, the shape of the fitted curve results from the induction coil measurements and the loss of information due to averaging is replaced by information stemming from the induction coil.
For given time t 0 and a variable time t i in its environment, data of fluxgate X F G and induction coil magnetometerẊ IC can be expected to satisfy the following equation, assuming that both sensors are aligned with the X-axis: Assuming that the induced voltage measured over the induction coil is object to scale factor C and offset ∆U : The blue line in fig. (2, left) is achieved evaluating the integral from the fixed central time t 0 to a time t i with i varying from −N to +N applied to an actually measured induction coil data set. The according time series of a flugate magnetometer is given as green line. The blue line (IC) is much smoother than the green one (fluxgate data). Choosing the right values for the parameters ∆U , C and B X (t 0 ) it can be fitted like a spline to the fluxgate data, as shown in the right panel of fig. 2. For this fit, equation (2) has to be solved for ∆U , C and B X (t 0 ) as unknowns. Equation (2)  numerical accuracy of the result. But the time interval must be small enough to ensure not to distort the fluxgate signal at lower frequencies.
In chapter 4 we show, that a further improvement is not necessary because the signal is clearly resolved up to the Nyquist frequency of 0.5Hz.
Assuming that both data series are available at a sample rate of 1Hz and approximating the integral by a sum, equation 2 can be written in the discrete form Assuming that T s = 1sec, one equation can be derived for each sample time t 1 . If i varies from −N to +N we get the following conditional system for the parameters C, ∆U and X F G (t 0 ):

This is a system of linear equations of the Form
U IC (k) and written as explicit matrices: . . .
Setting N to 100 we get a time interval of 200sec leading to 201 equations. Obviously we have more than three conditional equations at hand and a solution must be found in the sense of minimizing the mean square of the residuals. The so called normal equations (Schmucker and Weidelt, 1975) More explicit for this case: is the field value at the time t 0 we are looking for. Note that the numeric effort for this calculation is very small. It is restricted to the inversion of a 3x3 matrix and calculating six sums in equation (5) for each sample of the resulting time series.
The sums do not even have to be evaluated for each new t 0 . They can be updated from the previous time step.

Results
The success of merging both instruments can best be demonstrated by comparing power spectra of unprocessed FGE and merged data and looking at spectrograms of the respective data series. In figure (3)   As shown in figure (4) the coherence between merged data and FGE data is perfect at low frequencies (< 0.01Hz). At high freqencies (>0.03Hz) the coherence is very good between merged data and IC data. It is not perfect due to the low energy 10 content of the signal at these frequencies. A very high coherence is also observed between FGE and IC data at frequencies between 0.003Hz and 0.03Hz. In this frequency band both instruments are best comparable, allowing to check alignment and intercalibration of both instruments. Induction coils are usually run at a GPS synchronized data logger. This allows to verify  is observed at low frequencies (< 0.01Hz). At high freqencies (>0.03Hz) the coherence is high between merged data and IC data (green). (Turbitt et al., 2014). For such a test a natural or artificial signal clearly surpassing the FGE white noise level is required.
Though a time stamp error of 0.01sec can hardly be verified on a data series sampled at 1/sec which is accordingly Nyquist filtered. wave) and sweeps with falling frequency starting at 6:30 and 9:00. Both are man-made contamination. We found a lot of other remarkable signals revealed in such spectrograms, but discussing them is beyond the scope of this paper.  Providing data to a user is generally linked to the commitment to the best possible accuracy. Speaking in terms of data series, this means to optimize the signal to noise ratio over the entire available band width. Hence our effort improves magnetic data in view of the INTERMAGNET one sec standard. We are well aware, that merging two data sets does not produce new information. Merging data from different instruments at geomagnetic observatories is not a far-fetched idea. On the contrary, 5 it is already standard technique when definitive data are produced by combining absolute observations from DI-theodolite and scalar magnetometer with vector variometer (fluxgate) data. As INTERMAGNET does not cover induction coil data, applying our method makes additional information available to the INTERMAGNET community. Plots as presented in fig. 5 could also be produced using both data sets separately. But after applying our method the single one-second data series as administered for many observatories worldwide by INTERMAGNET is sufficient. The effort of storing and exchange of data sampled at 10 1/sec as proposed by INTERMAGNET makes more sense if the entire possible bandwidth contains information about the Earth magnetic field. However, in order to take full advantage of the high frequency information, the data should be reported to at least three decimal digits (1 pT). Otherwise quantization noise becomes visible. The quantization noise level due to truncation is shown in figure (3).
The INTERMAGNET one second standard prescribes a hardware low-pass filter with a corner frequency of 0.2 Hz for the 15 fluxgate data. This filter has absolutely no effect to the result of the merging process. The high frequency information stems from the induction coil side only. We verified this by numerical experiments.
The induction coil data should better be sampled at a rate higher than 1/sec and subsequently been filtered and down sampled numerically in order to best avoid aliasing into the Nyquist bandwidth of the fluxgate data. Variometer data are often explored in the time domain. The morphologies of events like sudden storm commencements, sub-storms or pulsations are investigated by 20 visual or automated inspection. In merged data events with even very small amplitudes can be observed, as noise is eliminated to a large extent.
It suggest itself to benefit from the possible high sample rate of an induction coil and to extend the bandwidth up to the 50Hz grid frequency limit. The method can easily be extended accordingly. But this is beyond the scope of this paper.
We have to assume some preconditions for the application of our method:

25
-We made the assumption, that the sensor of the fluxgate and the induction coil magnetometer are perfectly aligned. If this is not the case, the induction coil signal will be affected by the other components. The fitting of both data will become worse. This could be accounted for by numerically rotating the data beforehand. For the determining the rotation matrix a larger data set is needed. But induction coils can be used down to relatively low frequencies, so that the overlap with the fluxgate is broad enough to determine the misalignment.

30
-We assume that the induction coil output is proportional to the time derivative of the magnetic field. Induction coils normally have a ferrite core and use electronic amplification (Pulz, 2010). Both can influence the linear transfer function in the used frequency band from 30 mHz to 1 Hz. This effect could be compensated by an according numeric amplitude pre-filtering. This point is especially important, if the method shall be extended to higher frequencies. 9 Geosci. Instrum. Method. Data Syst. Discuss., doi:10.5194/gi-2017-15, 2017 Manuscript under review for journal Geosci. Instrum. Method. Data Syst. In the process of curve fitting depicted in fig. 2 it is a good idea to weight points in the direct environment of the time of interest t 0 higher than remoter ones. We made good experience with a Gaussian filter. For the sake of simplicity we did not include weights in the equations (3 to 5). In a final step the intermediate result is subtracted from the fluxgate data to get the merged output. The output contains the low-pass-filtered fluxgate data (the high-pass content has been subtracted) and the integrated, high-pass-filtered induction coil 10 data. That means, the high frequency content of the fluxgate, containing mainly noise, is eliminated in the resulting data and replaced by the according high frequency part of the induction coil side.
An advantage of the curve fitting method is, that the scale factor of the induction coil is automatically corrected. The numerical effort of the curve fitting method may be slightly higher. But the effort is restricted to solving one 3x3 matrix for each data sample. Extending the curve fitting method to a higher sampling rate of the induction coils (e.g. up to 50Hz) is 15 straightforward.

Conclusions
We have shown how data of fluxgate magnetometer and induction coil data can be merged. A numerical method based on the idea of fitting integrated induction coil data to fluxgate data has been presented in detail. The merged data set combines the excellent long-term stability of well-proven observatory fluxgate magnetometers with the low noise content of induction coil 20 magnetometers at high frequencies. The resulting merged data differ from unprocessed fluxgate data only within the range of the intrinsic fluxgate noise (<0.1nT). Hence the long term stability is in no way affected. We produce a data set that has the format of usual one second fluxgate data but surpasses the specification of the "INTERMAGNET Definitive One Second Data Standard" by far. The resulting data set reveals details like Pc-1, Pc-2 and Pi-1 pulsation of even small intensities which are not resolved by a fluxgate magnetometer. The spectral noise density in the resulting data set is up to the Nyquist of 0.5Hz clearly lower than the spectral density of the natural magnetic field even at days with low activity. Thus with our method we can fill the entire possible bandwidth resulting from the "INTERMAGNET Definitive One Second Data Standard" with meaningful information about the Earth magnetic field.