In geomagnetic observatories, variometers are employed to record continuous variations of the geomagnetic field. These variations are subsequently converted into absolute geomagnetic field values through the addition of baseline values derived from absolute measurements (Jankowski and Sucksdorff, 1996). This calibration process renders absolute measurements critical for ensuring the quality of continuous absolute geomagnetic data. However, the difference of absolute instruments between different observatories make systematic instrument comparisons as an essential component of modern geomagnetic observation systems.

Contemporary absolute measurements primarily utilize two high precision instruments: (i) fluxgate theodolites (designated as Declination Inclination Magnetometers, DIMs) for measuring declination (D) and inclination (I), and (ii) proton magnetometers for total field intensity (F) determinations. While technological advancements have reduced the required frequency of instrument comparisons, such calibrations remain indispensable for maintaining high quality geomagnetic datasets (Zhang et al., 2024). To standardize global geomagnetic observations, the IAGA Division V Working Group V-OBS has successfully organized over twenty biennial international instrument comparison sessions to date (Loubser, 2002; Masami et al., 2005; Reda and Neska, 2007; Love, 2009; He et al., 2011; Hejda et al., 2013; and so on). In China, the Geomagnetic Network Center (GNC) integrates these comparisons into its quality control framework, serving dual roles as both data hub and quality assurance authority for national observatories (Li et al., 2012; Zhang et al., 2016).

Since the digital transformation of Chinese geomagnetic observatories, the GNC has implemented successive generations of DIMs, including Hungarian MINGEO-DIM, British MAG 01-DIM, Chinese CTM-DIM, Chinese GEO-DIM, Chinese TDJ2E-NM-DIM, etc. It should be noted that the manual operation inherent to these DIM systems introduces two critical uncertainty sources: (i) inter instrument systematic biases, and (ii) operator dependency operational variances. To mitigate these effects and unify observational standards across the geomagnetic network, the GNC has organized instrument (DIMs) comparison measurements almost annually since its digital transformation (He et al., 2019b). Through systematic comparison measurements, each observatory's instrument is calibrated against the reference instrument (DIMs), which is typically designated as the GNC standard and has the characteristics of high measurement accuracy and operational stability, to quantify instrumental differences and achieve nationwide standardization of absolute geomagnetic observations.

The following sections will first briefly introduce the measurement methods used in comparison measurements and the baseline based comparison methods. Then, the research will primarily focus on the historical comparison datasets. Firstly, statistical analysis will be conducted to explore valuable information from the statistical results. Secondly, uncertainty analysis method, widely used in experimental science, is applied to further quantify and extract relevant information from the datasets, to evaluated the uncertainty of the reference instrument relative to the observation results of all instruments in each comparison session. So the long term robustness of the reference instrument can be investigated based on multiyear results.

The geomagnetic field, being a vector quantity, requires precise determination of both magnitude and directional components. The DIM, comprising a theodolite integrated with a fluxgate sensor, serves as the standard instrument for determining geomagnetic field direction (declination D and inclination I). The fluxgate sensor, mounted parallel to the theodolite's optical axis, operates on the null detection principle: it generates zero output (assuming zero offset) when aligned perpendicular to the geomagnetic field vector. Directional determination is achieved by identifying sensor null positions, with angular coordinates recorded via theodolite circle readings. The angular between two directions can be determined by computing the difference between their respective readings on the instrument's horizontal circle. This is the fundamental principle of fluxgate theodolite angle measurement.

The fluxgate theodolites are high-precision instruments, but they inevitably contain certain errors, such as misalignment errors between the mechanical axis of theodolite, the optical axis of the telescope, and the magnetic axis of the fluxgate sensor; collimation errors; non-orthogonality errors of the horizontal and vertical axes; uneven graduation errors of the reading circle; index errors; and errors caused by non-zero electronic offsets, which prevent accurate determination of magnetic declination and inclination from a single reading of the horizontal/vertical circle (Lauridsen, 1985; Newitt et al., 1996; Csontos and Šugar, 2024). However, in theory, most of these errors can be eliminated through the four position measurement process, and some of them (two misalignment errors between the fluxgate sensor axis and the optical axis of the telescope in the horizontal/vertical planes, and the offset error of the fluxgate sensor) can be calculated from the measurement results (Bitterly et al., 1984). The specific measurement methods and procedure are described in Appendix A1.

Ensuring high quality geomagnetic observations necessitates systematic verification of inter-instrument differences across observatories. The comparison analysis of absolute measurement instruments constitutes an essential quality assurance measure in geomagnetic monitoring networks. Practical constraints, including limited pillar availability, multiple DIMs, and operator skills proficiency, render synchronous multi instrument comparisons operationally challenging. Modern variometers exhibit high precision performance with quasi constant baseline characteristics under stable operating conditions, while underground observation rooms of geomagnetic observatories (far from cities or villages) can provide such operating conditions, including no influence of magnetic objects, low electromagnetic background noise, indoor annual temperature variation not exceeding 10 °C, daily variation not exceeding 0.3 °C, and so on. The current comparison protocols therefore employ independent absolute measurements followed by baseline value cross validation. The stability and accuracy of baseline values during a single calibration day were investigated by Zhang and Yang (2011), whose study confirmed that baseline values remained stable throughout the calibration period (08:30 to 16:30 local time), with geomagnetic activity exhibiting no significant impact on their accuracy. Consequently, direct comparison of baseline values is a valid approach for completing the analysis. The baseline calculation and instrument comparison methodology are presented in Appendix A2.

In each instrument comparison, all fluxgate theodolites instruments are brought to a specific observatory with excellent observation environment and compared with the reference instrument designated by the GNC. The reference instrument consists of a Zeiss 010B theodolite and a fluxgate sensor, with accuracies better than 1 arcsec and 0.1 nT, respectively. The other instruments, known as testing instrument, come from 46 observatories of GNC, including five types as mentioned earlier. The locations of all observatories and the specific observatory (code QIX highlighted in blue) for comparison are shown in Fig. 1. The codes of all the observatories and their corresponding instrument types, as well as the number of times each instrument participated in comparison and the number of operators (with non-repeated counts), are listed in Table 1. The No. 0 in Table 1 is the reference instrument. The relevant information and parameters for five types of fluxgate theodolites are provided in Table 2.

Based on the aforementioned methodology, the GNC has completed 12 comparisons of DIMs from 2010 to 2024 (no comparison was conducted in 2011, 2013, and 2021), and corresponding datasets have been accumulated. The datasets for comparing all testing instruments with reference instrument are displayed in Fig. 2, where panel (a) and panel (b) respectively illustrate the instrumental differences in declination (ΔUD) and inclination (ΔUI). Colored dots in this figure represent measurement results from different years, with vertical coordinates indicating instrumental differences, while the dot sizes indicate the standard deviations of measurements, scaled according to the legend on the right. This graphical representation enables a comprehensive evaluation of observational data quality at both individual instrument and the network levels.

This figures also provide insights into instrument performance and operator proficiency. Small dots with large central values indicate significant instrumental differences, suggesting potential instrument malfunctions or operational issues by personnel. Particularly when the difference of D is relatively large while that of I is small, the most likely cause is the positioning error of the theodolite on the pillar, which also reflects technical shortcomings in observational practices. Conversely, large dots with small central values signify dispersed data, which could arise from instrument related issues (e.g., unclear optical paths affecting reading accuracy) or inconsistent operational practices. More detailed explanations for tracking abnormal information can be found in He et al. (2019b). This graphical approach thus effectively monitors instrument performance and evaluates observational quality across operators.

To assess of network wide data quality, we conducted statistical analyses of all instrument differences, as shown in Fig. 3. The statistical result reveals that declination (D) and inclination (I) measurements approximate normal distributions, with means of 0.00^′ and 0.02^′, and standard deviations of 0.13^′ and 0.07^′, respectively. Approximately 75.1 % of declination and 86.8 % of inclination measurements fall within ±1σ of the mean. When adopting a cumulative probability of 90 % as the evaluation criteria for the entire network, the corresponding instrument differences thresholds are 0.21^′ for D and 0.11^′ for I, indicating excellent consistency among network fluxgate instruments. Notably, declination measurements exhibit greater dispersion than inclination values. This difference stems from the additional azimuth marker alignment required in declination measurements – a process more susceptible to operator error compared to inclination measurements. Another significant source of possible error in declination readings, which is not present in inclination readings, is the accuracy of setting (at 90 or 270°) on the vertical circle.

The dispersion of multiple dots corresponding to the same instrument also reflects its data quality and operation stability. Frequent personnel changes for same instrument have introduced operator dependency errors, manifesting as increased dispersion. To further explore the relationship between frequent personnel changes and the dispersion of instrument differences, the frequency personnel change was defined as the ratio of non-repeated operators to the total number of comparison measurements for each instrument (from Table 1), serving as the x axis, the dispersion degree was represented by the standard deviation of all instrumental differences for each instrument, serving as the y axis. To enhance statistical significance, only instruments that participated in 3 or more comparisons were included in the analysis. As shown in Fig. 4a, the dispersion degree of the instrumental differences increases with the frequent personnel changes, while this phenomenon is less pronounced in Fig. 4b. This indicates that frequent personnel changes increase observational errors and that personnel changes have a greater impact on D than on I. This result is consistent with the conclusion in the previous paragraph that D errors are larger than I errors. It further provides strong evidence supporting the explanation that operator errors primarily arise from the alignment of markers and the level adjustment of the theodolite.

The instrument differences of all 12 years' comparisons were classified to five group based on the instrument type, and the standard deviations were calculated for each group. This was done to simply compare the stability of observational results across different instrument types, as shown in Fig. 5. It can be clearly seen that MINGEO has better stability, followed by Mag-01H and TDJ2E-NM, while the other two have relatively large dispersion, which is directly related to the resolution of the theodolite.

This simplified comparison measurement provides an efficient mechanism for identifying inter station differences, monitoring instrument performance, and ensuring standardized high quality observations across the network. However, the efficacy of this mechanism critically depends on the precision and accuracy of reference instruments. Beyond routine maintenance and calibration of reference instrument, it is necessary to analyses long term stability and reliability. The following section will evaluate the reference instruments using all comparison measurement data through uncertainty analysis.

During the comparison process, fluxgate theodolites are employed to observe magnetic declination and inclination. To minimize errors, each observed magnetic declination and inclination is the results obtained after four measurement processes. And the final instrument differences are obtained by comparing the baseline values. However, errors (as mentioned in Sect. 2.1) cannot be completely eliminated and will still exist, which is the main reason for the instrument differences and the source of uncertainty in measurement results. Therefore, the instrument differences defined in this paper are the comprehensive differences of the entire instrument system, representing the differences between results obtained by the instruments after four measurement processes under the assumption of no personnel operation error. The impact of various internal errors of the fluxgate theodolite on the measurement results is already included in the measurement results and is part of the differences of the theodolites, which will not be discussed separately. Then, a quantitative analysis of the error will be conducted based on uncertainty analysis methods.

As described above, the uncertainty analysis in this process encompasses several key error sources, including internal errors of the theodolite, repeatability errors, operator dependency errors (differences between individuals), and pillar correction errors. Environmental interference is excluded from consideration due to controlled laboratory conditions. The uncertainty analysis process and calculation formula applied in the comparison datasets are described in Appendix B.

For an instrument comparison, after calculating the synthesized internal uncertainty (uinst,i) of each instrument, the standard deviation (srep,i) and associated Type A uncertainty (urep,i) of the repeatability error, as well as the standard deviation (sbetween) of all operator-instrument-combinations results, the operator uncertainty (uoper) can be calculated using Eq. (B7) based on these results. When the above calculation process is repeatedly applied to the results of 12 comparison datasets, 12 operator errors are obtained, as illustrated in Fig. 6. The light orange and light green filled areas represent the differences for D and I, respectively. Results show consistently higher operator dependency errors in declination measurements compared to inclination, with D errors persistently exceeding I values. This difference arises from the additional azimuth marker alignment step, and the accuracy of the vertical circle setting (at 90 or 270°) required for declination measurements, which introduces greater operator variability. The mean operator dependency errors were 0.13^′ for D and 0.06^′ for I, aligning closely with experimental results (0.18^′ for D and 0.08^′ for I) reported by He et al. (2019a), thereby validating the methodology's effectiveness in quantifying operator dependency errors.

The robustness evaluation of the reference instrument requires quantification of its systematic deviation relative to the true values and associated uncertainties. If each instrument comparison is regarded as an independent measurement experiment, then the average of the measurement results (after correction) from all instruments (excluding the reference instrument) can be taken as the true value of that measurement experiment. Then, the measurement result from the reference instrument can be compared with the true value, and the total uncertainty of the comparison result can be calculated based on the uncertainty formula. In this way, the operational status of reference instruments can be examined. By applying this same process to 12 sets of comparison data, the long-term stability of the reference instrument can be evaluated.

Using the methodology described above, we analyzed 12 years of instrumental difference data for declination (D) and inclination (I). The time series of mean differences (Δ) between the reference instrument and all instruments, along with their uncertainties (uΔ), are shown in Fig. 7. In this figure, orange and green histograms represent mean differences for D and I, respectively, while curves of corresponding colors indicate twice the uncertainty (2uΔ). According to the criterion Δ‾ ≤ 2uΔ‾, most mean differences fall within the 2uΔ range.

However, mean differences for both D and I exceeded this threshold in 2014. A retrospective review of the raw data revealed no definitive cause for this anomaly. Notably, 2014 involved an observational training program where measurements were conducted by inexperienced personnel, and the definite uncertainty was notably low. This suggests potential transient impacts on the reference instrument. Additionally, the mean difference for D in 2018 slightly exceeded 2uΔ, though no conclusive explanation has been identified. Nevertheless, the long term mean difference data demonstrate that the reference instrument has maintained stable operation and reliable performance throughout the study period.

Finally, using Eqs. (B16) and (B17), we evaluated the reference instrument's long term stability by calculating the multi years average mean differences and their uncertainties. For declination (D), the average mean difference was ΔD‾ = -0.004^′ with an uncertainty of uΔD‾ = 0.054^′. For inclination (I), the values were ΔI‾ = 0.022^′ and uΔI‾ = 0.023^′. Applying the criterion Δ‾ ≤ 2uΔ‾ (95 % confidence level), both D and I meet this requirement, confirming the reliability of the reference instrument's long term observational data.

This study systematically analyzed 12 sets of comparison measurement data from the GNC. The research findings demonstrate the critical importance of instrument comparisons in geomagnetic observatory networks, primarily manifested in the following two aspects: (i) the instrument comparison effectively monitors the operational status of instruments at various observatories, assesses the technical proficiency of operators, and enables timely repair and maintenance of problematic instruments; and (ii) the comparison data facilitates the analysis of factors contributing to absolute geomagnetic observation errors, allows for a comprehensive evaluation of data quality across the entire geomagnetic network, and provides a means to assess whether reference instrument meets standards. These assessments also offer valuable references for evaluating the reliability of scientific conclusions derived from geomagnetic data.

The statistical analysis results reveal that when the probability density of instrument differences accumulates to 90 %, the corresponding instrument difference are 0.21^′ (D component) and 0.11^′ (I component), which can serve as evaluation criteria at the network level. The statistical results of the comparison also effectively reflect the factors influencing the declination measurement. Specifically, they reveal that the frequent rotation of operators has a significant impact on the declination observation results, while its effect on the inclination is not as pronounced. This further suggests that operator dependency error is the primary source of error in absolute geomagnetic measurements.

Through uncertainty analysis of multi-source error, the systematic differences between the reference instruments and the test instruments were quantified. The operator dependency errors of D (0.13^′) and I (0.06^′) were successfully separated and consistent with the observed experimental results, confirming that operator dependency error is the primary factor contributing to instrument differences. Notably, operator dependent errors in D are significantly higher than I, which once again proves that D is more susceptible to human factors. A comprehensive evaluation was developed to assess the robustness of reference instrument. By constructing time series of mean differences (Δ) and their uncertainties between the reference instrument and all testing instruments, the long term stability of the reference instrument can be analyzed. The results showed that the data remained stable for a long time without significant drift, and the average of mean differences are -0.004^′ (D) and 0.022^′ (I), both within the 95 % confidence interval. This indicate the reference instrument exhibit high stability and reliability.

However, this approach has limitations. For instance, it overlooks operator instrument interactions and environmental factors (e.g., humidity fluctuations), which may introduce systemic biases. Long term stability analysis also requires extensive multi years comparison data to ensure statistical power, limiting rapid field applications. Future research should focus on model optimization, such as incorporating environmental sensor data to establish temperature/humidity compensation mechanisms or developing automated tools to streamline multi sources uncertainty synthesis. With continuous refinement, this methodology holds promise for advancing standardization and long term stability in geomagnetic observation networks.

Geomagnetic observatories serve as primary facilities for measuring the secular variation of the Earth's magnetic field. The measurement accuracy for directional elements (e.g., declination and inclination) is typically required to be below 0.1^′, while the accuracy for intensity elements (e.g., total field strength) should be within 1nT. Modern instruments, such as the Zeiss 010B fluxgate theodolite, theoretically possess sufficient precision to achieve these targets. However, in practice, attaining such accuracy remains challenging due to various sources of error, particularly operator dependency errors. With advancements in automation and the global adoption of high precision instruments (Rasson and Gonsette, 2011; Gonsette et al., 2017; Hegymegi et al., 2017) in the future, it is anticipated that operator dependency differences will be eliminated, thereby obtaining higher quality geomagnetic observational data.