Data were collected at the ES-LMa FLUXNET site near Majadas de Tiétar, Extremadura, Spain (39∘56′25.12′′ N, 5∘46′28.70′′ W). In May of 2015, 16 semi-automated soil efflux measurement chambers were installed in a stratified random sampling design grouped into different treatments and canopy positions. The chambers are an in-house-developed stainless-steel design, connected to a LI-820 (LI-COR, Lincoln, Nebraska, USA) measuring in a half-hourly cycle. During this cycle one chamber at a time would close for a 3 min measurement duration. While there were 16 chambers in all, only data from four chambers in the open grassland stratum within the control plot are used for the purposes of this paper. The aggregate across these four chambers, here, is referred to as the plot-level estimate, although it only represents the open grassland and four chambers are not enough to capture the full spatial variability. Fluxes and their variance were computed from CO2 concentration time series using the RespChamberProc R package (Sect. ) by estimating the initial slope of concentration increase. These fluxes (plotted in Supplement 1) and associated variance are the input data for this study. They are denoted by RSi and σoi2 in this paper. Data used in this paper range from November 2015 to November 2016. Additional details about the site can be found in .

Each measurement has uncertainty, and this uncertainty can be characterized by a density distribution. For similar environmental conditions, observed fluxes (RS) scatter around a basic flux (RB). The noise originates from both instrumentation error (IE) and process variation (PR), a stochastic component intrinsic to the measured soil system. While the nonsystematic component of IE is usually well described as a normally distributed random variable, PR can be described by a normal or Laplace distribution (Eq. 1), 1aRS=RB+ϵPR,add+ϵIE1bϵPR,add∼norm(0,σPR,add)or∼Laplace(0,b)1cϵIE∼norm(0,σIE), or alternatively with the lognormal distribution Eq. (2): 2aRS=RBϵPR,mult+ϵIE2bϵPR,mult∼lognorm(-σL2/2,σL2)2cR=log⁡(RS)≈log⁡(RB)+log⁡(ϵPR,mult), where ϵ are error terms and σPR,add, b, and σL are scale parameters of their respective distributions. ϵPR,mult is assumed to be lognormally distributed with an expected value of 1. ϵIE is usually small compared to RBϵPR,mult , and hence approximation (Eq. 2c) allows analysis of log-transformed observations. If variance of IE increased with flux magnitude, too, it could also be modeled by a lognormal distribution; however, studies on chamber measurement error did not show such an increasing pattern Fig. 8, Figs. 3 and 4.

Equations (1) and (2c) are extreme cases of a hierarchical model that accounts for both types of error (Appendix ). The lognormal model (Eq. 2c) is sometimes applied without further consideration when log-transforming observations to counteract heteroscedasticity e.g.,.

Error terms are the difference between observed fluxes and a true basic flux. The true flux is unknown but can be estimated by the average flux under similar environmental conditions.

A simple method of estimating the absolute error terms is daily differencing, excluding days with and after rain events . This daily differencing method (Eq. ), also called the paired-observation approach, assumes that records 24 h apart represent similar environmental conditions and hence differences in the observed flux (Eq. ) can be used to estimate the random error. It includes both the non-systematic component of IE and PR (Sect. ). 3ϵ=RS,t-RS,t+24/2

An alternative method is the lookup table approach (LUT). It is commonly used in the marginal distribution sampling method , a method used for filling gaps in data from eddy covariance sensors . When applied to soil CO2 efflux observations in this study, similar environmental conditions were determined by the hour of the day (±1h), temperature (±3∘C), soil moisture (±5%), and a time window. The time window size was increased from ±1 to ±3, 12, and 24 d until there were at least five valid measurements to average across.

A third alternative is modeling the base flux by its relationship with ancillary observations, such as temperature. We tried modeling the CO2 efflux temperature relationship with varying basal respiration . However, cross-validation showed that this approach did not achieve good results for RS at the Majadas site, because correlation with temperature is generally weak at water-limited sites . Moreover, during dry periods small precipitation events caused respiration pulses without observed concurrent increases in soil moisture at 5 cm soil depth, where soil moisture sensors were located.

When using the lognormal assumption, daily differencing was applied to the log-transformed observed fluxes, whereas for the LUT approach the difference between observed and mean flux was computed with the log-transformed values log⁡(ϵPR,mult)=R-log⁡(RS,LUT).

The aggregation across time (Sect. ) must take into account correlations among individual observations, because subsequent measurements are usually autocorrelated.

The correlation cannot be computed by the uncertainties of individual fluxes, σoi2, but requires the estimation of individual error terms. After estimating the error terms of all half-hourly fluxes by LUT (Sect. ), we computed the empirical autocorrelation function from the time series of error terms using the acf function implemented in R . Only the first components of the autocorrelation function can be estimated reliably from the time series. Hence, we only used those components ρk before the first negative autocorrelation to construct the variance–covariance matrix. Correlation of error terms farther apart than the maximum number of estimated components of the empirical autocorrelation function was set to 0. Other components were Cov(ϵi,ϵj)=corijσD2=ρ|i-j|σD2, where ϵ=ϵPR,add and ϵ=log⁡(ϵPR,mult) are the estimated error terms for the normal and lognormal assumption, respectively, and σD2 is the variance across those ϵ.

Gaps in the flux time series have to be filled before computing the annual aggregated flux. Shorter gaps were filled using the LUT with a window size up to ±24d (Sect. ). Longer gaps were filled by fitting a random-forest machine learning model with predictors “half hour of the day”, global radiation, air temperature, soil temperature, precipitation, vapor pressure deficit (VPD), mean daily soil temperature, mean daily air temperature, soil moisture, mean soil moisture across chambers, and day length. Gap filling extrapolated at maximum 5 d into gaps. The remaining long gaps were treated as missing.

For the plot-level annual aggregation, we estimated the fluxes during long gaps by the mean flux of the other chambers. Using this mean of the other chambers is not fully statistically valid, because one should correct for the chamber offsets that vary slowly across time. However, this was the best estimate we could get for the dataset used.

We are interested in the value and the uncertainty of the flux aggregated across time and across the replicate chambers of the recorded measurement. Across chambers we analyze a sample of four replicates. Across time, we are concerned with the propagation of the random variability induced by the random variations (measurement error and process variation) of the individual measurements (Eq. ). 4Var(RSi‾)=Var(ϵi‾)≈Var(ϵ‾i∈iobs)=f(σoi2,ρ)

The uncertainty of the aggregated value – here, the mean across several soil CO2 fluxes – is the propagated uncertainty of the uncertainty of the single fluxes, ϵi (Eq. 1a). While the mean flux can be computed including gap-filled records, those gap-filled records may introduce systematic errors but should not contribute to the reduction of average random uncertainty with more observations (Eqs. 5b, ). The uncertainty of error terms is provided by uncertainty, σoi2, reported with the observed fluxes, but their autocorrelation, ρ (Sect. ), requires the estimation of error terms.

If IE is dominating, the error is usually well described by independent normal distributions with a mean of 0 (Eq. 1c) with the well-known error propagation rules (Eq. 5). 5aE∑xi=∑E[xi]SD∑xi=∑SD(xi)2,5bExi‾=E[xi]‾SDxi‾=1nSD(xi)2‾, where the bar denotes the mean, SD denotes standard deviation, and n is the number of elements in sequence xi. In our case SD(xi)=SD(RSi)=σoi2.

However, for time series usually one must consider autocorrelation, where successive measurements are not independent of each other, i.e., where knowing the random error of one measurement holds information for predicting the error of other measurements close in time. One has to add covariance terms when summing variances. For autocorrelated series this leads to formulas dependent on the effective number of observations (Eq. ) based on the autocorrelation function, which describes how strongly errors are correlated across time lags . 6Var(x‾)=Var(x)neff,7Var(x)=neffn(neff-1)∑i=1n(xi-x‾)2,8neff=n1+2∑k=1n-11-knρk, where x‾ denotes the mean of a vector of random variable x, n is the number of records, and ρk denote the coefficients of the autocorrelation function. The autocorrelation function is usually not known, but its first components can be reliably estimated from the data. We followed , who recommend using only the components before the first negative component for k in Eq. () instead of all n-1 components (Sect. ).

In the studied case, x is the random error in half-hourly observations with an expected value of 0. One could use the estimated error terms (Sect. ) for (xi-0), but we used the original observation uncertainty, σoi2, given with each observation. Therefore, in Eq. () (xi-x‾)2 is replaced by σoi2 and (neff-1) is replaced by neff because the degree of freedom for computing the mean, xi, was not used. Then Eq. () becomes Eq. (). 9Var(x‾)=1neffσoi2‾

Hence, the uncertainty (Var(x‾)) declines with neff Eq. () compared to n with uncorrelated random errors of observed fluxes (Eq. 5b). Confidence intervals for aggregated mean fluxes were computed as x‾±1.96SD(x‾), where SD denotes the standard deviation.

For gap-filled records the residual error is missing. Hence, those records do not contribute to the number of effective observations. However, they are included in computing the mean aggregated flux.

An overview of the properties of the lognormal distribution is provided in Appendix .

For aggregating fluxes across chambers, we first log-transformed each observed flux, R=ln⁡RS. For the aggregation across replicates, we used the log-transformed values of the same time from different chambers to compute the parameters μ and σ of the distribution (Eq. A4) across chambers. Next, we used the distribution parameters to obtain the expected value (Eq. A2a) and prediction interval between quantiles 2.5 % and 97.5 % (Eq. ).

For aggregating fluxes of a single chamber across time, we considered the error term in each half-hourly measurement as a realization of a lognormally distributed random variable. The propagation of the error to the sum of such random variables (Eq. A7a) requires the distribution parameters. Hence, these parameters, μi and σLi, were first computed from the expected value, i.e., the measured flux, RSi, and its variance that was reported together with the flux, σoi2 (Eq. A5). Gap-filled values in the time series complicated the application of Eq. (A7), because they should contribute to the expected value of the sum but should not contribute to the reduction in uncertainty with aggregation across many measurements. Hence, we computed the sum's scale parameter, σS, based on original measurements only, but computed the expected value with the inclusion of gap-filled values (Appendix Eq. A2). Hence, the expected value of the sum corresponded to the sum of the gap-filled measured fluxes (Eq. A7a). We provide the R function estimateSumLognormalSample with the lognorm R package (Sect. ) to help with this aggregation.

At observations of low fluxes the instrumentation error component cannot be neglected and the lognormal assumption is violated. Such observations were treated as gap-filled for most of aggregation scenarios; i.e., they contributed to the expected value but not to the error propagation for the mean flux.

For applying these concepts to researchers data, we provide well-documented code in two publicly available packages for the R language.

Computing fluxes from series of concentrations measured inside chambers is provided by the package RespChamberProc (https://doi.org/10.5281/zenodo.3735807), available at GitHub (https://github.com/bgctw/RespChamberProc, last access: 26 May 2020).

Utilities dealing with lognormally distributed data are provided with the package lognorm (https://doi.org/10.5281/zenodo.3735804), available at CRAN (https://cran.r-project.org/web/packages/lognorm/index.html, last access: 26 May 2020). It includes functions for estimating moments and mode, estimating parameters from sample or from summary statistics, and approximating the sum of correlated lognormals.

The distribution of error terms obtained by daily differencing had strong tails, while when applying the daily differencing to log-transformed values, R=ln⁡RS, the distribution of the resulting error became closer to normal than on the original scale (Fig. ). For large negative outliers the lognormal distribution approximated error distribution even better than the Laplace distribution. Moreover, the log transformation avoided the problematic scaling of random error with flux magnitude. Standard deviation across random error within 1 d scaled with flux magnitude on the original scale (Fig.  top) but did not scale on a log-transformed scale (Fig.  bottom). Autocorrelation in error terms on a log scale was stronger than autocorrelation in error terms on the original scale (Appendix ).

We compared the aggregation of half-hourly fluxes across four neighboring chambers using the lognormal assumption (Sect. ) versus using the normal assumption (Sect. ). For periods without extreme fluxes, the aggregated value and prediction intervals were very similar (Fig. ). Differences became more evident with high fluxes after rainfall when there was larger variability across chambers. The prediction intervals differed for the following features. First, the upper prediction interval bound was not as strongly influenced by high fluxes; second, the lower bound of the prediction interval was usually close to the lowest observed value. Hence, the lognormal-based lower prediction interval bounds circumvented negative values.

The expected value of the aggregated fluxes across the 48 half-hourly measurements per day was the same across distributional assumptions. It corresponded to the mean of the observed values. The width of the 95 % prediction interval was similar for most records but differed in a few cases (Fig. , top row).

Instances where the lognormal assumption resulted in much wider prediction intervals occurred on days with very low fluxes. In these cases the process variation, which scales with the flux, is small compared to the instrumentation error, and the assumption that error is dominated by the multiplicative component (Eq. 2c) is violated. Those cases need to be treated differently. One way of counteracting the resulting overestimation of uncertainty is setting a gap-filling flag for the uncertainty estimate of very small fluxes (Fig. , second row) (Sect. ). This treatment of low fluxes tackles the overestimation of uncertainty for such periods but also leads to slightly wider confidence bounds because now a lower number of observations contribute to the lognormal uncertainty aggregation compared to the normal uncertainty aggregation. The few cases with clearly narrower prediction intervals given the lognormal assumption occurred on days with limited original measurements and large outliers in estimated uncertainty of the single measurements. The lognormal approach was much less sensitive to large outliers and yielded narrower prediction intervals of the aggregated value. When constraining the dataset to days with at least 10 original measurements, most of the differences disappeared (Fig.  bottom row).

Contrary to the short-term aggregation, distribution of annually aggregated fluxes of each chamber did not differ much between the two approaches (Fig. ). The skewness in the distribution of uncertainty of annual estimates almost disappeared, as seen by the similar distance to upper and lower prediction interval bounds in Fig. .

The combined temporal annual and cross-chamber aggregation to the plot level can be done with two alternatives. With one alternative, temporal aggregation (using either the normal or lognormal assumption) is done first, and aggregation across replicates (using the lognormal assumption) estimates is done using the annual estimates of each chamber. With the second alternative, the aggregation across replicates is done first for each half hour across all chambers, and these plot-level fluxes are then aggregated across time. The latter “replicate-first” alternative yielded lower uncertainty estimates (standard deviation of 0.005 instead of 0.02 gCm-2d-1). The reason is that it neglects any temporally constant or slowly varying component in the location effect. However such a component strongly contributes to the variation across the annual aggregates. This effect is similar to pseudo replicates, as locations did not change between successive measurements.

With the lognormal assumption the distribution of random error can be inspected on a log scale rather than on the original scale. This improves two distributional problems . First, the lognormal distribution better approximates the more frequent occurrence of large errors (Fig. ). Second, the heteroscedastic nature of the random error is reduced; i.e., on a log scale residual variance does not increase with flux magnitude (Fig. ).

Although the increase of variance could be handled alternatively by an explicit error model in generalized regression or flexible cost functions in model inversion , the log transformation tackles this problem in a basic way.

The increase of variance with flux magnitude also created the pattern of apparent Laplace distribution (Fig. ). When we inspected the distribution of subsets of flux errors with similar magnitudes (using LUT, Sect. ), we did not find the Laplace shape. This finding suggests that it is the superposition of normal distributions with different variance at different flux magnitudes that leads to the apparent Laplace shape. This finding is similar to what found for random error of NEE that was measured by eddy covariance. Hence, when the error magnitude is used in model data integration exercises, we argue against using the Laplace assumption and against the associated usage of median absolute deviations when model predictions are compared to single observations. Instead, we recommend using the usual normal-based formula for the cost function but with log-transformed predicted flux and log-transformed observed flux.

We assumed that, if a lognormally distributed process variation dominates the observation error of single chambers, then such a process variation also dominates the differences between chambers. Hence, we assumed also a lognormal distribution of measurements across several chambers. With only four replicates, we cannot inspect distributional properties. However, using the lognormal assumption was especially important for periods of high variability across chambers, which occurred at the Majadas de Tiétar site mostly during the dry summer period, similar to findings of . Without using the lognormal assumption, prediction interval bounds of plot-level fluxes would include negative values (Fig. ).

At the Majadas site, we attribute negative fluxes to measurement error; however, negative fluxes can be real, especially at sandy alkaline soils with low decomposition, i.e., with sparse vegetation. There are abiotic causes for these negative fluxes: carbonate dissolution, soil air shrinkage with temperature and pressure, and CO2 dissolution in soil water . However, the soil at the studied site is not a carbonate soil (inorganic carbon contents of 0.20 to 0.25 gCkg-1drysoil), and possible abiotic fluxes were magnitudes lower than the observed fluxes at Majadas that are dominated by decomposition of organic material. For examples, in Fig.  the confidence bound includes negative fluxes as well fluxes higher than 2.5 gCm-2d-1.

Nevertheless, the lognormal assumption is not applicable at karstic soils with a high proportion of conditions with real negative fluxes. However, if the proportion of observations with conditions for negative fluxes is low, these conditions can be flagged and the observations can be handled similar to gap-filled records or records where measurement error dominates, which contribute to the expected value but not to the uncertainty estimate.

Further, we explored consequences of aggregating measurements of a single chamber across time using the lognormal assumption compared to classical aggregation using the normal assumption. A single chamber measurement representing a time period can be assumed to be a normal or a lognormal random variable. These assumptions resulted in different aggregated uncertainties when aggregating across a few days (Fig. ). We argue that the choice of distributional assumptions depends on the sampling interval, the magnitude of measurement error, and the autocorrelation length of the process variation. If measurements are frequent relative to process autocorrelation length, the uncertainty is dominated by the instrumentation error (from the measurement device), which can be assumed to follow a normal distribution. Alternatively, if a single measurement represents a longer period, the uncertainty will be dominated by process variation. While process variation dominates random error at a daily measurement resolution , we cannot distinguish between those two cases from our series of half-hourly measurements.

However, we encountered a problem when fluxes were very low, where the instrumentation error component becomes dominant and the lognormal assumption is violated. If the lognormal assumption is applied to such cases, time aggregation leads to overestimation of uncertainty, because it overestimates the multiplicative error. Those records need to be flagged similar to gap-filled records before aggregation using the lognormal approach (Sect. ).

When half-hourly measurements of a single chamber were aggregated to longer timescales such as to annual aggregates, the differences in uncertainty bounds between distributional assumptions decreased (Fig. ). There was a tendency towards slightly narrower bounds with the lognormal assumptions. We argue that this is due to the lognormal approach being more robust to the influence of few large values. For chamber 5 the lognormal-based uncertainty is wider, because there were long gaps during the season of large fluxes, and hence there was a relatively larger proportion (15 %) of low fluxes that were excluded from error propagation where the assumption of dominating process variance was violated. Moreover, the autocorrelation structure in error terms was not detected properly on the original scale for this chamber (Appendix ).

Also the skewness disappeared (Fig. ). This was a consequence of relative uncertainty decreasing with the number of aggregated measurements, which led to less skew and to lognormal distributions which are close to normal (Fig. ). This is also in line with the general idea of the central limit theorem , although we could not find a version of the theorem that matches the combined non-iid and non-Gaussian case of single terms for the time series of this study.

Overall, we suggest using the lognormal assumption for aggregating across fluxes from replicated chambers but the normal assumption for aggregating half-hourly observations of a single chamber across time with number of records exceeding, say, 40.

When deciding whether to first aggregate across chambers or across time, the “cross-chamber-first” alternative (Sect. ) wrongly assumes that the cross-chamber aggregated values are only correlated in time. They are, however, measured at the same spatial locations and fully correlated in space. Therefore, whenever measurement locations are fixed and a plot-level estimate is required, the cross-chamber aggregation should be computed as the last step.

Our finding on the suitability of the model of a multiplicative, lognormal process variation sheds new light on the process variation, i.e., the as-yet-unattributed soil processes that generate random fluctuations in soil CO2 efflux observations. proposed two mechanisms for process variation. First, a higher diversity of active metabolic pathways associated with a wider range of pore-scale respiration rates at high temperature could result in larger variability of fluxes. Because higher temperatures are associated with higher fluxes, this would explain the increase of variance with flux magnitude. Second, gas diffusion rates might increase due to heat produced during respiration. Similarly, gas transport processes in soil can change with pore space varying with soil moisture .

We propose the alternative hypothesis based on small-scale spatial heterogeneity and stochasticity of the temperature sensitivity of chemical reactions involved. Metabolic rates associated with microbial communities differ across micrometer distances in their temperature sensitivity, and these metabolic rates in turn largely drive respiration and soil CO2 efflux. Respiration is related to such temperature sensitivity in an exponential manner . Hence, if variation in temperature sensitivity is normally distributed, then the log of respiration is normally distributed; i.e., variation in respiration is lognormally distributed. This argument is transferable to process variation distribution of fluxes on the leaf and ecosystem scale.

To obtain plot-level estimates of soil CO2 efflux, one typically has to aggregate time series of several chambers. For such cases we recommend the following procedure based on the experience gained with this study.

Estimate error terms by daily differencing or, preferentially, LUT.

In model data integration compare predictions and observations of soil CO2 efflux on a log scale.