Improving the conditioning of estimated observation error covariance matrices
New developments in the treatment of observation uncertainties have shown that accounting for correlated observation errors in data assimilation can improve analysis accuracy and forecast skill. In practice, sampled error covariance matrices are used to approximate the uncertainties. In high dimensional problems, due to restrictions on sample size, the estimated covariances are routinely rank deficient and/or ill-conditioned and marred by sampling noise; thus they require modification, or ‘reconditioning', before they can be used in a standard assimilation framework. We present new theory for two existing methods for improving the rank and conditioning of a multivariate sampled error covariance matrix: ridge regression, and the minimum eigenvalue method. The methods are compared in terms of their impact on the variances and correlations of the matrix. We apply the reconditioning methods to two examples, one based on a general correlation function, and one arising from numerical weather prediction. The minimum eigenvalue method results in smaller overall changes to the correlation matrix than the ridge regression method, but in contrast can increase off-diagonal correlations.
Ms. Jemima M. Tabeart University of Reading & National Centre for Earth Observation