Robust estimators are often lacking a closed-form expression for the computation of their residual covariance matrix. In fact, it is also a prerequisite to obtain critical values for normalized residuals. We present an approach based on Monte Carlo simulation to compute the residual covariance matrix and critical values for robust estimators. Although initially designed for robust estimators, the new approach can be extended for other adjustment procedures. In this sense, the proposal was applied to both well-known minimum L1-norm and least squares into three different leveling network geometries. The results show that (1) the covariance matrix of residuals changes along with the estimator; (2) critical values for minimum L1-norm based on a false positive rate cannot be derived from well-known test distributions; (3) in contrast to critical values for extreme normalized residuals in least squares, critical values for minimum L1-norm do not necessarily tend to be higher as network redundancy increases.
The least-squares (LS) estimator, also known as L2-norm minimization, is the standard method in surveying data processing. In case of absence of outliers, it is the best linear unbiased estimator for the unknown parameters [
However, if there are outliers in the sample, the LS will provide biased parameters [
The iterative approach of DS, also known as Iterative Data Snooping (IDS) [
The
However, IDS involves not only a single test but also multiple hypothesis testing. In that case, the “false positive rate” (type I error probability or significance level
As mentioned, IDS, however, is based on LS residuals, which are very “sensitive” to outliers [
Minimum L1-norm seeks the minimization of the sum of weighted absolute residuals [
The minimum L1-norm solution may not be unique [
Minimum L1-norm has no analytical direct solution and needs to be solved by numerical methods. This work focuses on the solution by the
Another technique commonly employed to solve minimum L1-norm in geodetic networks is the iterative reweighted least squares (IRLS) (see, e.g., [
Other techniques that were performed to compute minimum L1-norm in geodetic networks include simulated annealing [
To actually identify outliers based on minimum L1-norm results, geodesists have already tried two different criteria: (1) the ratio between residuals and respective observation standard deviation
In this context, this paper has three main contributions: (1) it provides a new procedure to compute critical values for normalized residuals in robust estimation based on MCS control of false positive rate; (2) it serves as a method to compare different quality control procedures by preserving respective critical values with the same false positive rate; and (3) it provides a Monte Carlo approach to compute the covariance matrix of residuals
The outline of this paper is as follows. First, we present the new approach to estimate
Given a geodetic network with
Hence, applying the general law of propagation of variances [
Assuming
Finally,
In matrix
For other adjustment procedures, however, the computation of synthetically generate For each MCS trial, let Considering the average of residuals of the fix the estimated
Proposed Monte Carlo approach to compute
Regarding
Recently, many works (see, e.g., [ compute synthetically generate compute the max-| order the set of all max-| The critical value will be the one in position (1 −
Proposed Monte Carlo procedure to compute critical values for robust estimators.
Experiments were performed in three simulated leveling networks (Figure
Configuration of networks A, B, and C.
For all networks, the standard deviation of the observations was given by
Minimum L1-norm adjustments were performed by the
At first, we computed
Covariance matrices computed for network A (mm2).
Ʃv(LS−A) | Ʃv(LS−MCS) | Ʃv(L1−MCS) |
---|---|---|
Then, we applied the new procedure to compute critical values for normalized residuals by MCS and based on the false positive rate (significance level
Critical values for normalized residuals.
Normal table | | Network A | Network B | Network C | ||||
---|---|---|---|---|---|---|---|
IDS | Min. L1-norm | IDS | Min. L1-norm | IDS | Min. L1-norm | ||
0.10 | 3.29 | 3.74 | 5.89 | 3.89 | 6.68 | 3.98 | 5.16 |
0.27 | 3.00 | 3.48 | 5.35 | 3.64 | 5.97 | 3.74 | 4.82 |
1.00 | 2.575 | 3.10 | 4.61 | 3.28 | 4.99 | 3.41 | 4.32 |
2.50 | 2.24 | 2.81 | 4.04 | 3.00 | 4.32 | 3.13 | 3.93 |
5.00 | 1.96 | 2.56 | 3.60 | 2.77 | 3.80 | 2.91 | 3.62 |
10.00 | 1.645 | 2.29 | 3.13 | 2.52 | 3.30 | 2.68 | 3.29 |
Although the use of MCS seems computationally expensive, today, this is no longer an obstacle even for personal computers [
The first issue to be addressed is the covariance matrix of residuals for network A, B, and C. Table
Statistics of differences of elements (mm2).
Network | Between matrices | Maximum | Minimum | Average | |||
---|---|---|---|---|---|---|---|
VAR | COV | VAR | COV | VAR | COV | ||
A | Ʃ | 0.101 | 0.096 | 0.004 | 0.004 | 0.037 | 0.041 |
11.703 | 5.104 | 0.322 | 0.421 | 5.978 | 2.796 | ||
B | 0.078 | 0.105 | 0.001 | 0.002 | 0.040 | 0.031 | |
24.524 | 7.448 | 2.363 | 0.063 | 12.846 | 3.393 | ||
C | 0.293 | 0.144 | 0.003 | 0.001 | 0.053 | 0.037 | |
6.606 | 3.009 | 3.265 | 0.001 | 4.920 | 1.454 |
On the other hand, we can clearly see that elements of
As expected, critical values for normalized residuals in IDS and minimum L1-norm presented in Table
Finally, we note also that critical values for both minimum L1-norm and IDS vary from different networks. Although IDS values tend to increase with network redundancy, as already shown by [
In this work, we successfully developed and presented an approach by means of MCS to compute the covariance matrix of residuals and critical values for normalized residuals in any adjustment procedure. Since the LS method has a well-established analytical expression for the covariance matrix of residuals, our MCS strategy to estimate it was first applied to LS. We found that differences in respective elements between our strategy and analytical formulation were negligible, which validates our approach.
Numerical results of the whole procedure of computing critical values, which includes the estimation of the respective residuals covariance matrix, were presented in three leveling networks for minimum L1-norm solved by the
We have shown that the covariance matrix of residuals may change along with the adjustment procedure (in our case, from LS to minimum L1-norm). Therefore, since robust estimators generally do not have a well-established solution to compute the covariance matrix of residuals, the approach presented for any adjustment procedure (including robust estimators) herein is a valuable strategy.
Surveyors cannot rely on critical values from univariate normal distribution either for IDS or minimum L1-norm. Moreover, critical values vary even among robust estimators. However, unlike IDS, the critical values in minimum L1-norm do not necessarily tend to increase with network redundancy. Hence, the main contribution of this work was the proposed Monte Carlo-based critical values to control the false positive rate for normalized residuals of robust estimators.
Future research should perform this proposal in order to provide a fair comparison among different quality control procedures with the same false positive rate. Furthermore, one can investigate effects of chosen false positive rates in probability levels of classes of errors in outlier identification, i.e., type II error, type III error, overidentification positive and negative, and statistical overlap (see [
The proposed approach for the computing residuals covariance matrix can be extended to covariance matrices other than the residuals one in future works. One can, e.g., compute the network parameters in each MCS trial and then compute the parameter covariance matrix of the chosen adjustment procedure.
The relationship between network redundancy and critical values for normalized residuals in robust estimation also needs further investigation. Besides, the new approach for the computation of the covariance matrix of residuals and for the estimation of critical values for normalized residuals described here should be applied for other robust estimators and other types of geodetic networks, such as Global Navigation Satellite System (GNSS) networks.
Tables
Matrix elements (mm2) | |||||
---|---|---|---|---|---|
24.875 | 10.565 | −0.195 | 6.755 | 6.560 | 10.370 |
10.565 | 20.919 | 7.215 | −0.699 | 6.516 | −9.866 |
−0.195 | 7.215 | 13.367 | 6.613 | −7.020 | −6.418 |
6.755 | −0.699 | 6.613 | 9.331 | −6.056 | 5.914 |
6.560 | 6.516 | −7.020 | −6.056 | 9.924 | −0.504 |
10.370 | −9.866 | −6.418 | 5.914 | −0.504 | 16.716 |
Matrix elements (mm2) | |||||
---|---|---|---|---|---|
24.901 | 10.661 | −0.110 | 6.782 | 6.546 | 10.307 |
10.661 | 20.939 | 7.187 | −0.682 | 6.551 | −9.807 |
−0.110 | 7.187 | 13.313 | 6.629 | −6.990 | −6.328 |
6.782 | −0.682 | 6.629 | 9.349 | −6.060 | 5.921 |
6.546 | 6.551 | −6.990 | −6.060 | 9.920 | −0.546 |
10.307 | −9.807 | −6.328 | 5.921 | −0.546 | 16.615 |
Matrix elements (mm2) | |||||
---|---|---|---|---|---|
34.951 | 6.134 | 0.530 | 4.509 | 4.830 | 5.344 |
6.134 | 25.380 | 5.259 | −5.786 | 4.477 | −4.986 |
0.530 | 5.259 | 25.016 | 3.118 | −3.139 | −4.844 |
4.509 | −5.786 | 3.118 | 16.556 | −2.896 | 4.366 |
4.830 | 4.477 | −3.139 | −2.896 | 12.063 | 0.000 |
5.344 | −4.986 | −4.844 | 4.366 | 0.000 | 16.937 |
Tables
Matrix elements (mm2) | |||||||||
---|---|---|---|---|---|---|---|---|---|
23.099 | −6.372 | 0.038 | 0.231 | −7.799 | −7.529 | −7.567 | −6.103 | −6.334 | 0.269 |
−6.372 | 15.454 | 6.372 | 0.430 | −0.627 | 6.174 | −0.197 | −5.745 | −6.175 | 6.801 |
0.038 | 6.372 | 19.855 | 5.922 | 0.890 | −6.334 | 6.812 | −0.851 | −6.774 | −7.223 |
0.231 | 0.430 | 5.922 | 13.604 | 6.276 | −0.198 | −6.121 | −6.044 | 6.352 | −6.474 |
−7.799 | −0.627 | 0.890 | 6.276 | 25.663 | −7.172 | −8.061 | 6.538 | 0.262 | 7.165 |
−7.529 | 6.174 | −6.334 | −0.198 | −7.172 | 18.296 | −7.370 | −0.358 | −0.159 | −6.532 |
−7.567 | −0.197 | 6.812 | −6.121 | −8.061 | −7.370 | 24.818 | 0.494 | 6.614 | 0.691 |
−6.103 | −5.745 | −0.851 | −6.044 | 6.538 | −0.358 | 0.494 | 16.359 | −6.596 | −6.896 |
−6.334 | −6.175 | −6.774 | 6.352 | 0.262 | −0.159 | 6.614 | −6.596 | 21.052 | −0.422 |
0.269 | 6.801 | −7.223 | −6.474 | 7.165 | −6.532 | 0.691 | −6.896 | −0.422 | 27.303 |
Matrix elements (mm2) | |||||||||
---|---|---|---|---|---|---|---|---|---|
23.035 | −6.417 | 0.021 | 0.230 | −7.805 | −7.553 | −7.465 | −6.095 | −6.229 | 0.255 |
−6.417 | 15.478 | 6.363 | 0.436 | −0.615 | 6.201 | −0.195 | −5.751 | −6.156 | 6.813 |
0.021 | 6.363 | 19.894 | 5.994 | 0.904 | −6.366 | 6.854 | −0.908 | −6.682 | −7.242 |
0.230 | 0.436 | 5.994 | 13.638 | 6.262 | −0.250 | −6.044 | −6.081 | 6.389 | −6.489 |
−7.805 | −0.615 | 0.904 | 6.262 | 25.685 | −7.176 | −8.071 | 6.565 | 0.222 | 7.170 |
−7.553 | 6.201 | −6.366 | −0.250 | −7.176 | 18.334 | −7.387 | −0.325 | −0.208 | −6.501 |
−7.465 | −0.195 | 6.854 | −6.044 | −8.071 | −7.387 | 24.740 | 0.421 | 6.603 | 0.663 |
−6.095 | −5.751 | −0.908 | −6.081 | 6.565 | −0.325 | 0.421 | 16.405 | −6.650 | −6.876 |
−6.229 | −6.156 | −6.682 | 6.389 | 0.222 | −0.208 | 6.603 | −6.650 | 20.996 | −0.446 |
0.255 | 6.813 | −7.242 | −6.489 | 7.170 | −6.501 | 0.663 | −6.876 | −0.446 | 27.304 |
Matrix elements (mm2) | |||||||||
---|---|---|---|---|---|---|---|---|---|
46.912 | −3.882 | 0.243 | −0.090 | 1.851 | −6.036 | 1.964 | 2.750 | 3.470 | 1.470 |
−3.882 | 13.115 | 4.654 | 0.372 | −4.172 | 2.125 | −4.687 | −3.836 | −3.791 | 0.639 |
0.243 | 4.654 | 31.286 | 2.254 | 4.222 | −3.682 | −0.287 | −7.010 | −0.249 | 0.587 |
−0.090 | 0.372 | 2.254 | 6.702 | 2.975 | −0.490 | −3.026 | −2.519 | 2.781 | −3.569 |
1.851 | −4.172 | 4.222 | 2.975 | 41.976 | −5.582 | −0.622 | 1.139 | 3.542 | −0.858 |
−6.036 | 2.125 | −3.682 | −0.490 | −5.582 | 13.078 | −5.069 | −1.601 | −1.096 | 1.657 |
1.964 | −4.687 | −0.287 | −3.026 | −0.622 | −5.069 | 42.890 | 4.207 | −1.276 | 2.077 |
2.750 | −3.836 | −7.010 | −2.519 | 1.139 | −1.601 | 4.207 | 24.088 | −7.454 | −0.043 |
3.470 | −3.791 | −0.249 | 2.781 | 3.542 | −1.096 | −1.276 | −7.454 | 32.981 | −1.847 |
1.470 | 0.639 | 0.587 | −3.569 | −0.858 | 1.657 | 2.077 | −0.043 | −1.847 | 51.828 |
Tables
Matrix elements (mm2) | ||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
19.478 | 5.322 | −0.182 | −0.047 | 0.074 | 5.356 | −5.201 | −5.383 | −5.430 | 0.156 | −0.027 | −5.166 | −5.093 | −5.140 | 0.229 |
5.322 | 23.978 | 4.498 | 0.124 | −0.186 | 0.264 | −4.701 | −0.203 | −0.079 | −4.436 | 0.062 | 5.586 | 5.401 | 5.525 | −4.622 |
−0.182 | 4.498 | 15.426 | 5.447 | −0.136 | −0.053 | 4.316 | −5.258 | 0.189 | 4.263 | −5.311 | −0.235 | −0.371 | 5.076 | 4.127 |
−0.047 | 0.124 | 5.447 | 26.579 | 4.860 | 0.037 | 0.077 | 5.524 | −4.897 | 0.114 | 5.561 | −0.010 | 4.849 | −5.571 | 4.973 |
0.074 | −0.186 | −0.136 | 4.860 | 18.059 | 5.329 | −0.112 | −0.248 | 4.611 | 5.217 | 5.081 | 5.403 | −4.538 | 0.322 | −4.723 |
5.356 | 0.264 | −0.053 | 0.037 | 5.329 | 27.066 | 5.621 | 5.568 | 5.605 | −5.314 | −5.366 | −5.578 | −0.249 | −0.212 | 0.016 |
−5.201 | −4.701 | 4.316 | 0.077 | −0.112 | 5.621 | 19.098 | −5.586 | −5.509 | −4.281 | 0.035 | 0.420 | 0.308 | 0.385 | −4.393 |
−5.383 | −0.203 | −5.258 | 5.524 | −0.248 | 5.568 | −5.586 | 24.156 | −5.320 | −0.018 | −5.276 | 0.185 | −0.063 | 5.461 | −0.266 |
−5.430 | −0.079 | 0.189 | −4.897 | 4.611 | 5.605 | −5.509 | −5.320 | 20.784 | 0.096 | 0.285 | 0.175 | 4.786 | −0.110 | 4.707 |
0.156 | −4.436 | 4.263 | 0.114 | 5.217 | −5.314 | −4.281 | −0.018 | 0.096 | 16.406 | −5.331 | −5.158 | 0.059 | 0.173 | −4.377 |
−0.027 | 0.062 | −5.311 | 5.561 | 5.081 | −5.366 | 0.035 | −5.276 | 0.285 | −5.331 | 22.358 | −5.393 | −0.312 | 5.249 | −0.250 |
−5.166 | 5.586 | −0.235 | −0.010 | 5.403 | −5.578 | 0.420 | 0.185 | 0.175 | −5.158 | −5.393 | 25.256 | −5.341 | −5.351 | 0.245 |
−5.093 | 5.401 | −0.371 | 4.849 | −4.538 | −0.249 | 0.308 | −0.063 | 4.786 | 0.059 | −0.312 | −5.341 | 17.121 | −5.029 | −4.478 |
−5.140 | 5.525 | 5.076 | −5.571 | 0.322 | −0.212 | 0.385 | 5.461 | −0.110 | 0.173 | 5.249 | −5.351 | −5.029 | 21.399 | 0.495 |
0.229 | −4.622 | 4.127 | 4.973 | −4.723 | 0.016 | −4.393 | −0.266 | 4.707 | −4.377 | −0.250 | 0.245 | −4.478 | 0.495 | 14.900 |
Matrix elements (mm2) | ||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
19.458 | 5.295 | −0.162 | −0.090 | 0.032 | 5.227 | −5.170 | −5.464 | −5.477 | 0.176 | 0.076 | −5.238 | −5.117 | −5.051 | 0.223 |
5.295 | 23.975 | 4.534 | 0.095 | −0.144 | 0.172 | −4.757 | −0.210 | −0.061 | −4.365 | 0.112 | 5.573 | 5.335 | 5.639 | −4.604 |
−0.162 | 4.534 | 15.441 | 5.430 | −0.147 | −0.090 | 4.285 | −5.252 | 0.166 | 4.309 | −5.310 | −0.277 | −0.363 | 5.117 | 4.100 |
−0.090 | 0.095 | 5.430 | 26.624 | 4.815 | 0.031 | 0.126 | 5.550 | −4.931 | 0.095 | 5.567 | −0.044 | 4.918 | −5.603 | 4.953 |
0.032 | −0.144 | −0.147 | 4.815 | 18.100 | 5.317 | −0.114 | −0.282 | 4.676 | 5.165 | 5.092 | 5.527 | −4.565 | 0.327 | −4.713 |
5.227 | 0.172 | −0.090 | 0.031 | 5.317 | 26.772 | 5.547 | 5.549 | 5.596 | −5.253 | −5.328 | −5.442 | −0.287 | −0.235 | 0.051 |
−5.170 | −4.757 | 4.285 | 0.126 | −0.114 | 5.547 | 19.018 | −5.600 | −5.502 | −4.231 | 0.042 | 0.410 | 0.318 | 0.297 | −4.363 |
−5.464 | −0.210 | −5.252 | 5.550 | −0.282 | 5.549 | −5.600 | 24.192 | −5.269 | −0.053 | −5.306 | 0.243 | −0.031 | 5.451 | −0.210 |
−5.477 | −0.061 | 0.166 | −4.931 | 4.676 | 5.596 | −5.502 | −5.269 | 20.772 | 0.102 | 0.230 | 0.319 | 4.741 | −0.118 | 4.662 |
0.176 | −4.365 | 4.309 | 0.095 | 5.165 | −5.253 | −4.231 | −0.053 | 0.102 | 16.331 | −5.333 | −5.176 | 0.078 | 0.212 | −4.356 |
0.076 | 0.112 | −5.310 | 5.567 | 5.092 | −5.328 | 0.042 | −5.306 | 0.230 | −5.333 | 22.329 | −5.380 | −0.323 | 5.189 | −0.289 |
−5.238 | 5.573 | −0.277 | −0.044 | 5.527 | −5.442 | 0.410 | 0.243 | 0.319 | −5.176 | −5.380 | 25.298 | −5.353 | −5.311 | 0.239 |
−5.117 | 5.335 | −0.363 | 4.918 | −4.565 | −0.287 | 0.318 | −0.031 | 4.741 | 0.078 | −0.323 | −5.353 | 17.126 | −5.059 | −4.450 |
−5.051 | 5.639 | 5.117 | −5.603 | 0.327 | −0.235 | 0.297 | 5.451 | −0.118 | 0.212 | 5.189 | −5.311 | −5.059 | 21.412 | 0.491 |
0.223 | −4.604 | 4.100 | 4.953 | −4.713 | 0.051 | −4.363 | −0.210 | 4.662 | −4.356 | −0.289 | 0.239 | −4.450 | 0.491 | 14.817 |
Matrix elements (mm2) | ||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
23.867 | 2.552 | −0.210 | −0.005 | 0.210 | 2.671 | −2.605 | −2.985 | −2.921 | 0.020 | −0.253 | −3.042 | −2.940 | −2.710 | 0.317 |
2.552 | 30.580 | 2.193 | 0.127 | −0.078 | −0.056 | −2.166 | −0.208 | −0.296 | −1.808 | 0.071 | 2.711 | 2.569 | 2.783 | −2.116 |
−0.210 | 2.193 | 18.706 | 2.920 | −0.227 | −0.027 | 2.399 | −2.847 | 0.364 | 2.607 | −2.965 | −0.488 | −0.246 | 2.914 | 2.334 |
−0.005 | 0.127 | 2.920 | 32.974 | 2.183 | −0.034 | 0.276 | 2.662 | −2.225 | 0.097 | 2.558 | −0.063 | 2.175 | −2.853 | 2.544 |
0.210 | −0.078 | −0.227 | 2.183 | 22.781 | 2.792 | −0.100 | −0.495 | 2.430 | 2.760 | 2.654 | 2.934 | −2.500 | 0.264 | −2.609 |
2.671 | −0.056 | −0.027 | −0.034 | 2.792 | 32.067 | 2.829 | 2.830 | 2.942 | −2.849 | −2.804 | −2.643 | −0.138 | 0.054 | −0.069 |
−2.605 | −2.166 | 2.399 | 0.276 | −0.100 | 2.829 | 23.804 | −2.987 | −3.015 | −2.135 | −0.131 | 0.710 | 0.463 | 0.551 | −2.317 |
−2.985 | −0.208 | −2.847 | 2.662 | −0.495 | 2.830 | −2.987 | 29.950 | −2.372 | −0.009 | −2.458 | 0.229 | −0.019 | 2.724 | −0.429 |
−2.921 | −0.296 | 0.364 | −2.225 | 2.430 | 2.942 | −3.015 | −2.372 | 25.646 | 0.001 | 0.226 | 0.408 | 2.731 | −0.130 | 2.486 |
0.020 | −1.808 | 2.607 | 0.097 | 2.760 | −2.849 | −2.135 | −0.009 | 0.001 | 20.687 | −3.006 | −2.415 | 0.140 | 0.440 | −2.468 |
−0.253 | 0.071 | −2.965 | 2.558 | 2.654 | −2.804 | −0.131 | −2.458 | 0.226 | −3.006 | 27.456 | −2.900 | −0.169 | 2.689 | −0.449 |
−3.042 | 2.711 | −0.488 | −0.063 | 2.934 | −2.643 | 0.710 | 0.229 | 0.408 | −2.415 | −2.900 | 30.696 | −2.870 | −2.564 | 0.479 |
−2.940 | 2.569 | −0.246 | 2.175 | −2.500 | −0.138 | 0.463 | −0.019 | 2.731 | 0.140 | −0.169 | −2.870 | 21.329 | −2.811 | −2.421 |
−2.710 | 2.783 | 2.914 | −2.853 | 0.264 | 0.054 | 0.551 | 2.724 | −0.130 | 0.440 | 2.689 | −2.564 | −2.811 | 26.342 | 0.777 |
0.317 | −2.116 | 2.334 | 2.544 | −2.609 | −0.069 | −2.317 | −0.429 | 2.486 | −2.468 | −0.449 | 0.479 | −2.421 | 0.777 | 18.573 |
All codes that support the findings of this study are available from the corresponding author upon request.
The authors declare that there are no conflicts of interest regarding the publication of this paper.
This work was supported by the Department of Science and Technology of the Brazilian Army. The authors would like to thank the research group “Controle de Qualidade e Inteligência Computacional em Geodesia” (dgp.cnpq.br/dgp/espelhogrupo/0178611310347329).