Comparative precision of laboratory methods

Different quantitation techniques or methods ofmeasurement are sometimes employed to measure the same substance in a laboratory. Higher precision, i.e. lower variability, is one of the criteria for selecting a better measurement method. This could be the case when one wants to compare a new (test) method to a standard (reference) method. When independent samples are used between methods, then it is well known that one can use the F-test to compare their variances. However, when the same samples are employed for both quantitative methods, the F-test is inappropriate for comparing the between sample variation and could fail to detect the more precise method.


Introduction
Sample Astra Flame Different quantitation techniques or methods of measurement are sometimes employed to measure the same substance in a laboratory. Higher precision, i.e. lower variability, is one of the criteria for selecting a better measurement method. This could be the case when one wants to compare a new (test) method to a standard (reference) method. When independent samples are used between methods, then it is well known that one can use the F-test to compare their variances. However, when the same samples are employed for both quantitative methods, the F-test is inappropriate for comparing the between sample variation and could fail to detect the more precise method.

Method
Denote by SDi the standard deviation of the quantitation method (i 1,,2) from Nsamples, and the variance ratio between the two methods: If the N samples for method are separate (independent) samples from those for method 2, then the F value in equation (1) behaves like an F-distribution with (N-1) and (N-1) degrees of freedom. One can therefore use this statistic for comparing their precision. However, if the same N samples are used for both methods, SD1 and SD2 are correlated and the F value is no longer distributed as an F-distribution. A modified test by Pitman is as follows" is the correlation coefficient between the two methods and F is defined in equation (1).

Example
To illustrate the Pitman's test an example has been taken from Griffiths et al. [3]: the sodium levels of 21 patient serum specimens were analysed by Beckman Astra-8 and flame photometry methods. The raw data are reproduced in table 1. From equation (1)  For the between sample variation (the same variability discussed in this note so far), one can take the average values over replicates and apply the Pitman's test on the sample averages. The individual replications do not enter into the statistical test directly.
The within sample variation measures the repeatability of the quantitation method on the same sample. Unlike the between sample variations, the within sample variations are not correlated between methods. In this instance one can use the F-test again for comparing the within sample variations. The statistical model with replication within sample is given in the Appendix.

Appendix
The model: yijk tjk + i + "[0" " ijk wherey/j is the determination from sample (i 1, N), quantitation method j (/" 1,2) and replicate k (k 1,..., R). jk is the mean response of method j and replicate k, and li, ij and e/k are the residual error terms due to sample i, method j and replicate k respectively. Further assumptions are made on the residual error terms such that they are mutually independent and normally distributed: 'OMj2) and 0 N(O, ow ) Denote by.} 0. the mean ofsample and methodj over the R replicates (i.e.)0. Zyo.k/R etc., then the between sample variance for method j: var 00) s 2 + OMj 2 + Ow 2/R and can be compared by the Pitman's test (using)o.).
owj is the within sample variance, i.e. the variation between determinations due to replication. It is estimated by: Swj Z (Yok )ij .j + ))2/(N 1) (R 1) i,k with (N 1) (R 1) degrees of freedom. The F-test can be used here to compare Swj 2's: F Swl/Sw22 is distributed as an F-distribution with (N-1) (R 1) and (N 1) (R 1) degrees of freedom.