^{1}

^{1}

Sensitivity and specificity as the reference value of the ability to detect sick and healthy patients are used in diagnostic test evaluation with a gold standard test. However, in clinical practice, the gold standard tests are not given in patients due to expensive or invasive reasons [

Some studies try to evaluate the diagnostic test characteristics by combining multiple diagnostic tests in the absence of a gold standard [

In classical statistical view, sensitivity and specificity are regarded as fixed parameters and the population prevalence is calculated from them. However, it has been proved that sensitivity and specificity are not fixed values, but change with external factors [

Bayesian methods have been increasingly used to evaluate the true accuracy of diagnostic tests in the absence of a gold standard [

In the combined application of multiple diagnostic tests, the interdependence between different tests also needs to be considered in the Bayesian model. If two tests have the same biological attribute, it is logical to believe that the tests are conditionally dependent; if the result is positive in one test, the result of another test is likely to be positive [

Under the basic Bayesian framework, the Bayesian method is very flexible when considering various influencing factors. The correlation scenarios, prior distribution, and the number of the prior parameters are the important factors that cannot be ignored in the Bayesian estimation. The objective of this study is to compare the two Bayesian methods under different scenarios with tuberculosis (TB) data and to explore the application scenarios for each of the two Bayesian methods.

The data analyzed in this study was gathered during studies of patients presenting to the Nanjing Chest Hospital with suspected tuberculosis. In brief, a case report of patients was collected between June and October 2015 at the Nanjing Chest Hospital. Informed consent was completed for all participated in the study. T-SPOT.Tb and KD38 tuberculosis antibody test was combined as the nongold standard diagnostic test to estimate the prevalence and diagnostic test characteristic. The discharge diagnosis was used to verify the model results.

Conditional covariance Bayesian model directly considers the correlation between the two tests and estimates the conditional correlation between two diagnostic tests using the covariance between tests within the diseased and nondiseased populations. Two diagnostic test evaluation models without a gold standard were shown in Table

Two diagnostic test evaluation models without the gold standard.

True result(+) | Total | True result(-) | Total | ||||
---|---|---|---|---|---|---|---|

Test2 | Test2 | ||||||

+ | - | + | - | ||||

Test1 | + | | | | | | |

- | | | | | | | |

Total | | | | | | |

Vector

According to the Bayesian principle, the conjugate distribution of the binomial distribution is the beta distribution. The prevalence, sensitivities, and specificities are assumed to follow beta prior distribution, list like the following:

The prior information of the Conditional Covariance Bayesian model.

Method | Knot | N | Mean | VAR | SD | CI | a | b |
---|---|---|---|---|---|---|---|---|

T-SPOT | | 18 | 0.893 | 0.002 | 0.049 | 0.869-0.917 | 41.770 | 5.005 |

| 18 | 0.848 | 0.010 | 0.098 | 0.799-0.896 | 10.082 | 1.807 | |

KD38 | | 9 | 0.572 | 0.018 | 0.132 | 0.47-0.674 | 7.207 | 5.393 |

| 9 | 0.717 | 0.007 | 0.082 | 0.654-0.781 | 20.067 | 7.920 | |

Prev | | 11 | 0.417 | 0.023 | 0.153 | 0.315-0.52 | 3.991 | 5.579 |

In practical application, there is a lack of available prior information in covariance

The conditionally independent assumption is usually made when two diagnostic tests were combined. However, the conditionally independent assumption cannot easily be made when the two diagnostics have a similar biologic mechanism; extra information will be required in the estimation process [

However, in some cases, it is often difficult to directly specify external prior information for some parameters, such as the covariance in the Conditional Covariance Bayesian model. The prior distributions for the covariance are quite difficult to elicit from experts or other studies, because they are not the indicator used in a real-life situation. In Bayesian probabilistic constraint model, prior information on conditional probabilities is easier to specify [

The likelihood function was used to express the cell probabilities of the collapsed

The prior information about Bayesian probabilistic constraints model was collected from four experienced tuberculosis physicians. The tuberculosis physician answered the probability of the parameter under the defined question. After obtaining the expert answer, the mean of each parameter had been calculated, and the prior distribution also had been specified. This study was a joint evaluation of two diagnostic tests,

The prior information about the Bayesian probabilistic constraint model.

Parameter | Knot | N | D1 | D2 | D3 | D4 | Mean | Alpha | Beta |
---|---|---|---|---|---|---|---|---|---|

| P2.5 | 4 | 0.3 | 0.5 | 0.7 | 0.3 | 0.450 | 75.83 | 66.73 |

P97.5 | 4 | 0.55 | 0.7 | 0.8 | 0.4 | 0.613 | |||

| P2.5 | 4 | 0.9 | 0.7 | 0.8 | 0.6 | 0.572 | 70.84 | 35.86 |

P97.5 | 4 | 0.99 | 0.8 | 0.9 | 0.8 | 0.750 | |||

| P2.5 | 4 | 0.7 | 0.7 | 0.7 | 0.8 | 0.417 | 21.90 | 16.21 |

P97.5 | 4 | 0.8 | 0.8 | 0.8 | 0.9 | 0.725 | |||

| P2.5 | 4 | 0.7 | 0.4 | 0.4 | 0.75 | 0.563 | 99.74 | 56.18 |

P97.5 | 4 | 0.8 | 0.6 | 0.6 | 0.85 | 0.713 | |||

| P2.5 | 4 | 0.3 | 0.3 | 0.4 | 0.1 | 0.275 | 53.43 | 100.05 |

P97.5 | 4 | 0.5 | 0.5 | 0.5 | 0.2 | 0.425 | |||

| P2.5 | 4 | 0.7 | 0.4 | 0.4 | 0.75 | 0.563 | 143.25 | 85.38 |

P97.5 | 4 | 0.8 | 0.6 | 0.5 | 0.85 | 0.688 | |||

| P2.5 | 4 | 0.2 | 0.3 | 0.6 | 0.5 | 0.400 | 112.32 | 130.69 |

P97.5 | 4 | 0.3 | 0.5 | 0.7 | 0.6 | 0.525 |

D1: Doctor 1; D2: Doctor 2; D3: Doctor 3; D4: Doctor4. Alpha and Beta are two parameters of the beta distribution.

All parameters in two Bayesian methods were estimated with 95% credible intervals using OpenBUGS 3.2.3[

The prediction accuracy of the different models was evaluated using clinical discharge diagnosis as the gold standard. The clinical discharge diagnosis was a comprehensive judgment made by doctors according to various diagnostic tests, expert experience, and disease progression.

In total, 637 patients with suspected tuberculosis were included in the study. The mean age was 50.12 years (range 15-90 years); 61.3% of the patients were male and 38.7% were female. 130 patients (20.41%) were negative for T-SPOT.TB test and KD38 tuberculosis antibody test, 235 patients (36.89%) were positive for both of them, the four possible combinations of results for the two tests were listed (Table

The results of 637 persons subjected to 2 diagnostic tests.

T-SPOT | KD38 | Count |
---|---|---|

- | - | 130 |

- | + | 81 |

+ | - | 191 |

+ | + | 235 |

- indicates negative test result and + positive test result.

Four models under conditional independence situation were applied to the data for the two tests, which assumed that the result of the first test had no influence on the result of the second test. Using the observed of the two tests as the sample data, combined with prior information, we calculated the posterior distribution of sensitivity and specificity of the two tests (Table

The posterior estimation of four models using the TB data under conditional independence situation.

Situation | Method | Model | Knot | Mean | SD | Median | 95% Bayesian CI |
---|---|---|---|---|---|---|---|

(P2.5-P97.5) | |||||||

No prior constraints | Bayesian probabilistic constraint model | NP | | 0.660 | 0.243 | 0.762 | 0.503-0.841 |

| 0.418 | 0.251 | 0.426 | 0.203-0.584 | |||

| 0.511 | 0.251 | 0.568 | 0.302-0.688 | |||

| 0.569 | 0.249 | 0.631 | 0.377-0.759 | |||

| 0.512 | 0.194 | 0.515 | 0.361-0.667 | |||

Conditional | NC | | 0.618 | 0.250 | 0.631 | 0.071-0.974 | |

| 0.377 | 0.250 | 0.295 | 0.025-0.929 | |||

| 0.468 | 0.255 | 0.435 | 0.036-0.947 | |||

| 0.526 | 0.253 | 0.480 | 0.055-0.962 | |||

| 0.498 | 0.194 | 0.497 | 0.148-0.850 | |||

| |||||||

Prior constraints | Bayesian probabilistic constraint model | PP | | 0.738 | 0.036 | 0.739 | 0.714-0.762 |

| 0.454 | 0.051 | 0.453 | 0.419-0.487 | |||

| 0.585 | 0.031 | 0.585 | 0.563-0.606 | |||

| 0.525 | 0.028 | 0.525 | 0.506-0.544 | |||

| 0.534 | 0.042 | 0.534 | 0.506-0.562 | |||

Conditional | PC | | 0.898 | 0.039 | 0.910 | 0.814-0.963 | |

| 0.765 | 0.125 | 0.775 | 0.515-0.968 | |||

| 0.594 | 0.048 | 0.586 | 0.523-0.712 | |||

| 0.679 | 0.048 | 0.676 | 0.592-0.780 | |||

| 0.636 | 0.086 | 0.650 | 0.435-0.773 |

The results of fitting indicator between four models under the conditional independence situation.

Model | DIC | | | T-SPOT | KD38 | ||
---|---|---|---|---|---|---|---|

| | | | ||||

Method NP | 3.128 | -19.05 | 0.512 | 0.660 | 0.418 | 0.511 | 0.569 |

Method NC | 1.404 | -20.80 | 0.498 | 0.660 | 0.418 | 0.511 | 0.569 |

Method PP | 38.56 | 1.343 | 0.5334 | 0.738 | 0.454 | 0.585 | 0.525 |

Method PC | 24.15 | 2.264 | 0.636 | 0.898 | 0.765 | 0.594 | 0.679 |

Conditional dependence situation assumed that the two diagnostic tests could be correlated. The posterior distributions of sensitivity and specificity of the two tests under conditional dependence situation were evaluated by four models (Table

The posterior estimation of four models using the TB data under conditional dependence situation.

Situation | Method | Model | Knot | Mean | SD | Median | 95% Bayesian CI |
---|---|---|---|---|---|---|---|

(P25-P75) | |||||||

No prior constraints | Bayesian probabilistic constraint model | NP | | 0.626 | 0.218 | 0.665 | 0.516-0.773 |

| 0.369 | 0.217 | 0.330 | 0.222-0.476 | |||

| 0.490 | 0.167 | 0.495 | 0.391-0.585 | |||

| 0.508 | 0.167 | 0.503 | 0.414-0.616 | |||

| 0.498 | 0.248 | 0.497 | 0.305-0.692 | |||

Conditional | NC | | 0.690 | 0.183 | 0.708 | 0.622-0.807 | |

| 0.435 | 0.226 | 0.390 | 0.287-0.568 | |||

| 0.547 | 0.197 | 0.535 | 0.455-0.657 | |||

| 0.571 | 0.213 | 0.562 | 0.444-0.722 | |||

| 0.553 | 0.254 | 0.578 | 0.366-0.755 | |||

| |||||||

Prior constraints | Bayesian probabilistic constraint model | PP | | 0.713 | 0.036 | 0.714 | 0.689-0.738 |

| 0.423 | 0.051 | 0.421 | 0.387-0.456 | |||

| 0.535 | 0.025 | 0.535 | 0.518-0.552 | |||

| 0.537 | 0.022 | 0.537 | 0.522-0.552 | |||

| 0.538 | 0.042 | 0.539 | 0.510-0.567 | |||

Conditional | PC | | 0.904 | 0.037 | 0.907 | 0.881-0.931 | |

| 0.796 | 0.119 | 0.814 | 0.715-0.892 | |||

| 0.588 | 0.055 | 0.580 | 0.551-0.614 | |||

| 0.677 | 0.068 | 0.677 | 0.631-0.723 | |||

| 0.649 | 0.076 | 0.662 | 0.608-0.701 |

The results of fitting indicator between four models under the conditional dependence situation.

Model | DIC | PD | p | T-SPOT | KD38 | ||
---|---|---|---|---|---|---|---|

| | | | ||||

Method NP | 19.01 | -3.005 | 0.498 | 0.626 | 0.369 | 0.490 | 0.508 |

Method NC | 14.32 | -7.692 | 0.553 | 0.690 | 0.435 | 0.547 | 0.571 |

Method PP | 24.26 | 1.657 | 0.538 | 0.713 | 0.423 | 0.535 | 0.537 |

Method PC | 24.40 | 2.40 | 0.649 | 0.904 | 0.797 | 0.588 | 0.677 |

The Conditional Covariance Bayesian model was chosen to explore the influence of the prior number on the posterior estimation because it has only five unknown parameters corresponding to only five prior distributions, which was convenient for simulation studies. When the number of priors was equal to n, it means that the rest of the prior (5-n) was prior without information. From the results of simulation under conditional independence, when the prior number was three, the estimation result and the model were stable (Table

The impact of the number of prior information on the assessment result (conditional independence situation).

Number | DIC | | p | T-SPOT | KD38 | ||
---|---|---|---|---|---|---|---|

| | | | ||||

0 | 1.404 | -20.8 | 0.5059 | 0.74 | 0.4078 | 0.5533 | 0.6127 |

1 | 18.12 | -3.992 | 0.4524 | 0.8125 | 0.4462 | 0.6763 | 0.6624 |

2 | 24.03 | 2.0 | 0.4357 | 0.8935 | 0.4948 | 0.7144 | 0.6678 |

3 | 24.21 | 2.174 | 0.6421 | 0.9056 | 0.7688 | 0.5856 | 0.6617 |

4 | 24.5 | 2.497 | 0.6482 | 0.9054 | 0.7788 | 0.5827 | 0.6612 |

The impact of the number of prior information on the assessment result (conditional dependence situation).

Number | DIC | | p | T-SPOT | KD38 | ||
---|---|---|---|---|---|---|---|

| | | | ||||

0 | 14.32 | -7.692 | 0.5776 | 0.7081 | 0.3903 | 0.5353 | 0.562 |

1 | 20.43 | -1.594 | 0.4503 | 0.7325 | 0.3825 | 0.5814 | 0.5804 |

2 | 24.32 | 2.298 | 0.455 | 0.8941 | 0.5115 | 0.5963 | 0.6073 |

3 | 24.56 | 2.503 | 0.6526 | 0.9075 | 0.7941 | 0.5548 | 0.6185 |

4 | 24.69 | 2.652 | 0.6579 | 0.9081 | 0.8063 | 0.5575 | 0.6219 |

The patient discharge diagnosis was used as the gold standard to evaluate the sensitivity and specificity of two diagnostic tests (Table

Post-hoc model validation with the gold standard.

Indicator | T-SPOT | KD38 | ||
---|---|---|---|---|

No. | 95% CI | No. | 95% CI | |

Sensitivity | 390/528 | 0.739 (0.699-0.776) | 290/528 | 0.549 (0.507-0.591) |

Specificity | 73/109 | 0.670 (0.573-0.757) | 83/109 | 0.761 (0.670-0.838) |

Prevalence | 528/637 | 0.829 (0.797-0.857) | 528/637 | 0.829 (0.797-0.857) |

With the development of computer technology and Bayesian theory, Bayesian model has been widely used in the practice of medical research. In the evaluation of diagnostic tests, when the real disease status is unknown and there is no gold standard, Bayesian method can be used to integrate external prior information and sample data to evaluate diagnostic test characteristics by combining two or more imperfect tests [

The result of the different model indicated that the estimate of prevalence rate and diagnostic test characteristics depends on the model chosen, prior selection, and dependencies between tests. Compared with the gold standard verification, the result of model PC in conditional independence situation was closest to the result of the gold standard evaluation. The reasons for the above phenomenon may be as follows: firstly, the weak correlation between diagnostic tests cannot have a significant effect on the result; secondly, the prior constraint model can reflect the real situation of the diagnostic test than the nonpriority constraint model; thirdly, the objective prior information from previous studies is more accurate than expert opinion.

The dependencies between diagnostic tests have always been a key issue for Bayesian models. The results of the two Bayesian models showed that the change for the possibility of conditional dependence between diagnostic tests had a certain impact on the posterior estimates of diagnostic test characteristics. The two Bayesian methods deal with the conditional dependencies between tests in different ways. Conditional covariance Bayesian method combined prior information on covariance parameters with the test result to calculate the posterior distribution of the correlation coefficients. However, obtaining the prior distribution of covariance from experts or literature is pretty hard, because it is not specific parameters in a real-life situation. In addition, the complex correlation will be difficult to estimate with multiple diagnostic tests. In order to overcome this problem, the Bayesian probabilistic constraint model does not directly calculate the correlation coefficient, it just elicits prior information for experts on the conditional performance on one test given the results of another test, and this can be easier to answer by experts in a real-life situation. However, such prior information from this model is the expert subjective opinion, and its credibility is not better than objective prior information.

Our results showed that the likelihood functions of the two Bayesian methods were consistent with the conditions of independence situation, and the posterior estimation strongly depended on the prior information. The results of the two Bayesian methods both illustrated that posterior estimation was mainly affected by the available prior information. Hence, it is very important to elicit the prior distribution accurately. On the one hand, the objectivity of prior information is crucial. In the Conditional Covariance Bayesian method, the prior distribution of unknown parameters can be gathered from previous studies, and objective prior information is suggested to ensure the credibility of the result. In the Bayesian probabilistic constraint model, it may be easier to specify expert prior information for unknown parameters, but it is also significant to realize that the unstable expert opinion may have a great impact on the result; if you use the prior by different experts, you may end up with distinctive conclusions. On the other hand, the number of prior information as well has an important effect on the stability of the results. As we all know, the more the number of a prior, the more accurate the result, but it will increase the burden of obtaining prior information. Our results show that three prior distributions can achieve full prior results in the Conditional Covariance Bayesian method. Therefore, obtaining stable results based on minimal prior information is the best choice.

In fact, the influences of prior information and dependencies on the results are inseparable. Because the correlation coefficient itself is an unknown parameter, it also requires the prior distribution. In the evaluation of diagnostic tests in the absence of the gold standard, many factors should be considered in the method selection. DIC is also an important index of the model selection. Both the Conditional Covariance Bayesian method and the Bayesian probabilistic constraint method have their specific applicable scenarios; the users should choose the appropriate method according to the needs of the actual situation. When there are only two diagnostic tests and the correlation coefficient can be objectively specified, the Conditional Covariance Bayesian method is more applicable. The Conditional Covariance Bayesian method could also be extended to include more than two tests by adding more covariance in the model. At this time, the calculation of covariance will become complex, and the determination of prior distribution will be more difficult. Hence, from the point of view of practical application, the Bayesian probabilistic constraint method is more suitable when there are more than two combined diagnostic tests without gold standards. Finally, although these two methods are not perfect, they provide a feasible way for the evaluation of diagnostic test in the absence of a gold standard diagnostic; at the same time, it is of great significance to promote the application of Bayesian method in medical research.

Both of the two Bayesian methods are the feasible way for the evaluation of diagnostic test in the absence of a gold standard diagnostic. Prior source, priority number, and conditional dependencies should be considered in the method selection, the accuracy of posterior estimation mainly depending on the prior distribution.

The data used to support the findings of this study are included within the supplementary information file (Table

The authors declare that they have no conflicts of interest.

Taishun Li analyzed and interpreted the patient data and was a major contributor in writing the manuscript. Pei Liu revised the manuscript and provided methodological guidance. All authors read and approved the final manuscript.

This work was supported by the Fundamental Research Funds for the Central Universities and Postgraduate Research & Practice Innovation Program of Jiangsu Province (Fund number: KYCX17_0186). We would like to thank Jiaying Yang for her help in the modification of the language of the article.