For most of the time, biomedical researchers have been dealing with ordinal outcome variable in multilevel models where patients are nested in doctors. We can justifiably apply multilevel cumulative logit model, where the outcome variable represents the mild, severe, and extremely severe intensity of diseases like malaria and typhoid in the form of ordered categories. Based on our simulation conditions, Maximum Likelihood (ML) method is better than Penalized Quasilikelihood (PQL) method in threecategory ordinal outcome variable. PQL method, however, performs equally well as ML method where fivecategory ordinal outcome variable is used. Further, to achieve power more than 0.80, at least 50 groups are required for both ML and PQL methods of estimation. It may be pointed out that, for fivecategory ordinal response variable model, the power of PQL method is slightly higher than the power of ML method.
Data collected from hospitals and educational institutions are mostly multilevel or hierarchical data. This type of data is frequently used by researchers to construct statistical models such as multilevel models, hierarchical models, or mixed effects models [
It is really challenging to decide about an appropriate sample size for multilevel ordinal logistic models. In the contemporary literature, only [
A very popular concept is used in social sciences to develop a dichotomous multilevel logistic model through a latent continuous variable model [
That is why the squared sigma
Now
For identification purpose, it is common to set the first threshold to zero and to allow an intercept in the model [
The fixed effect parameters
The accuracy of different fixed effect and random effect parameters estimates was calculated through the relative parameter bias, that is,
Average relative bias of fixed effect estimates obtained as a function of groups, group size, and ICC, collapsing over the category distribution and number of categories.
Groups  Group size  ICC  ML method  PQL method  






 
30  5  0.1  0.0953  0.0960  0.0523  −0.0925  −0.0858  −0.0810 
0.3  0.0790  0.0823  0.0397  −0.1061  −0.0967  −0.0899  
0.5  0.0636  0.0675  0.0313  −0.1186  −0.1074  −0.0999  


30  30  0.1  0.0708  0.0376  0.0469  −0.0601  −0.0595  −0.0603 
0.3  0.0577  0.0292  0.0496  −0.0677  −0.0714  −0.0797  
0.5  0.0506  0.0263  0.0347  −0.0732  −0.0816  −0.0878  


30  50  0.1  0.0314  0.0562  0.0785  −0.0427  −0.0433  −0.0452 
0.3  0.0256  0.0507  0.0778  −0.0539  −0.0551  −0.0720  
0.5  0.0161  0.0466  0.0696  −0.0642  −0.0598  −0.0829  


50  5  0.1  0.0750  0.0607  0.0656  −0.0587  −0.0616  −0.0678 
0.3  0.0577  0.0485  0.0558  −0.0645  −0.0669  −0.0763  
0.5  0.0471  0.0489  0.0423  −0.0742  −0.0690  −0.0796  


50  30  0.1  0.0175  0.0196  0.0285  −0.0410  −0.0518  −0.0506 
0.3  0.0202  0.0154  0.0219  −0.0474  −0.0443  −0.0559  
0.5  0.0160  0.0154  0.0337  −0.0536  −0.0570  −0.0611  


50  50  0.1  0.0281  0.0248  0.0501  −0.0252  −0.0384  −0.0400 
0.3  0.0340  0.0260  0.0333  −0.0391  −0.0370  −0.0447  
0.5  0.0265  0.0243  0.0307  −0.0412  −0.0402  −0.0400  


100  5  0.1  0.0319  0.0357  0.0421  −0.0520  −0.0505  −0.0472 
0.3  0.0188  0.0169  0.0255  −0.0549  −0.0416  −0.0357  
0.5  0.0235  0.0188  0.0211  −0.0518  −0.0536  −0.0303  


100  30  0.1  −0.0025  0.0066  0.0092  −0.0206  −0.0270  −0.0236 
0.3  −0.0033  0.0044  0.0060  −0.0216  −0.0324  −0.0230  
0.5  −0.0037  0.0019  0.0041  −0.0248  −0.0172  −0.0182  


100  50  0.1  0.0064  0.0069  0.0041  −0.0102  −0.0114  −0.0084 
0.3  0.0051  0.0048  0.0025  −0.0105  −0.0125  −0.0243  
0.5  0.0030  0.0031  0.0042  −0.0148  −0.0120  −0.0205 
Average relative bias of random effect estimates obtained as a function of groups, group size, and ICC, collapsed over the category distribution and number of categories.
Groups  Group size  ICC  ML method  PQL method  




 
30  5  0.1  −0.1106  0.2000  −0.0812  0.1143 
0.3  −0.0701  −0.0176  −0.1158  −0.1403  
0.5  −0.0326  0 .0127  −0.1469  −0.1725  


30  30  0.1  −0.0806  −0.0572  −0.0645  −0.1429 
0.3  −0.0578  −0.0702  −0.0926  −0.1053  
0.5  −0.0327  −0.0633  −0.1196  −0.1311  


30  50  0.1  −0.0645  −0.0858  −0.0483  −0.1066 
0.3  −0.0578  −0.0351  −0.0689  −0.0878  
0.5  −0.0490  −0.0620  −0.0794  −0.1079  


50  5  0.1  −0.0645  0.0286  −0.0967  0.0282 
0.3  −0.0330  −0.0702  −0.1239  −0.1555  
0.5  0.0226  −0.0507  −0.1414  −0.1809  


50  30  0.1  −0.0483  −0.1143  −0.0725  −0.1525 
0.3  −0.0413  −0.0527  −0.0893  −0.1209  
0.5  −0.0272  −0.0380  −0.1022  −0.1461  


50  50  0.1  −0.0483  −0.0572  −0.0512  −0.0882 
0.3  −0.0330  −0.0176  −0.0656  −0.0933  
0.5  −0.0272  −0.0291  −0.0751  −0.1131  


100  5  0.1  −0.0322  0.0000  −0.0754  −0.1220 
0.3  −0.0330  −0.0179  −0.1063  −0.1425  
0.5  0.0055  −0.0254  −0.1095  −0.1520  


100  30  0.1  −0.0322  −0.0286  −0.0442  −0.0832 
0.3  −0.0328  −0.0176  −0.0611  −0.1051  
0.5  −0.0055  0.0000  −0.0709  −0.1165  


100  50  0.1  −0.0330  0.0000  −0.0391  −0.0595 
0.3  −0.0333  −0.0180  −0.0476  −0.0682  
0.5  −0.0540  0.0000  −0.0465  −0.0729 
Similarly, power was computed as
Empirical coverage rates of 95% confidence intervals were used to judge the accuracy of the standard errors of estimated parameters [
Tables
Tables
Power of fixed effect estimates of fivecategory response multilevel ordinal logistic model (method: ML and PQL, category distribution: symmetrical).
Groups  Group size  ICC  ML method  PQL method  






 
30  5  0.1  0.536  0.551  0.488  0.607  0.611  0.523 
0.3  0.561  0.559  0.492  0.596  0.621  0.539  
0.5  0.572  0.550  0.501  0.599  0.616  0.531  


30  30  0.1  0.624  0.661  0.569  0.661  0.659  0.591 
0.3  0.639  0.639  0.589  0.650  0.672  0.602  
0.5  0.635  0.669  0.594  0.658  0.649  0.611  


30  50  0.1  0.715  0.709  0.661  0.739  0.693  0.681 
0.3  0.731  0.721  0.679  0.749  0.716  0.699  
0.5  0.739  0.732  0.703  0.745  0.726  0.693  


50  5  0.1  0.850  0.835  0.731  0.886  0.849  0.756 
0.3  0.861  0.849  0.729  0.897  0.863  0.769  
0.5  0.873  0.858  0.740  0.897  0.869  0.763  


50  30  0.1  0.871  0.864  0.778  0.903  0.914  0.809 
0.3  0.877  0.856  0.789  0.914  0.895  0.795  
0.5  0.869  0.879  0.768  0.909  0.899  0.799  


50  50  0.1  0.891  0.859  0.815  0.956  0.916  0.849 
0.3  0.882  0.880  0.826  0.969  0.931  0.871  
0.5  0.896  0.878  0.831  0.972  0.939  0.878  


100  5  0.1  0.992  1.000  0.851  1.000  1.000  0.914 
0.3  0.998  1.000  0.859  1.000  0.997  0.902  
0.5  0.998  0.995  0.867  1.000  1.000  0.909  


100  30  0.1  0.995  0.989  0.890  1.000  1.000  0.929 
0.3  0.997  0.998  0.897  1.000  1.000  0.936  
0.5  0.992  0.998  0.879  1.000  1.000  0.939  


100  50  0.1  0.989  0.992  0.906  1.000  1.000  0.952 
0.3  0.996  0.996  0.919  1.000  1.000  0.959  
0.5  0.996  0.997  0.899  1.000  1.000  0.951 
95% CI coverage rates for the estimates in a threecategory ordinal response variable model by groups, group size, and intraclass correlation (method: ML, category distribution: symmetrical).
Parameters  Number of groups  Group size  ICC  

30  100  120 

5  30  50 

0.1  0.3  0.5 



0.942  0.949  0.953  0.0006  0.948  0.946  0.949  0.7886  0.945  0.948  0.951  0.0270 

0.941  0.947  0.952  0.0008  0.946  0.946  0.950  0.3865  0.943  0.947  0.951  0.0165 

0.939  0.946  0.951  0.0007  0.945  0.945  0.947  0.4895  0.946  0.946  0.946  0.9737 

0.905  0.908  0.918  0.0041  0.908  0.910  0.913  0.3086  0.911  0.910  0.909  0.6199 

0.907  0.915  0.927  0.0000  0.912  0.917  0.920  0.0632  0.918  0.913  0.918  0.8929 
95% CI coverage rates for the estimates in a threecategory ordinal response variable model by groups, group size, and intraclass correlation (method: ML, category distribution: skewed).
Parameters  Number of groups  Group size  ICC  

30  100  120 

5  30  50 

0.1  0.3  0.5 



0.941  0.948  0.951  0.0053  0.946  0.948  0.949  0.5283  0.943  0.948  0.949  0.0503 

0.946  0.949  0.951  0.1780  0.947  0.948  0.950  0.4000  0.947  0.948  0.949  0.7360 

0.942  0.946  0.955  0.0001  0.944  0.950  0.949  0.0821  0.948  0.947  0.948  0.9733 

0.900  0.908  0.917  0.0000  0.904  0.908  0.913  0.0494  0.908  0.908  0.910  0.7760 

0.909  0.913  0.922  0.0011  0.909  0.915  0.920  0.0077  0.918  0.913  0.918  0.8939 
95% CI coverage rates for the estimates in a fivecategory ordinal response variable model by groups, group size, and intraclass correlation (method: ML, category distribution: symmetrical).
Parameters  Number of groups  Group size  ICC  

30  100  120 

5  30  50 

0.1  0.3  0.5 



0.946  0.948  0.950  0.2542  0.945  0.949  0.951  0.0702  0.947  0.946  0.951  0.3144 

0.945  0.946  0.949  0.4840  0.945  0.947  0.948  0.2440  0.947  0.946  0.947  0.9730 

0.943  0.947  0.949  0.0490  0.946  0.947  0.946  0.9470  0.948  0.944  0.948  0.9210 

0.904  0.907  0.918  0.0001  0.909  0.910  0.912  0.4497  0.907  0.912  0.912  0.2305 

0.903  0.915  0.926  0.0000  0.911  0.914  0.918  0.0585  0.914  0.915  0.914  1.0000 
95% CI coverage rates for the estimates in a fivecategory ordinal response variable model by groups, group size, and intraclass correlation (method: ML, category distribution: skewed).
Parameters  Number of groups  Group size  ICC  

30  100  120 

5  30  50 

0.1  0.3  0.5 



0.939  0.944  0.946  0.0340  0.947  0.933  0.948  0.8470  0.944  0.943  0.941  0.3860 

0.945  0.947  0.949  0.1940  0.951  0.937  0.953  0.6170  0.947  0.947  0.947  0.9470 

0.944  0.947  0.948  0.1650  0.948  0.939  0.951  0.4080  0.944  0.947  0.947  0.3720 

0.903  0.911  0.916  0.0024  0.910  0.904  0.917  0.0945  0.908  0.911  0.912  0.3885 

0.907  0.916  0.922  0.0003  0.912  0.910  0.922  0.0328  0.913  0.915  0.916  0.5935 
95% CI coverage rates for the estimates in a threecategory ordinal response variable model by groups, group size, and intraclass correlation (method: PQL, category distribution: symmetrical).
Parameters  Number of groups  Group size  ICC  

30  100  120 

5  30  50 

0.1  0.3  0.5 



0.921  0.925  0.928  0.1135  0.918  0.926  0.932  0.0030  0.925  0.926  0.922  0.5156 

0.918  0.922  0.925  0.0671  0.917  0.922  0.928  0.0170  0.928  0.924  0.912  0.0001 

0.923  0.927  0.929  0.1099  0.920  0.926  0.933  0.0014  0.931  0.929  0.918  0.0009 

0.897  0.900  0.907  0.0087  0.896  0.902  0.908  0.0044  0.905  0.903  0.895  0.0215 

0.903  0.906  0.908  0.1327  0.898  0.906  0.912  0.0018  0.910  0.909  0.897  0.0055 
95% CI coverage rates for the estimates in a threecategory ordinal response variable model by groups, group size, and intraclass correlation (method: PQL, category distribution: skewed).
Parameters  Number of groups  Group size  ICC  

30  100  120 

5  30  50 

0.1  0.3  0.5 



0.918  0.920  0.924  0.1155  0.915  0.919  0.927  0.0053  0.928  0.923  0.912  0.0000 

0.921  0.926  0.928  0.0245  0.920  0.927  0.931  0.0044  0.931  0.929  0.917  0.0002 

0.921  0.924  0.928  0.0965  0.918  0.924  0.931  0.0031  0.929  0.926  0.918  0.0032 

0.899  0.902  0.905  0.0567  0.897  0.902  0.907  0.0274  0.903  0.905  0.898  0.1459 

0.905  0.908  0.911  0.1633  0.901  0.910  0.914  0.0015  0.916  0.909  0.900  0.0003 
95% CI coverage rates for the estimates in a fivecategory ordinal response variable model by groups, group size, and intraclass correlation (method: PQL, category distribution: symmetrical).
Parameters  Number of groups  Group size  ICC  

30  100  120 

5  30  50 

0.1  0.3  0.5 



0.940  0.945  0.942  0.6312  0.936  0.942  0.948  0.0007  0.944  0.945  0.937  0.0590 

0.940  0.945  0.946  0.0997  0.938  0.940  0.948  0.0027  0.945  0.945  0.940  0.1292 

0.938  0.944  0.942  0.2029  0.937  0.942  0.946  0.0156  0.943  0.942  0.939  0.2029 

0.906  0.910  0.911  0.2328  0.904  0.910  0.914  0.0458  0.913  0.911  0.903  0.0196 

0.907  0.907  0.909  0.6620  0.904  0.909  0.910  0.2170  0.909  0.910  0.902  0.1050 
95% CI coverage rates for the estimates in a fivecategory ordinal response variable model by groups, group size, and intraclass correlation (method: PQL, category distribution: skewed).
Parameters  Number of groups  Group size  ICC  

30  100  120 

5  30  50 

0.1  0.3  0.5 



0.941  0.941  0.943  0.5893  0.937  0.942  0.946  0.0047  0.944  0.943  0.937  0.0611 

0.939  0.940  0.941  0.5301  0.935  0.940  0.946  0.0013  0.942  0.941  0.937  0.1578 

0.942  0.943  0.944  0.6300  0.936  0.944  0.950  0.0000  0.946  0.944  0.938  0.0160 

0.903  0.907  0.909  0.1598  0.901  0.908  0.910  0.0658  0.907  0.907  0.904  0.6272 

0.909  0.911  0.914  0.3463  0.905  0.911  0.917  0.0060  0.915  0.915  0.903  0.0037 
Table
Similarly, ML method random effects estimates were substantially biased when the number of groups was 30 as shown in Table
Table
Table
Table
Table
Table
Table
Table
Table
Table
In both symmetrical and skewed distribution shapes of category responses,
The accuracy of standard errors of the estimates (excluding threshold estimates) was judged through empirical coverage rates. The influence of the number of groups was significant on the accuracy of both threecategory and fivecategory multilevel ordinal logistic models estimates standard errors under ML method of estimation. On the contrary, the group size factor and ICC had an insignificant effect on estimates standard errors under ML method of estimation. Estimates standard errors were least biased when number of groups was 100. On the other hand, the influence of group size factor was highly significant on the accuracy of estimates standard errors under PQL method of estimation. Furthermore, ICC also influenced estimates standard errors; that is, standard errors were substantially biased when population random effects were medium (
The power rates of PQL estimates were slightly higher than that of ML estimates when ordinal response variable had five categories, which indicate that PQL standard errors were least biased due to increases in the number of categories of ordinal response variable.
In general, ML method performed well in terms of estimates small bias, high coverage rates, and high power rates when ordinal response variable had three categories. However, in fivecategory ordinal response variable model, PQL method performances were comparable to those of ML method. PQL estimates were poor when population random effects (ICC) were medium and large while ML estimates were poor in small population random effects. In addition, ML estimates and estimates standard errors benefitted from larger number of groups while PQL estimates and estimates standard errors benefitted from larger group sizes. We recommend at least 100 groups and 30 units per group to achieve accurate multilevel ordinal logistic model estimates and estimates standard errors when method of estimation is ML. Furthermore, 100 groups and at least 50 units per group are essential for accurate multilevel ordinal logistic model estimates and estimates standard errors when method of estimation is PQL. Similarly, at least 50 groups are essential to achieve 0.80 or more power for both ML and PQL methods of estimation. On the basis of this study, it is recommended that PQL method may be avoided when group sizes are small, number of groups are large, random effects are medium and large, and outcome variable has three categories. In such conditions, ML method is the best option. However, when the outcome variable has five or more categories, random effects are small, group sizes are large, and number of groups is small, PQL method may be better option.
The authors declare that there is no conflict of interests regarding publication of this paper.