BioMed Research International
Volume 2016 (2016), Article ID 6019603, 13 pages
http://dx.doi.org/10.1155/2016/6019603

Research Article

The Development and Validation of a Rapid Assessment Tool of Primary Care in China

Jie Mei,¹ Yuan Liang,¹ LeiYu Shi,² JingGe Zhao,¹ YuTan Wang,¹ and Li Kuang¹

¹Institute of Social Medicine and Health Management, School of Public Health, Sun Yat-sen University, 74 Zhongshan 2nd Road, Guangzhou, Guangdong 510080, China
²Johns Hopkins Bloomberg School of Public Health, Johns Hopkins Primary Care Policy Center, 624 North Broadway, Baltimore, MD 21205, USA

Received 5 October 2015; Accepted 22 December 2015

Academic Editor: Alberto Raggi

Copyright © 2016 Jie Mei et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

Introduction. With Chinese health care reform increasingly emphasizing the importance of primary care, the need for a tool to evaluate primary care performance and service delivery is clear. This study presents a methodology for a rapid assessment of primary care organizations and service delivery in China. Methods. The study translated and adapted the Primary Care Assessment Tool-Adult Edition (PCAT-AE) into a Chinese version to measure core dimensions of primary care, namely, first contact, continuity, comprehensiveness, and coordination. A cross-sectional survey was conducted to assess the validity and reliability of the Chinese Rapid Primary Care Assessment Tool (CR-PCAT). Eight community health centers in Guangdong province have been selected to participate in the survey. Results. A total of 1465 effective samples were included for data analysis. Eight items were eliminated following principal component analysis and reliability testing. The principal component analysis extracted five multiple-item scales (first contact utilization, first contact accessibility, ongoing care, comprehensiveness, and coordination). The tests of scaling assumptions were basically met. Conclusion. The standard psychometric evaluation indicates that the scales have achieved relatively good reliability and validity. The CR-PCAT provides a rapid and reliable measure of four core dimensions of primary care, which could be applied in various scenarios.

1. Introduction

Primary care is a fundamental part in the health care systems of both high and low income countries and there is ample evidence that primary care is closely related to the improvement of health outcomes [1, 2]. In 1994, the Institute of Medicine (IOM) proposed a function-orientated definition of primary care, which is consistent with the widely acknowledged multidimensional concept of primary care with its emphasis on the four core dimensions of primary care (i.e., first contact, continuity, comprehensiveness, and coordination) [3–5].

In urban China, primary care is defined as “the delivery of comprehensive, continuous, and convenient episodic and preventive health care services to families in the community” [6], which are mostly provided by general practitioners (GPs) in community health centers (CHCs) or other health stations or clinics [7]. Since 2009, the Chinese central government has launched a new health care reform, an essential part of which is to strengthen the primary care system [8–10]. Significant government investments have been made on developing GP workforce, constructing CHCs infrastructure, improving CHCs distribution network in order to improve the geographic accessibility, providing 12 basic public health services in CHCs, and so on [9, 11]. Meanwhile, the social health insurance have covered approximately 95% of the population and its reimbursement strategy has strengthened the use of primary care and the promotion of CHCs as a potential gatekeeper [12–14].

As an economically developed district in southern China, Guangdong province has long recognized the importance of primary care in promoting health for the general public and has launched pilot programs for regional primary care reform since 1996 [15]. There are some favorable changes; for example, general practitioners (GPs) who are at the frontier in providing primary care have gradually acquired a good local reputation and attracted large numbers of patients. In the meantime, a notable characteristic of the province is that various management patterns of CHCs emerged after years of reform and the performance of CHCs varied under different management patterns (discussed later).

Whilst there has been considerable government investment and policy attention on primary care, evidence of the quality of primary care services is urgently needed [10, 16]. Previous researches in China have been carried out to assess the quality of primary care in disease-focused or task-orientated manners [17–20]. Although these studies have been essential, they largely fell short on evidence of the core dimensions of primary care. A previous systematic review has demonstrated that the four core dimensions determine the primary care process [21]. Primary care is deemed stronger with better fulfillment of these dimensions [22]. Therefore, it is reasonable and necessary to develop a valid tool to measure the quality of primary care in a multidimensional manner and include its four core dimensions [23]. Several studies in developed countries have successfully developed valid tools in this way [24–27] and such evaluation tools have been used in China before [7, 28]. Our preliminary research tried out such tools in practice. However, the results indicated that they were time-consuming and generated a significant amount of “not sure/don’t remember” responses. Therefore, constructing a rapid assessment tool seems imperative so that the Chinese primary care evaluation tool becomes concise and user-friendly in order to obtain key information with minimum time cost as well as to ensure patient compliance and quality of data.

This study was undertaken in close collaboration with health authorities of Guangdong province in response to the need for a rapid assessment tool to measure the quality of primary care service delivery. The objective of this paper was first to construct a rapid assessment tool that was based on the core dimensions of primary care and, secondly, to apply analytic methods to assess the validity and reliability of this tool. In addition, analysis was undertaken to compare primary care achievements across different health care settings.

2. Methods

2.1. Instrument

Primary Care Assessment Tool (PCAT) has been developed based on the theoretical model of primary care attributes established by John Hopkins [29] and Donabedian’s quality framework [30]. PCAT has been highly recommended over other measurement tools when assessing the process of primary care and is available in various formats [26, 30–32]. Primary Care Assessment Tool-Adult Edition (PCAT-AE) is designed to measure adult patients’ experience of primary care. It has originally been used in the US [26] and has been adopted in several other countries with different health systems, including Brazil [33], Korea [34], Spain [35], and China [7, 28]. After certain adaptions, all these tools had been validated in their corresponding countries, and some of them have been used to measure quality of primary care [33, 36, 37].

In our study, the translation and adaption of PCAT-AE involved several steps. First, the questionnaire was translated into Chinese. Second, a group of four general practitioners and two experts of primary care policy reviewed the translated PCAT and some of the items that did not fit in the Chinese context were rephrased. During this stage, any item that did not adapt to the Chinese context, shared similar conceptual basis, or lacked conciseness was rewritten, combined, or deleted. Modifications were only made on condition that a consensus was reached; for example, “home safety, such as getting and checking smoke detectors and storing medicines safely” was eliminated, because it was considered to be inappropriate or unpractical in the current Chinese context; “When your PCP is closed and you get sick during the night, would someone from there see you that night” was deleted since most of the CHCs in China do not offer medical services at night. Then pilot tests, which focused on item wording, were conducted in three facilities. Twenty patients were randomly selected to complete the questionnaire by themselves in each of the pilot test and items were further revised into more understandable words to ensure that elderly or subjects of lower socioeconomic status can fully comprehend (specific item changes can be found in Table 9).

In this stage, the Chinese Rapid Primary Care Assessment Tool (CR-PCAT) consisted of 42 items. 23 items assessed the four core dimensions of primary care—first contact, continuity, comprehensiveness, and coordination—and three derivative dimensions—family centeredness, community orientation, and cultural competence. Two items were developed to identify individuals’ usual source of care and another 2 measured the frequency of visits to the general practitioners while the rest of the items mainly reflected patients’ social demographic information. The services received by patients were represented by a 4-point Likert scale (1 = never; 2 = sometimes; 3 = often; 4 = always). An additional option of Don’t know/Not sure was added in case of lack of knowledge of a certain item. The Don’t know/Not sure response and missing data were assigned a neutral value of 2.5 when conducting the analyses to be consistent with the methods used in other countries [34, 35].

2.2. Data Collection

A stratified, three-stage sampling approach was used to decide study sample. Three cities, namely, Guangzhou, Shenzhen, and Dongguan, were chosen, since they displayed significant socioeconomic variations and also CHCs were under different management pattern in these cities. In Guangzhou, 90% of the CHCs were owned and operated by government, and the remaining 10% were private-owned. The study chose one government-owned CHC and one private-owned CHC that were both granted as a state-level demonstration site, and another ordinary government-owned CHC. In Dongguan, all CHCs were government-owned and government-run; one state-level demonstration site along with one ordinary CHC was selected. In Shenzhen, all CHCs were affiliated with local hospitals. We selected two state-level demonstration sites and one ordinary CHC. Convenience sampling was then used to select patients in each of the CHC to participate in the survey.

The standard sample size formula for cross-sectional survey was used to calculate the sample size, where = standard normal variant corresponding to 5% type I error of 1.96; = expected proportion in population based on previous study; = absolute error or precision [38]. An estimated of the proportion was drawn from previously published research [7]. Typically, a conservative estimate of the proportion (i.e. 50/50) was made, if data was not sufficient. In addition, the sample size was adjusted for refusal rate of approximately 10%. Finally, a target sample size of 400 was set within each area. The CHCs in Guangzhou were oversampled because of additional planned analysis (not the focus of this paper). Furthermore, expert consultation was also used to ensure the precision and feasibility of the sample size, which confirmed the calculation above.

Data collection began in June 2014 and lasted for three months. Interviewers who were postgraduate students from Sun Yat-sen University were trained by two researchers in advance in order to assist the patients to complete the questionnaires. One-to-one interviews were conducted to guarantee the quality of survey data. Adult patients who were 18 years or older and could speak Mandarin or Cantonese were selected at the waiting area of each site. Subjects selected were exclusively to patients who were visiting general practitioners. Those who had visited the same general practitioners at least three times were eligible since they were considered to have a better understanding of primary care services provided by GPs. These patients were asked for permission to participate in the interview with a full explanation of the research purpose and were told that the survey would not influence their GP visits. Patients who took part in the survey were given a small gift as a token of appreciation upon completion of the questionnaire.

2.3. Data Analysis

Data analysis consisted of description of the sample population, validation of the CR-PCAT, and descriptive summary of the primary care achievement of each city. First, socio-demographic characteristics and health data of the sample population were summarized to demonstrate a diverse sample of populations and their utilization of primary care, meanwhile, the characteristics of each city and CHC were also presented.

Second, validation of the CR-PCAT was conducted, which was applied exclusively to the 23 items evaluating the seven dimensions of primary care [39]. For construct validity, factor analysis (principal component analysis and varimax rotation) was undertaken to test the dimensionality of the hypothesized scales. The item selection criteria were: eigenvalues >1.0, factor loading >0.35, and all retained items should not have a secondary loading over 0.35. Then internal consistency reliability of each scale was assessed by Cronbach’s coefficient alpha () and item-total correlation. Cronbach’s coefficient alpha over 0.50 was recommended as the benchmarks of adequacy of reliability coefficients [40–42], while the minimum acceptable item-total correlation was 0.30 [41].

Next, five Likert scaling assumptions were tested, which included (1) item-convergent validity (a test by item-scale correlations); (2) item-discriminant validity (a test by the scaling success rate, i.e. items within a scale correlate more substantially with their hypothesized scale than with any other scale); (3) equal item variance (a test by examining item means, standard deviations, the equivalence of the intra-class correlation and Scotts homogeneity ratio for each scale); (4) equal item-scale correlation (a test by examining the range of item-scale correlations); and (5) score reliability (a test by Cronbach’s coefficient alpha) [43, 44]. The analysis was only applied to the retained items.

Then, we examined the following score distribution characteristics of the revised scales: mean, standard deviation, quartiles, skewness, range, and Kurtosis. Each of the five scales was also assessed for inter-factor correlations by computing a matrix of inter-factor correlations [45]. Finally, primary care achievement of different cities was compared using analysis of variance.

All analyses were performed using the statistical package STATA 13.

3. Results

3.1. Subjects

A total of 1831 eligible individuals were asked to participate in the survey, and 81.9% () of them completed the questionnaire. After eliminating samples with more than 5 missing responses (), 1465 effective samples were included for analysis.

Table 1 shows the socio-demographic features and health care utilization of the survey participants. The analytic sample included adults aged 18 to 89 years (mean = 46.1 years). They were predominantly female (60.7%), and were resident or temporary resident population (98.6%) with generally lower level of household income (only 10.9% over 10000 yuan/month). Over half of respondents were employed (54.6%) with relatively lower education (69.1% did not finish high school). Approximately one third (41.8%) indicated having chronic disease and 23.3% rated their health as fair or bad. Most participants (78.7%) were covered by health insurance and had been visiting their PCP for more than 1 year (82.3%). 73.3% reported over 3 times of visits to their PCP and over half of the respondents had visited a specialist in the past year (53.4%). Less than one tenth of all users had signed contracts with their PCP in spite of government’s vigorous promotion (8.9%) [46].

Table 1: Sociodemographic characteristic of the study subjects.

Total (%)

1465

Gender

    Male
574 (39.2)

    Female
889 (60.7)

Age (years)

    <40
616 (42.0)

    40–65
600 (41.0)

    >65
249 (17.0)

Household registration

    Resident population
715 (48.8)

    Temporary resident population
729 (49.8)

    Floating population
17 (1.2)

Household income (yuan/month)

    <5000
459 (31.4)

    5000–10000
496 (33.9)

    >10000
291 (19.9)

Education

    High school or lower
1006 (69.1)

    Vocational/associate
291 (20.0)

    College or higher
159 (10.9)

Employment

    Employed (including part-time)
798 (54.6)

    Unemployed
224 (15.3)

    Retired
423 (29.0)

    In school
16 (1.1)

Medical insurance

    Urban-employees
629 (42.9)

    Urban-citizens
524 (35.8)

    Out-of-pocket expenditure
312 (21.3)

Period of time since the first visit

    Less than 1 year
254 (17.3)

    1-2 years
229 (15.6)

    3–5 years
305 (20.8)

    More than 5 years
673 (45.9)

Number of PCP visits in the past year

    <3
391 (26.7)

    3–5
403 (27.5)

    6–15
440 (30.1)

    >15
229 (15.7)

Specialist visits in the past year

    No
683 (46.6)

    Yes
782 (53.4)

Contracted PCP

    No
1334 (91.1)

    Yes
131 (8.9)

Chronic disease

    No
853 (58.2)

    Yes
612 (41.8)

Self-perceived health status

    Excellent
101 (6.9)

    Very good
568 (38.8)

    Good
455 (31.1)

    Fair
304 (20.8)

    Poor
37 (2.5)

Table 2 displays the characteristics of the included cities and CHCs. Of the eight CHCs being studied, one CHC from Guangzhou was private-owned with the other 7 owned by the local government. Gatekeeping system was not implemented in Guangzhou, while in Dongguan and Shenzhen it has been implemented in practice. It is also worth noting that only the patients who were enrolled in a specific insurance plan were under the impact of the gatekeeping system (not the focus of this paper). Of the three CHCs in Shenzhen, they were all affiliated with the local public hospitals while CHCs in the other two cities were operated independently.

Table 2: The characteristics of each city and CHC.

Cities
CHCs
Owned status
Gatekeeping
Operational mode
Sample size

Guangzhou
Hong shan
Private-owned
No
Independent
292

Sha yuan
Government-owned
No
Independent
208

Huang hua gang
Government-owned
No
Independent
198

Shenzhen
Niu hu
Government-owned
Yes
Affiliated to public hospital
183

Yi kang yuan
Government-owned
Yes
Affiliated to public hospital
95

Liu tang
Government-owned
Yes
Affiliated to public hospital
90

Dongguan
Liao bu
Government-owned
Yes
Independent
199

Da lang
Government-owned
Yes
Independent
200

3.2. Construct Validity and Reliability

Twenty-three items were included in the initial principal component analysis. Based on the criteria that eigenvalue was larger than 1.0, five components that corresponded to the four core dimensions of primary care remained: first contact utilization, first contact accessibility, ongoing care, comprehensiveness, and coordination (Table 3). These extracted factors explained 58.91% of the total variance, with eigenvalues ranging from 2.08 to 1.39.

Table 3: Factor analysis result with five factors retained.

Final rotated factor loadings

Items
Factor 1
Factor 2
Factor 3
Factor 4
Factor 5

(Q9) When you have a new health problem, do you go to your PCP before going somewhere else?

0.7166

(Q10) When you need a regular general checkup, do you go to your PCP before going somewhere else?

0.7052

(Q11) When you have to see a specialist, does your PCP have to approve or give you a referral?

0.4621

(Q13) When your PCP is open, can you get advice quickly over the phone if you need it?

0.8661

(Q14) When your PCP is closed, can you get advice quickly over the phone if you need it?

0.8801

(Q15) When you go to your PCP’s, are you taken care of by the same doctor or nurse each time?
0.8111

(Q16) If you have a question, can you call and talk to the doctor or nurse who knows you best?
0.7755

(Q17) Does your PCP know what problems are most important to you?
0.7318

(Q18) When you have a mental or emotional problems, do you talk to your PCP?

0.5721

(Q19) Does your PCP talk about healthy diet, adequate sleep, appropriate exercise and so forth with you?

0.6509

(Q20) Does your PCP recommend you to have a regular checkup?

0.6917

(Q21) Generally, how much time will your PCP spend talking to you?

0.5588

(Q24) Did your PCP discuss with you different places you could have gone to get help with that problem?

0.7305

(Q26) Did your PCP or someone working with your PCP help you make the appointment for that visit?

0.7278

(Q27) After you went to the specialist or special service, did your PCP talk with you about what happened at the visit?

0.5421

Eigenvalue
2.0856
1.8367
1.8277
1.6934
1.3925

Variance (%)
13.90
12.24
12.18
11.29
9.28

Accumulative variance (%)
13.90
26.15
38.33
49.62
58.91

Eight of the 23 items were eliminated based on the criteria imposed for factor analyses (Table 4). No items were deleted for first contact utilization, ongoing care, and comprehensiveness. One item was eliminated for first contact accessibility and coordination dimension, respectively. Although first contact accessibility dimension contained only two items after elimination, it was retained since it represented unique conceptual meaning of primary care. All items were deleted for family centeredness, community orientation, and cultural competence.

Table 4: Item statistics and internal consistency reliability of original primary care scales and revised scales.

Primary care scale items
Reliability-alpha
Sample size for revised scale
Item mean
Item SD
Orig item-
total corr
Rev item-
total corr

First contact utilization
Orig 3-item  
Rev 3-item ;  

(Q9) When you have a new health problem, do you go to your PCP before going somewhere else?
3.63
0.66
0.56
0.56

(Q10) When you need a regular general checkup, do you go to your PCP before going somewhere else?
2.84
1.24
0.75
0.75

(Q11) When you have to see a specialist, does your PCP have to approve or give you a referral?
2.02
1.23
0.70
0.70

First contact accessibility
Orig 3-item   
Rev 2-item ;  

(Q12) Once you get there, do you have to wait more than 15 minutes before you are checked by the doctor or nurse?
2.74
0.88
0.77
Deleted

(Q13) When your PCP is open, can you get advice quickly over the phone if you need it?
1.14
0.42
0.57
0.91

(Q14) When your PCP is close, can you get advice quickly over the phone if you need it?
1.09
0.36
0.56
0.88

Ongoing care
Orig 3-item  
Rev 3-item ;  

(Q15) When you go to your PCP’s, are you taken care of by the same doctor or nurse each time?
2.19
1.13
0.83
0.83

(Q16) If you have a question, can you call and talk to the doctor or nurse who knows you best?
3.00
1.10
0.80
0.80

(Q17) Does your PCP know what problems are most important to you?
2.40
0.93
0.78
0.78

Comprehensiveness
Orig 4-item  
Rev 4-item ;  

(Q18) When you have a mental or emotional problems, do you talk to your PCP?
1.31
0.67
0.63
0.63

(Q19) Does your PCP talk about healthy diet, adequate sleep, appropriate exercise and so forth with you?
2.06
0.83
0.72
0.72

(Q20) Does your PCP recommend you to have a regular checkup?
1.78
0.89
0.72
0.72

(Q21) Generally, how much time will your PCP spend talking to you?
2.00
0.72
0.59
0.59

Coordination
Orig 4-item  
Rev 3-item ;  

(Q24) Did your PCP discuss with you different places you could have gone to get help with that problem?
1.61
0.90
0.68
0.74

(Q25) Did your doctor write down any information for the specialist about the reason for the visit?
2.76
1.27
0.70
Deleted

(Q26) Did your PCP or someone working with your PCP help you make the appointment for that visit?
1.37
0.76
0.65
0.77

(Q27) After you went to the specialist or special service, did your PCP talk with you about what happened at the visit?
1.76
0.89
0.60
0.70

Derivative dimensions
Orig 6-item  
All rev items were deleted

(Q28) Does your doctor ask your ideas and opinions when they are planning treatment/care for you or a family member?
1.93
0.98
0.37
Deleted

(Q29) Has your doctor asked about illness or problems that might run in your family?
1.54
0.77
0.46
Deleted

(Q30) Would anyone at doctor s office ever make home visits?
1.11
0.38
0.40
Deleted

(Q31) Does your doctor know about health problems of your neighborhood?
1.37
0.63
0.49
Deleted

(Q32) How does (Doctor/Place P) get opinions/ideas from people that will help them provide better health care?
1.26
0.58
0.37
Deleted

(Q33) Would you recommend your doctor to a friend or relative?
1.97
0.99
0.31
Deleted

Table 4 presents the item descriptive results, as well as results of reliability tests for both the original items and the final items. The distribution of the items varied significantly from a mean of 1.09 (When your PCP is closed, can you get advice quickly over the phone if you need it?) to 3.63 (When you have a new health problem, do you go to your PCP before going somewhere else?) on the 4-point Likert scale. The distribution tends to skew toward less favorable answers, with only five items falling above 2.5. The first contact utilization and ongoing care scales achieved the highest mean scores, whereas scales with lower means were first contact accessibility, comprehensiveness, and coordination.

Reliability tests include item-total correlation and Cronbach’s alpha coefficient reliability. Item-total correlations show that each item was strongly associated with its corresponding scale, with value ranging from 0.56 (When you have a new health problem, do you go to your PCP before going somewhere else? and When your PCP is closed, can you get advice quickly over the phone if you need it?) to 0.83 (When you go to your PCP’s, are you taken care of by the same doctor or nurse each time?). Cronbach’s alpha demonstrates internal consistency reliability that was higher than or equal to the original scales, despite dropping items from original scales.

3.3. Testing the Likert Scaling Assumptions

Table 5 demonstrates a summary of the results of the tests of Likert scaling assumptions using the five multi-item revised scales. All item-scale correlations exceeded 0.5 with the majority greater than 0.6 and all scales demonstrated a relatively narrow range of item-scale correlations (from 0.04 for first contact accessibility to 0.19 for first contact utilization). All 5 multi-item scales achieved 100% scaling success, indicating that all items in these scales got a greater correlation with items in their hypothesized scale than with items in other scales. Formal evidence of equal item variance was supported by the proximity of the intraclass correlation and Scotts homogeneity ratio for each scale. Two multi-item scales achieved Cronbach’s alpha level over 0.70, and two scales were below the threshold, but exceeded 0.50. One scale, first contact utilization, was far below the standard, verifying the low homogeneity of variances among items within the scale (0.2). However, the scale was retained because of their conceptual significance.

Table 5: Likert scaling assumptions using revised items.

Scale
Range of item-scale correlations 
(assumptions 1, 4)
Item scaling tests (assumption 2)
Success/total
scaling success rate
Measures of equal item variance
(assumption 3)
Cronbach’s alpha (assumption 5)

Scott’s homogeneity
Intraclass correlation

First contact utilization
0.56–0.75
15/15
0.20
0.17
0.38

First contact accessibility
0.87–0.91
10/10
0.60
0.58
0.74

Ongoing care
0.78–0.83
15/15
0.47
0.47
0.72

Comprehensiveness
0.67–0.77
20/20
0.26
0.30
0.58

Coordination of services
0.70–0.77
15/15
0.32
0.26
0.57

3.4. Descriptive Feature of CR-PCAT

Table 6 presents estimates of central tendency, dispersion, and other features of scale score distributions for the five primary care scales. The full range of possible scores was observed for all scales. Except for first contact utilization and ongoing care, the other three scales were positively skewed, indicating distributions with more negative ratings of primary care. The percentage of respondents scoring at the floor (the lowest score) or ceiling (the highest score) was acceptably low for all scales. The most significant floor effects existed in the first contact accessibility scale where over 80% of the participants reported minimum score.

Table 6: Estimates of central tendency and dispersion of PCAT scales.

First contact utilization
First contact accessibility
Ongoing care
Comprehensiveness
Coordination

Number of items
3
2
3
4
3

Mean
8.49
2.23
7.59
7.15
4.73

25th percentile
7.00
2.00
6.00
6.00
3.00

50th percentile
9.00
2.00
7.50
7.00
4.00

75th percentile
10.00
2.00
10.00
8.00
6.00

Observed range
3.0–12.0
2.0–8.0
3.0–12.0
4.0–16.0
3.0–12.0

SD
2.16
0.70
2.54
2.09
1.87

Skewness
−0.03
3.86
−0.15
0.88
1.17

Kurtosis
2.31
20.79
2.17
3.95
4.17

Table 7 compares the internal reliability coefficient and interfactor correlation for each primary care scale. Cronbach’s alpha of each scale was substantially greater than its correlation with all other primary care scales. None of the interfactor correlations were excessively high, indicating the uniqueness of each primary care scale. All significant correlations were positive, implying that each scale was complementary to some extent. Relatively high and positive interfactor correlations were observed between first contact utilization and coordination (0.32), with the latter and comprehensiveness (0.27), and with comprehensiveness and ongoing care (0.29).

Table 7: Comparison of internal consistency reliability and interfactor correlation.

Factor
Cronbach’s alpha
Interfactor correlations

First contact utilization
First contact accessibility
Ongoing care
Comprehensiveness
Coordination

Without coordination items ()

    First contact utilization
0.38
1.00

    First contact accessibility
0.74
0.00
1.00

    Ongoing care
0.72
0.03
0.13
1.00

    Comprehensiveness
0.58
0.01
0.27
0.32
1.00

With coordination items ()

    First contact utilization
0.38
1.00

    First contact accessibility
0.74
0.03
1.00

    Ongoing care
0.72
0.07
0.11
1.00

    Comprehensiveness
0.58
0.12
0.21
0.29
1.00

    Coordination
0.57
0.23
0.32
0.11
0.27
1.00

3.5. Primary Care Achievements in Three Cities

Table 8 illustrates that the overall scores of primary care achievements were relatively low in general, with Shenzhen achieving the highest total score (2.19), followed by Dongguan (2.10), and Guangzhou achieving the lowest score (2.09). For each scale, Guangzhou got the highest scores on two scales (ongoing care and comprehensiveness) and Shenzhen on first contact accessibility and coordination while Dongguan achieved the highest score on one scale (first contact utilization). Besides, most of the scores of each scale differed significantly among the three cities.

Table 8: Primary care quality scores in three cities.

Scale
Guangzhou (SE)
Dongguan (SE)
Shenzhen (SE)

First contact utilization
2.63 (0.02)
3.10 (0.04)
2.92 (0.04)

First contact accessibility
1.67 (0.01)
1.54 (0.02)
1.76 (0.02)

Ongoing care
2.70 (0.03)
2.45 (0.05)
2.29 (0.04)

Comprehensiveness
1.87 (0.02)
1.60 (0.02)
1.83 (0.03)

Coordination of services
1.57 (0.02)
1.80 (0.02)
2.17 (0.03)

Total
2.09 (0.01)
2.10 (0.02)
2.19 (0.02)

(a) Significance is indicated at , based on one-way ANOVA (Bonferroni).

(b) Indicating significant differences between Guangzhou and Dongguan.

(c) Indicating significant differences between Guangzhou and Shenzhen.

(d) Indicating significant differences between Dongguan and Shenzhen.

Table 9: Specific item changes compared to the original tool.

Scales
Items contained in original tool
Items retained in preliminary tool
Comments

First contact utilization
(B1) When you need a regular general checkup, do you go to your PCP before going somewhere else?
(Q9) When you have a new health problem, do you go to your PCP before going somewhere else?
All original items in “first contact utilization” scale were retained; slight change was made on item order

(B2) When you have a new health problem, do you go to your PCP before going somewhere else?
(Q10) When you need a regular general checkup, do you go to your PCP before going somewhere else?

(B3) When you have to see a specialist, does your PCP have to approve or give you a referral?
(Q11) When you have to see a specialist, does your PCP have to approve or give you a referral?

First contact accessibility
(C3) When your PCP is open and you get sick, would someone from there see you the same day?
(Q12) Once you get there, do you have to wait more than 15 minutes before you are checked by the doctor or nurse?
(C3) was replaced by (Q12) because almost all of the patients can be seen the same day when they turn to a CHC in China;
(C7) was deleted because most of the CHCs in China do not offer medical services at night

(C4) When your PCP is open, can you get advice quickly over the phone if you need it?
(Q13) When your PCP is open, can you get advice quickly over the phone if you need it?

(C5) When your PCP is closed, is there a phone number you can call when you get sick?
(Q14) When your PCP is closed, can you get advice quickly over the phone if you need it?

(C7) When your PCP is closed and you get sick during the night, would someone from there see you that night?

Ongoing care
(D1) When you go to your PCP’s, are you taken care of by the same doctor or nurse each time?
(Q15) When you go to your PCP’s, are you taken care of by the same doctor or nurse each time?
(D7) was deleted due to its unfitness for the local context

(D4) If you have a question, can you call and talk to the doctor or nurse who knows you best?
(Q16) If you have a question, can you call and talk to the doctor or nurse who knows you best?

(D7) Does your PCP know you very well as a person, rather than as someone with a medical problem?
(Q17) Does your PCP know what problems are most important to you?

(D8) Does your PCP know what problems are most important to you?

Comprehensiveness
(service available)
(G2) Immunizations (shots)
(Q18) When you have a mental or emotional problems, do you talk to your PCP?
(G2), (G6), and (G10) were deleted because they are the regular healthcare services provided by CHCs

(G6) Family planning or birth control methods

(G8) Counseling for mental health problems

(G10) Sewing up a cut that needs stitches

Comprehensiveness
(service provided)
(H1) Advice about healthy foods and unhealthy foods or getting enough sleep
(Q19) Does your PCP talk about healthy diet, adequate sleep, appropriate exercise and so forth with you?
(H1), (H5) were combined; (H2), (H4), and (H7) were deleted due to their unfitness for the local context

(H2) Home safety, like getting and checking smoke detectors and storing medicines safely
(Q20) Does your PCP recommend you to have a regular checkup?

(H4) Ways to handle family conflicts that may arise from time to time
(Q21) Generally, how much time will your PCP spend talking to you?

(H5) Advice about appropriate exercise for you

(H7) Checking on and discussing the medications you are taking

Coordination
(E8) Did your PCP discuss with you different places you could have gone to get help with that problem?
(Q24) Did your PCP discuss with you different places you could have gone to get help with that problem?
All original items in “coordination” scale were retained; slight change was made on item order

(E9) Did your PCP or someone working with your PCP help you make the appointment for that visit?
(Q25) Did your doctor write down any information for the specialist about the reason for the visit?

(E10) Did your PCP write down any information for the specialist about the reason for the visit?
(Q26) Did your PCP or someone working with your PCP help you make the appointment for that visit?

(E12) After you went to the specialist or special service, did your PCP talk with you about what happened at the visit?
(Q27) After you went to the specialist or special service, did your PCP talk with you about what happened at the visit?

Coordination (information systems)
(F1) When you go to your PCP, do you bring any of your own medical records, such as shot records or reports of medical care you had in the past?

(F1), (F2), and (F3) were all deleted due to their unfitness for the local context

(F2) Could you look at your medical record if you wanted to?

(F3) When you go to your PCP, is your medical record always available?

Family centredness
(I1) Does your PCP ask you about your ideas and opinions when planning treatment and care for you or a family member?
(Q28) Does your doctor ask your ideas and opinions when they are planning treatment/care for you or a family member?
(I3) was deleted due to its unfitness for the local context

(I2) Has your PCP asked about illnesses or problems that might run in your family?
(Q29) Has your doctor asked about illness or problems that might run in your family?

(I3) Would your PCP meet with members of your family if you thought it would be helpful?

Community orientation
(J1) Does anyone at your PCP’s office ever make home visits?
(Q30) Would anyone at doctor s office ever make home visits?
All original items in “community orientation” scale were retained

(J2) Does your PCP know about the important health problems of your neighborhood?
(Q31) Does your doctor know about health problems of your neighborhood?

(J3) Does your PCP get opinions and ideas from people that will help to provide better health care?
(Q32) Does your PCP get opinions/ideas from people that will help to provide better health care?

Cultural competence
(K1) Would you recommend your PCP to a friend or relative?
(Q33) Would you recommend your doctor to a friend or relative?
(K2) and (K3) were deleted due to their unfitness for the local context

(K2) Would you recommend your PCP to someone who does not speak English well?

(K3) Would you recommend your PCP to someone who uses folk medicine, such as herbs or homemade medicines, or has special beliefs about health care?

4. Discussion

The CR-PCAT introduced in this paper is an explicit and brief adaptation of the original PCAT-AE to the Chinese context. The final version of CR-PCAT comprises 15 items with five scales measuring four core dimensions of primary care. The core dimensions and items in corresponding dimensions display high consistency with the original PCAT-AE.

A standard psychometric evaluation method was used to validate the CR-PCAT. The analytic results indicate that the hypothesized scales have achieved relatively good reliability and validity. Principal component factor analysis supports the scale’s construct validity and the extracted five factors explained 58.91% of the total variance in the item scores. The panel of experts in primary care helped facilitate the best possible content validity [39]. Reliability tests indicated adequate item-total correlation and internal consistency of the tool. All of the five Likert scaling assumptions, including item-convergent validity, item-discriminant validity, equal item variance, equal item-scale correlation, and score reliability, were basically met, which suggests the appropriateness of the usage of Linker’s method.

The three derivative dimensions—family centeredness, community orientation, and cultural competence—were eliminated, because retaining all these three derivative dimensions did not increase the explained variations. Besides, the items within the three dimensions did not converge together as the other studies had displayed [26]. To balance between the amounts of information captured and time costs, only the four core dimensions with five scales were retained, which also met our goal of developing a compact measure tool. In addition, we also ran factor analysis using data from each of the three cities separately. The results confirmed the validity of the factor analysis done with combined dataset (results are not shown due to large amount of information but are available upon request).

Item descriptive results indicated that most of the items scores skew toward less favorable answers, which contrasted with the results of other studies [7, 34]. High rate of favorable response and “Don’t know/Not sure” answers was common among Asian populations, since they were not used to commenting or critiquing. However, the study adopted one-to-one interview to ensure that all participants fully understood each item and reported their actual perception of primary care.

4.1. Application of the Tool

CR-PCAT has multiple uses given its small number of items. Compared to the PCAT-AE and the other version of PCAT, which needs approximately 20–40 minutes to complete [26], the CR-PCAT questionnaire took about 5 to 7 minutes to administer. The tool greatly reduces the burden of patients and interviewers while insuring the key information was captured. CR-PCAT can be applied in periodic evaluation activities to assess the latest performance of primary care provided by CHCs. Also, since the tool is readily linked to specific implementations of primary care, it can be used to assess the effectiveness of policy changes on the delivery of care from both CHCs and district levels. On condition that primary care is provided by GPs in China, CR-PCAT can also serve as a part of the performance evaluation tools of GPs to provide evidence for further improvement.

In addition, from the primary care achievements of three cities, low scores of the overall primary care quality and each dimension suggest that primary care in China is far from perfect. The low scores of each dimension were consistent with our previous studies, which focused specifically on continuity or coordination of care [47–49]. Besides, significant differences of scale scores between three cities were observed, implying that there might be system or organizational level factors that cause the differences [50]. For example, the score of first contact utilization in Dongguan was the highest among the three cities, which might imply the effects of gatekeeping system; and Shenzhen achieved the highest scores on the scale of coordination, indicating that patients experience better referral services when specialist care is needed, which may be due to the reason that CHCs in Shenzhen were affiliated to hospitals while the CHCs in the other two cities were independent of hospitals.

4.2. Limitations

Potential limitations of the study are discussed as follows. First, the CR-PCAT is a very short version of the original measure tool and the development and adjustment of the tool were straightforward. Only quantitative analyses were conducted in the study, which might imply that qualitative methods such as focus groups or in-depth interviews with patients might be necessary to fully explore their perception of primary care. Also, CR-PCAT does not measure disease specific quality of care, but rather general experience of primary care. Further research could combine CR-PCAT with disease specific quality indicators to have more extensive representation.

Secondly, the CR-PCAT was developed based on CHCs samples obtained in Guangdong province, where local people mainly speak Cantonese, suggesting that the linguistic habits were unique. If the tool were to be used in other provinces in China where there were distinct differences between the dialects, slight revisions of the wording were deemed necessary.

Thirdly, the measurement of primary care achievement was based on patients’ self-report. While this may be the best way to ascertain individual experience, it is subject to recall and response bias. Nevertheless, one-to-one interviews were undertaken in this study in an attempt to minimize the bias. In addition, criterion-related validity (or more specifically concurrent validity) and stability were not tested due to time and economic reasons; however, the tool will be applied in more regions and populations in the future, the results of which could add to the validation evidence of CR-PCAT.

Finally, as the CR-PCAT only measured adult patient’s perception of primary care, which was adapted from one part of the original methodology, our future research would center on developing rapid assessment tools for providers and children.

5. Conclusion

CR-PCAT appears to be a valid and effective tool to capture reliable performance of four core dimensions of primary care in China. Because of its simplicity and easy administration, it is a feasible and practical tool that can be used in CHCs’ daily administration, performance evaluation and monitoring, and policy assessments. The poor performance of primary care highlights the urgency of recognition and understanding of its core dimensions and the development of corresponding policies in the future to strengthen them in China. The next phase of the study will focus on identifying characteristics at the organization and health care system levels that account for the observed differences in primary care performance.

Abbreviations

PCAT: | Primary Care Assessment Tool
PCAT-AE: | Primary Care Assessment Tool-Adult Edition
CR-PCAT: | Chinese Rapid Primary Care Assessment tool
IOM: | Institute of Medicine
GP: | General practitioner
CHC: | Community health center.

Ethical Approval

The study protocol was reviewed and approved by the Ethics Review Committees of Sun Yat-sen University School of Public Health.

Consent

All participants gave verbal consent.

Conflict of Interests

None of the authors have any competing interests.

Authors’ Contribution

Jie Mei carried out the study, was involved in the acquisition and interpretation of data, and drafted the paper. Yuan Liang carried out the study and was involved in collection and interpretation of data. LeiYu Shi critically revised the paper and supported analysis and interpretation of data. JingGe Zhao and YuTan Wang were involved in collection and interpretation of data. Li Kuang conceived and designed the study and was involved in the collection and interpretation of data. All authors read and approved the final paper.

Acknowledgments

This study was supported by a grant from the Health Department of Guangdong province of China. The authors thank all the participating public health experts and health care staffs for their expertise and support during the translation and adaption and data collection process.

References

B. Starfield, “Is primary care essential?” The Lancet, vol. 344, no. 8930, pp. 1129–1133, 1994.

J. Macinko, B. Starfield, and L. Shi, “The contribution of primary care systems to health outcomes within Organization for Economic Cooperation and Development (OECD) countries, 1970–1998,” Health Services Research, vol. 38, no. 3, pp. 831–865, 2003.

W. Europe, The European Definition of General Practice/Family Medicine, WONCA Europe, Barcelona, Spain, 2002.

World Health Organization, Alma Ata Declaration, World Health Organization, Geneva, Switzerland, 1978.

S. Donaldson Molla, D. Yordy Karl, and V. Neal Arthur, Defining Primary Care: An Interim Report, National Academy Press, 1994.

O. Bhattacharyya, Y. Delu, S. T. Wong, and C. Bowen, “Evolution of primary care in China 1997–2009,” Health Policy, vol. 100, no. 2-3, pp. 174–180, 2011.

H. Yang, L. Shi, L. A. Lebrun, X. Zhou, J. Liu, and H. Wang, “Development of the chinese primary care assessment tool: data quality and measurement properties,” International Journal for Quality in Health Care, vol. 25, no. 1, pp. 92–105, 2013.

H. Wang, M. K. Gusmano, and Q. Cao, “An evaluation of the policy on community health organizations in China: will the priority of new healthcare reform in China be a success?” Health Policy, vol. 99, no. 1, pp. 37–43, 2011.

Q. Liu, B. Wang, Y. Kong, and K. Cheng, “China's primary health-care reform,” The Lancet, vol. 377, no. 9783, pp. 2064–2066, 2011.

D. Browne, “The long march to primary health care in China: from collectivism to market economics,” Public Health, vol. 115, no. 1, pp. 2–3, 2001.

The State Council of People's Republic of China, Guidelines for Furthering the Reform of Health-Care System, Ministry of Health of China, Beijing, China, 2009 (Chinese), http://www.gov.cn/jrzg/2009-04/06/content_1278721.htm.

D. Blumenthal and W. Hsiao, “Lessons from the East—China's rapidly evolving health care system,” The New England Journal of Medicine, vol. 372, no. 14, pp. 1281–1285, 2015.

T. M. Cheng, “Early results of China's historic health reforms: the view from minister Chen Zhu. Interview by Tsung-Mei Cheng,” Health Affairs, vol. 31, no. 11, pp. 2536–2544, 2012.

X.-X. Zou, Y. Yao, W. Li, M.-G. Zhao, and Z. Liu, “Cognitive appraisal on hierarchical medical service system in China on the side of the referral patients,” Chinese Hospital Management, vol. 35, no. 7, pp. 18–21, 2015 (Chinese).

Z. Zhiguang, P. Ji, C. Jinquan et al., “Development of community health service in Bao'an District of Shenzhen,” Medicine and Society, vol. 18, no. 4, pp. 4–7, 2005.

L.-M. Hung, S. Rane, J. Tsai, and L. Shi, “Advancing primary care to promote equitable health: Implications for China,” International Journal for Equity in Health, vol. 11, no. 1, article 2, 2012.

H. Li, R. Y.-N. Chung, X. Wei et al., “Comparison of perceived quality amongst migrant and local patients using primary health care delivered by community health centres in Shenzhen, China,” BMC Family Practice, vol. 15, no. 1, article 76, 2014.

S. Thomas, Y. Hui, C. Browning et al., “Quality of care in different community health facilities in China: from patients points of view,” Chinese General Practice, vol. 10, no. 21, pp. 1760–1764, 2007 (Chinese).

P. Yingchun, W. Yuan, C. Wenhu, L. Jun, and L. Wannian, “Study of evaluating index systemon the quality of general practice,” Chinese General Practice, vol. 7, no. 3, pp. 158–160, 2004 (Chinese).

W. Yun, X. Juyang, C. Yong, and Z. Liang, “Study on the index of performance evaluation of the community health service system at district level,” Chinese Primary Health Care, vol. 22, no. 9, pp. 13–16, 2008 (Chinese).

D. S. Kringos, W. G. Boerma, A. Hutchinson, J. van der Zee, and P. P. Groenewegen, “The breadth of primary care: a systematic literature review of its core dimensions,” BMC Health Services Research, vol. 10, no. 1, article 65, 2010.

B. Starfield, L. Shi, and J. Macinko, “Contribution of primary care to health systems and health,” Milbank Quarterly, vol. 83, no. 3, pp. 457–502, 2005.

S. M. Campbell, M. O. Roland, and S. A. Buetow, “Defining quality of care,” Social Science & Medicine, vol. 51, no. 11, pp. 1611–1625, 2000.

D. G. Safran, M. Kosinski, A. R. Tarlov et al., “The Primary Care Assessment Survey: tests of data quality and measurement performance,” Medical Care, vol. 36, no. 5, pp. 728–739, 1998.

S. A. Flocke, “Measuring attributes of primary care: development of a new instrument,” The Journal of Family Practice, vol. 45, no. 1, pp. 64–74, 1997.

L. Shi, B. Starfield, and J. Xu, “Validating the adult primary care assessment tool,” The Journal of Family Practice, vol. 50, no. 2, pp. 161–164, 2001.

J. Ramsay, J. L. Campbell, S. Schroter, J. Green, and M. Roland, “The general practice assessment survey (GPAS): tests of data quality and measurement properties,” Family Practice, vol. 17, no. 5, pp. 372–379, 2000.

W. Wang, L. Shi, A. Yin, Y. Lai, E. Maitland, and S. Nicholas, “Development and validation of the tibetan primary care assessment tool,” BioMed Research International, vol. 2014, Article ID 308739, 7 pages, 2014.

B. Starfield, Primary Care: Balancing Health Needs, Services, and Technology, Oxford University Press, 1998.

A. Donabedian, “Evaluating the quality of medical care,” The Milbank Memorial Fund Quarterly, vol. 44, no. 3, pp. 166–206, 1966.

R. A. Malouin, B. Starfield, and M. J. Sepulveda, “Evaluating the tools used to assess the medical home,” Managed Care, vol. 18, no. 6, pp. 44–48, 2009.

J. L. Haggerty, F. Burge, M.-D. Beaulieu et al., “Validation of instruments to evaluate primary healthcare from the patient perspective: overview of the method,” Healthcare Policy, vol. 7, pp. 31–46, 2011.

J. Macinko, C. Almeida, and P. K. de Sá, “A rapid assessment methodology for the evaluation of primary care organization and performance in Brazil,” Health Policy and Planning, vol. 22, no. 3, pp. 167–177, 2007.

J. H. Lee, Y.-J. Choi, N. J. Sung et al., “Development of the Korean primary care assessment tool—measuring user experience: tests of data quality and measurement performance,” International Journal for Quality in Health Care, vol. 21, no. 2, pp. 103–111, 2009.

K. B. Rocha, M. Rodríguez-Sanz, M. I. Pasarín, S. Berra, M. Gotsens, and C. Borrell, “Assessment of primary care in health surveys: a population perspective,” European Journal of Public Health, vol. 22, no. 1, pp. 14–19, 2012.

X. Wei, H. Li, N. Yang et al., “Comparing quality of public primary care between Hong Kong and Shanghai using validated patient assessment tools,” PLoS ONE, vol. 10, no. 3, Article ID e0121269, 2015.

J. Tsai, L. Shi, W.-L. Yu, L.-M. Hung, and L. A. Lebrun, “Physician specialty and the quality of medical care experiences in the context of the Taiwan National Health Insurance System,” Journal of the American Board of Family Medicine, vol. 23, no. 3, pp. 402–412, 2010.

J. Charan and T. Biswas, “How to calculate sample size for different study designs in medical research?” Indian Journal of Psychological Medicine, vol. 35, no. 2, pp. 121–126, 2013.

C. L. Kimberlin and A. G. Winterstein, “Validity and reliability of measurement instruments used in research,” American Journal of Health-System Pharmacy, vol. 65, no. 23, pp. 2276–2284, 2008.

J. C. Nunnally Jr., Introduction to Psychological Measurement, McGraw-Hill, New York, NY, USA, 1970.

R. F. DeVellis, “Guidelines in scale development,” in Scale Development: Theory and Applications, pp. 51–91, Sage, Newbury Park, Calif, USA, 1991.

J. E. Helms, K. T. Henze, T. L. Sass, and V. A. Mifsud, “Treating cronbach's alpha reliability coefficients as data in counseling research,” The Counseling Psychologist, vol. 34, no. 5, pp. 630–660, 2006.

R. Likert, “A technique for the measurement of attitudes,” Archives of Psychology, vol. 22, no. 140, pp. 1–55, 1932.

R. D. Hays and T. Hayashi, “Beyond internal consistency reliability: rationale and user's guide for Multitrait Analysis Program on the microcomputer,” Behavior Research Methods, Instruments, & Computers, vol. 22, no. 2, pp. 167–175, 1990.

L. J. Cronbach, Essentials of Psychological Testing, Harper & Row, New York, NY, USA, 1949.

The State Council of People's Republic of China, “The State Council's guideline on establishment of a GP system,” Chinese Journal of General Practices, vol. 10, no. 9, pp. 609–612, 2011 (Chinese).

L. J. Jie, The Coordination between Community Health Centers and Hospitals in Guangzhou, Sun Yat-sen University, Guangzhou, China, 2014.

J.-Q. Zhang, L. Kuang, and Y.-Z. Ma, “Rererral mechanism of rural health service and the influencing factors: an empirical research,” Chinese Rural Health Service Administration, vol. 32, no. 6, p. 556, 2012 (Chinese).

X.-Y. Zhu, H.-H. Xia, L. Kuang, and L.-T. Yin, “Current status, problems and improvement strategy of referral service system for community general practitioners,” Chinese General Practice, vol. 18, no. 16, pp. 1888–1892, 2015 (Chinese).

H. H. X. Wang, S. Y. S. Wong, M. C. S. Wong et al., “Patients' experiences in different models of community health centers in southern China,” Annals of Family Medicine, vol. 11, no. 6, pp. 517–526, 2013.