International Journal of Family Medicine
Volume 2016 (2016), Article ID 8938420, 13 pages
http://dx.doi.org/10.1155/2016/8938420

Research Article

Impacts of Québec Primary Healthcare Reforms on Patients’ Experience of Care, Unmet Needs, and Use of Services

Raynald Pineault,1,2,3,4 Roxane Borgès Da Silva,1,3,4,5 Sylvie Provost,1,2,4 Mylaine Breton,6,7 Pierre Tousignant,1,8 Michel Fournier,¹ Alexandre Prud’homme,¹ and Jean-Frédéric Levesque9,10

¹Direction de Santé Publique du Centre Intégré Universitaire de Santé et de Services Sociaux du Centre-Est-de-l’Île-de-Montréal, 1301 rue Sherbrooke Est, Montréal, QC, Canada H2L 1M3
²Institut National de Santé Publique du Québec, 190 boulevard Crémazie Est, Montréal, QC, Canada H2P 1E2
³Centre de Recherche du Centre hospitalier de l’Université de Montréal, 900 rue St-Denis, Montréal, QC, Canada H2X 0A9
⁴Institut de Recherche en Santé Publique de l’Université de Montréal, 7101 avenue du Parc, Montréal, QC, Canada H3N 1X9
⁵Faculté des Sciences Infirmières de l’Université de Montréal, 2375 chemin de la Côte-Ste-Catherine, Montréal, QC, Canada H3T 1A8
⁶Centre de Recherche de l’Hôpital Charles-Lemoyne, 150 place Charles-Lemoyne, Bureau 200, Longueuil, QC, Canada J5C 2B6
⁷Département des Sciences de la Santé Communautaire de l’Université de Sherbrooke, 3001 12e avenue Nord, Sherbrooke, QC, Canada J1H 5H3
⁸Department of Epidemiology, Biostatistics and Occupational Health, McGill University, 1020 avenue des Pins Ouest, Montréal, QC, Canada H3A 1A2
⁹Centre for Primary Health Care and Equity, University of New South Wales, Kensington, NSW 2033, Australia
¹⁰Bureau of Health Information, 67 Albert Avenue, Chatswood, NSW 2067, Australia

Received 9 October 2015; Accepted 11 January 2016

Academic Editor: Hakan Yaman

Copyright © 2016 Raynald Pineault et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

Introduction. Healthcare reforms launched in the early 2000s in Québec, Canada, involved the implementation of new forms of primary healthcare (PHC) organizations: Family Medicine Groups (FMGs) and Network Clinics (NCs). The objective of this paper is to assess how the organizational changes associated with these reforms have impact on patients’ experience of care, use of services, and unmet needs. Methods. We conducted population and organization surveys in 2005 and 2010 in two regions of the province of Québec. The design was a before-and-after natural experiment. Changes over time between new models and other practices were assessed using difference-in-differences statistical procedures. Results. Accessibility decreased between 2003 and 2010, but less so in the treatment than in the comparison group. Continuity of care generally improved, but the increase was less for patients in the treatment group. Responsiveness also increased during the period and more so in the treatment group. There was no other significant difference between the two groups. Conclusion. PHC reform in Québec has brought about major organizational changes that have translated into slight improvements in accessibility of care and responsiveness. However, the reform does not seem to have had an impact on continuity, comprehensiveness, perceived care outcomes, use of services, and unmet needs.

1. Introduction

Most industrialized countries are facing efficiency problems in their healthcare systems associated with increased longevity and levels of morbidity among their populations. Issues with healthcare systems raise concerns since some countries, such as Canada and the United States, rank lower than other countries on most performance indicators, particularly those related to patients’ experience of care, even though they devote comparable or even higher levels of financial resources [1].

To rectify the situation, different measures have been proposed such as implementing practices that are based on Patient-Centered Medical Home and Chronic Care Model concepts [2, 3]. These proposals recognize the widely accepted notion that primary care is the cornerstone in higher-performing healthcare systems [4, 5]. Search for solutions culminated in 2007 with the Institute for Health Improvement launching the Triple Aim Initiative [6]. The initiative promotes and supports projects aimed at achieving better population health and patients’ experience of care and enhancing healthcare costs containment. The Canadian Foundation for Healthcare Improvement recently adopted the initiative, and some provinces, including Québec, are currently participating in a Triple Aim Collaborative [7].

Several components of the Triple Aim Initiative are found in the reforms initiated in Québec in the early 2000s. The aims of the reforms were to improve accessibility and continuity of primary healthcare and patients’ experience of care while reducing unnecessary use of services, particularly among specialists services. The reforms also aimed to strengthen coordination of care in order to enable patients to navigate more easily the complexity of the healthcare system. Two new types of primary healthcare (PHC) organizations were created: Family Medicine Groups (FMGs) and Network Clinics (NCs) [8, 9]. FMGs incorporate elements of the Chronic Care and the Medical Home Models [3, 10, 11]. A typical FMG consists of 6 to 10 physicians working with nurses and providing services for 8,000 to 15,000 registered patients. FMGs, contracting with the Provincial Ministry of Health and Social Services, agree to increase services provision (e.g., extended opening hours and 24/7 phone access) in exchange for complementary public funding to support computerization and provide the means to hire additional staff, such as nurses. At onset of the reform, the target was to establish 300 FMGs in the province of Québec; as of 2010, there were 217 accredited FMGs. An FMG can comprise more than one clinical setting since it is based on physician participation and engagement, regardless of practice setting. Hence, an FMG can encompass more than one clinic with participating doctors. A complementary organizational model of PHC is the Network Clinic (NC), which was mainly implemented in Montréal under the initiative of the Regional Health Agency. Those clinics more specifically aim to foster accessibility through walk-in visits and provide access to diagnostic support, such as X-rays and lab tests, and to specialists. The distinction between FMG and NC is often blurred, as many clinics have both gained status and became FMG-NC. These clinics thus benefit from two sources of funding, provincial and regional.

In 2004, a local coordination structure was also created in Québec: 95 Health and Social Services Centres were entrusted by the Ministry of Health and Social Services to create local service networks [12]. The centres serve geographically defined populations. They were created by law, merging, on a geographical basis, long-term care hospitals, Local Community Services Centres, and, in most cases, acute care hospitals. Local Community Services Centres are public organizations created in the early 1970s under the governance of provincial authorities. They provide health and social services and physicians are generally paid on salary. Health and Social Services Centres were also mandated to increase collaboration among local actors and consequently to support the development of new emerging forms of local PHC organizations.

Structural changes and implementation processes associated with healthcare reforms have been well documented [13–16]. Likewise, outcomes associated with the introduction of innovative practices or new organizational models have been widely assessed [13, 14, 17–19]. However, the impact of PHC reforms on patients’ experience of care and use of services at the population level has received much less attention [5, 18–21]. The Québec reform provided us with an opportunity to address that question. In close collaboration with the Regional Health and Social Services Agencies of Montréal and Montérégie, we conducted two studies in 2005 and 2010 to assess the evolution of PHC organizations and their impacts on patients’ experience of care, unmet needs, and use of services in both regions [22, 23].

In a previous article, we discussed effects of the reforms on PHC organizations’ changes [16]. Specifically, we found that significant organizational changes had resulted from the reform, mainly due to implementation of new types of PHC organizations (FMGs, NCs, and FMG-NCs). We also found that the main mediating factor accounting for these changes was the mimetic influence of exemplary PHC organizations that acted as promoters for implementation of the new models supported by provincial and regional policies.

The question raised in the conclusion of that article was whether these organizational changes had translated into improvement of patients’ experience of care, unmet needs, and use of services. This is the issue we address in the current paper. More specifically, this paper aims to compare FMGs, NCs, and FMG-NCs implemented since 2003 with the other PHC organizations, in regard to the evolution of patients’ experience of care, unmet needs, and use of services between 2003 and 2010.

2. Methods

2.1. Design

The study design corresponded to a before-and-after natural experiment in which FMGs, NCs, and FMG-NCs constituted the treatment group, and the other clinics formed the comparison group (Figure 1). The intervention was the reforms initiated in 2003 as described in the Introduction. The study consisted of population and organization surveys carried out in 2005 and 2010 in the two most populous regions of the province of Québec: Montréal and Montérégie [23]. In both years, surveys were conducted on two independent population samples. They included 9,206 adults aged 18 or more in 2005 with a response rate of 64% and 9,180 adults in 2010 with a response rate of 56%. Concurrent surveys of all PHC organizations were also carried out; they included 659 organizations in 2005 with a response rate of 71% and 606 organizations in 2010 with a response rate of 62%. These surveys were preceded by telephone calls to the receptionists of all PHC practices to gather basic information such as type of practice (FMG, NC, and FMG-NC), number of physicians, presence of a nurse, and offering walk-in services.

Figure 1: Study design.

The population samples were stratified but not proportionally with approximately 400 respondents in each of the 23 territories. Data were weighted to account for unequal selection probabilities resulting from this type of sampling. Poststratification weighting was applied for age and sex, based on 2006 and 2010 estimated Canadian census data. The population-based questionnaire focused on respondents’ attachment to a PHC organization, healthcare and preventive services received, and experience of care in the two years preceding the surveys as well as unmet needs in the previous six months. The questionnaire drew mainly from two validated instruments, the Primary Care Assessment Survey and the Primary Care Assessment Tool, to which we added questions when a topic had not been addressed [24–26]. In addition to experience of care, the population questionnaire inquired about individual characteristics such as sex, age, education level, economic status, perceived health, and presence of morbidities [23].

The two organizational surveys included all PHC organizations in the two regions (659 in 2005 and 606 in 2010). In every organization, a key informant, usually the physician responsible for professional and administrative matters, completed the questionnaire. The organization questionnaire explored elements of vision, resources, structure, and practices [16]. Population and organization surveys were linked through respondents’ identification of their regular source of primary care in the two years preceding the survey (2003–2005 and 2008–2010 corresponding to the T0 and T1 in Figure 1). Failing to identify a usual source of care, the respondent identified the PHC organization most frequently attended in the past two years. This organization was then regarded as the respondent’s usual source of care.

2.2. Organizational Variables

PHC organizations were classified into two groups: a treatment group that included FMGs, NCs, and FMG-NCs; a comparison group that included Local Community Services Centres (LCSC) and Family Medicine Units (FMU) that were not FMG or NC, and all other clinics. Family Medicine Units are academic training units that are likely to reflect family medicine current practice and thus espouse hosting PHC organizations’ dominant philosophy of care. In 2010, among the different PHC organization models in Montréal and Montérégie, group and solo practices were dominant, as they represented 73% of all organizations and were the usual source of care for 50% of all service users. They are privately run by physicians, who are paid on a fee-for-service basis. Costs are shared but not income. For the purposes of this study, FMGs implemented in Local Community Services Centres and Family Medicine Units were included in the treatment group. In the 2010 survey, there were only 11 LCSCs in the treatment group and 38 in the comparison group. The number of FMU remains marginal in the study as 9 were in the treatment group and only 3 in the comparison group.

Among the numerous indicators derived from responses to the organizational questionnaire, we retained the following (Appendix A):(i)number of physicians;(ii)presence of nurses;(iii)number of information technologies used in the practice;(iv)blood taking services available in the building;(v)radiology services available in the building;(vi)collaboration with other PHC clinics;(vii)collaboration with hospitals;(viii)predominant type of visits in the practice (walk-in, by-appointment, and mixed);(ix)scope of diagnostic, therapeutic, and preventive services offered.

At the time of the 2005 survey, 50 FMGs had already been implemented. Therefore, the baseline data were adjusted for those FMGs to take into consideration time elapsed between accreditation and the survey. We simply applied a regression analysis using pace of change of FMGs accredited after 2005 to those accredited before [16].

2.3. Experience of Care and Utilization-Related Variables

Patients’ experience of care, unmet needs, and use of services were reported by population survey respondents. Experience of care variables were accessibility, continuity, comprehensiveness, responsiveness, and perceived care outcomes [27]. Operationalization details for these variables are presented in Appendix B.

We selected 28 indicators of experience of care and grouped them under five dimensions: accessibility (6 items), continuity (5 items), comprehensiveness (5 items), responsiveness (7 items), and perceived care outcomes (5 items). We carried out factor analyses within each of the five dimensions and calculated Cronbach’s alpha on 2010 population data. Cronbach’s alpha was over .60 for four of the five dimensions: continuity (.61), comprehensiveness (.79), responsiveness (.63), and care outcomes (.82). Cronbach’s alpha was low for accessibility (.30), presumably reflecting the great heterogeneity of the indicators and the formative nature of this index [28]. In fact, the accessibility items are not highly correlated, which characterizes composite indices rather than reflective scales [29]. Experience of care variables were expressed as scores on a 10-point scale.

Aside from experience of care, which referred to the two years preceding the surveys, the population questionnaire contained information on use of emergency rooms, hospital admissions, visits to usual source of care, having a family doctor, reporting unmet needs, preventive care received, and sociodemographic characteristics of respondents [23]. Presence of unmet needs in the six months preceding the survey as well as utilization variables was dichotomous variables.

2.4. Data Analysis

To test for differences between 2003 and 2010 in the treatment and comparison groups, we applied the difference-in-differences technique and matched individuals with the propensity score method to ensure better comparability of the two groups [30–34]. This method is particularly well suited to compare change in outcomes over time between individuals exposed to an intervention (the treatment group) and individuals that are not (comparison group) [32]. Individuals were matched on the basis of propensity scores that estimate the probability that an individual with given characteristics is attached to a PHC organization of the treatment group [35]. The propensity score acts as a balancing score that renders the distribution of observed baseline covariates similar between the treatment group and the comparison group [36]. Analyses were carried out using STATA (version 13). Groups were compared after matching by propensity scores with -tests. The results showed no significant values, indicating that covariates were successfully balanced.

3. Results

3.1. Differences between Treatment and Comparison Groups on Selected Organizational Characteristics

As shown in Table 1, the treatment and comparison groups differ considerably with respect to the organizational characteristics presented, particularly in 2010.

Table 1: Percentage of PHC organizations with selected organizational characteristics, in comparison and treatment groups, 2005 and 2010 surveys.

Comparison group
Treatment group

(other medical clinics)
(FMG-NC, FMG, NC)

2005
2010
Diff.
2005
2010
Diff.

%
%
%
%

Size of practice (number of physicians)

    1
39.3
41.4
2.1
4.1
4.1
0.0

    2 to 3
23.3
24.0
0.7
15.7
14.0
−1.7

    4 to 6
19.0
16.9
−2.1
19.0
14.9
−4.1

    7 or more
18.4
17.7
−0.7
61.2
67.0
5.8

Presence of nurses

    Yes
26.2
30.7
4.5
68.6
87.6
19.0

Information technologies used in the practice

    1 or more
50.0
60.0
10.0
80.2
92.5
12.3

Blood taking service available in the same building

    Yes
40.7
40.7
0.0
66.9
73.6
6.7

Radiology available in the same building

    Yes
6.2
7.4
1.2
22.3
28.9
6.6

Collaboration with other PHC practices

    Yes
41.0
25.2
−15.8
58.7
77.7
19.0

Collaboration with hospitals

    Yes
43.8
36.7
−7.1
60.3
77.7
17.4

Type of visits in the practice predominantly

    Walk-in
19.5
13.8
−5.7
24.0
9.9
−14.1

    By-appointment
66.5
70.5
4.0
48.7
43.0
−5.7

    Mixed
14.0
15.7
1.7
27.3
47.1
19.8

Quantity of diagnostic and therapeutic services available in the practice

    Several (8 out of 10 or more)
12.6
8.3
−4.3
36.4
39.7
3.3

Treatment group’s PHC practices are larger and include virtually no solo practices. They are more likely to have nurses and use information technologies and to have blood taking and radiology services available in the building. They have also developed greater collaboration with other PHC practices and hospitals and offer a wider scope of diagnostic, therapeutic, and preventive procedures. Finally, a higher percentage of practices in the treatment group provide a mix of visits (walk-in and by-appointment) at the expense of other types of visits. This percentage increased between 2005 and 2010 from 27.3% to 47.1% whereas the percentage of those providing predominantly walk-in visits decreased from 24.0% to 9.9%. All other indicators followed the same trend, increasing between 2005 and 2010 in the treatment group but remaining more stable or even decreasing in the comparison group. For example, collaboration with other PHC practices decreased with 15.8% in the comparison group while it increased with 19.0% in the treatment group. We observe the same pattern for collaboration with hospitals, although the decrease was less in the comparison group (−7.1%).

3.2. Differences between Treatment and Comparison Groups in the Evolution of Patients’ Experience of Care, Unmet Needs, and Use of Services

Table 2 shows the evolution of patients’ experience of care, unmet needs, and use of services between 2003–2005 and 2008–2010, as well as differences in evolution between the two groups.

Table 2: Difference-in-differences (DD) of experience of care, unmet needs, and utilization of services (%) between users having NC, FMG, or FMG-NC as usual source of care and those having another type of medical clinic, 2003–2005 and 2008–2010.

2003–2005
2008–2010
DD
% of change

Comparison group
Treatment group
Diff.

Comparison group
Treatment group
Diff.

(other medical clinics)
(FMG-NC, FMG, NC)
(other medical clinics)
(FMG-NC, FMG, NC)

()
()
()
()

Accessibility of services (max score: 10)
7.20
7.25
0.05
.252
6.75
6.98
0.23
<.001
0.18
2.5
.003

Continuity (max score: 10)
8.16
8.00
−0.16
.002
8.65
8.33
−0.32
<.001
−0.16
−2.0
.107

Comprehensiveness (max score: 10)
8.51
8.43
−0.08
.106
8.42
8.27
−0.15
.002
−0.07
−0.8
.343

Responsiveness (max score: 10)
8.71
8.62
−0.09
.004
8.86
8.86
0.00
.031
0.09
1.0
.031

Perceived care outcomes (max score: 10)
8.58
8.50
−0.08
.105
8.75
8.57
−0.18
<.001
−0.10
−1.2
.147

% of users who reported unmet needs for care
19.1
18.8
−0.30
.777
16.1
16.4
0.30
.691
0.6
3.2
.632

% of users who attended usual source of care six times or more
22.8
24.2
1.4
<.001
19.9
19.3
−0.6
.607
−2.0
−8.5
.200

% of users who attended emergency room at least once
34.7
36.7
2.0
.101
35.2
36.2
1.0
.396
−1.0
−2.8
.545

% of users hospitalized at least once
16.1
16.5
0.4
.681
17.8
19.8
2.0
.105
1.6
9.8
.413

The most surprising results concern accessibility which declined between 2003–2005 and 2008–2010 in both groups. However, it declined to a lesser degree for individuals attached to practices of the treatment group than those of the comparison group. This result, in favor of the treatment group, is shown with the positive and significant DD (+0.18; ) and corresponds to a 2.5% change. The percentage of change was calculated by dividing the DD value by the mean baseline scores of both groups. Indicators that contributed the most to this change were the possibility of seeing another doctor when the regular doctor was not available and adequate opening hours of the clinic (data not presented).

A positive DD value is also noted for responsiveness that, unlike accessibility, increased during the same period in the two groups but at a higher pace in the treatment group (+0.09; ), corresponding to a 1.0% change. Indicators that contributed the most to this change are shorter waiting time between arrival at the clinic and seeing the doctor and courtesy of reception (data not presented).

Continuity generally improved between 2003–2005 and 2008–2010. At the two periods, the comparison group performed better, as shown by the negative and significant differences. The differences widened over the years but not enough to reach the level of statistical significance for the DD value (−0.16; ). Indicators that contributed the most to the differences at the two periods are “how long the patient goes to his usual source of primary care” and “whether he always sees the same doctor at this place” (data not presented).

Perceived care outcomes scores slightly improved over time and more so in the comparison group. Although the difference between the two groups was significant in 2008–2010 (), the DD value failed to reach statistical significance (−0.10; ).

The score for comprehensiveness was higher in the comparison than in the treatment group at the two time periods but especially in 2008–2010 (). Still, the change over time was not sufficient to yield a significant DD value. For all other indicators of experience of care, unmet needs, and use of services, DD values were not statistically significant.

4. Discussion

Implementation of FMGs in the two regions has brought about important organizational changes, mainly in terms of resources and structures of PHC organizations [16]. However, the impact of these organizational changes on patients’ experience of care, unmet needs, and use of services has been so far limited. The major impact has been on accessibility. Introduction of the new PHC models has potentially slowed deterioration in accessibility. A second positive effect is on responsiveness. A main contributing factor to increased responsiveness was decreased waiting time between arrival of a patient in a doctor’s office and the face-to-face encounter with the doctor. In that sense, this indicator of responsiveness partly mirrors accessibility.

The positive effect of FMGs and NCs on continuity has been less important than that of other PHC organizations. This result raises concerns because FMGs have been entrusted with improving continuity of care. Again, we should recall that main contributing factors to the relative decrease of continuity in the treatment group were not being able to always see the same doctor at the regular source of care and length of time a patient has been attached to the clinic. We further explored the lower increase in continuity in the treatment group by carrying out separate DD analyses for PHC organizations accredited between 2003 and 2006 and those accredited between 2007 and 2010. The DD values were very close in both analyses for all indicators except continuity which showed a negative and significant DD value for clinics of the treatment group accredited between 2007 and 2010 (data not presented). This difference could be explained partly by the fact that NCs which focus on accessibility more than continuity were mostly implemented during that period. An additional factor is that incentive-linked enrolment of patients presenting specific condition was implemented for all PHC organizations in 2003, thus lessening the advantageous position of FMGs and NCs [37].

Our findings for continuity contradict those reported by Tourigny et al. in their study of 1275 patients followed in five FMGs implemented in two regions of Québec [38]. That study found an increase in continuity but a decrease in accessibility after 18 months among the patients enrolled in those FMGs. Our study is not comparable to the Tourigny et al. study in many regards, namely, number of FMGs studied, inclusion of NCs as new forms of PHC organizations, and regions covered by the study [23, 38]. Our population sample size and the large number of PHC organizations surveyed allow us to generalize with a fair degree of confidence, at least for the two regions under study.

Like most studies that have examined continuity of care, our measures of continuity focus on face-to-face contacts with the regular doctor and not on group continuity (seeing different doctors or other professionals in the same clinic). This is presumably the reason why studies report solo practices ranking the highest in terms of patient experience of care, with the exception of accessibility [39]. This limitation in the measure of continuity has been raised by authors and new measures have been proposed to correct it [40, 41]. Finally, our results did not show that FMGs and NCs have had an impact on use of services or unmet needs.

Our results concur with results reported by other studies on the partial failure of FMGs and NCs in Québec to attain expected results. In a study involving the entire population of Quebec, Dunkley-Hickine found that a higher degree of physician participation in FMGs in a geographical area was not associated with improved accessibility for the population living in this area [42]. In a cohort study based on large administrative data banks and involving 79 FMGs implemented between 2002 and 2004, Strumpf observed no improvement in accessibility but a slight reduction in use of primary care and specialist services [43]. Using the same data bank, Héroux et al. followed a cohort of 122,722 patients enrolled in 79 FMGs compared to 675,102 who were not [44]. They found a small reduction in number of emergency visits for those in FMGs but, as in our study, no change in hospitalizations.

We must recognize that the PHC reform in Quebec has added resources: financial, technical, and human. However, the impact on the population has been limited. This contrasts, for example, with Ontario where the introduction of major financial incentives for physicians to adopt other modes of payment and particularly per capita and blended payment schemes has led to major changes in medical practice [9, 21, 45–47].

A survey conducted by the Commonwealth Fund in 2013 compared different countries and three Canadian provinces on several indicators of experience of care [48]. The report shows that experience of care reported by Quebec respondents, particularly regarding accessibility, was generally less favorable in Quebec than in Ontario, Alberta, and the rest of Canada. Since most of these indicators are related to PHC services, they reflect a relatively poor performance, given the major financial efforts devoted to reforms in this sector.

A recent report of the Auditor General of Quebec to the National Assembly drew a very critical picture of the situation around the implementation of FMGs and NCs [49]. It concluded that FMGs and NCs failed to fully attain the objectives set by the Ministry. It attributes this failure mainly to the absence of clear rules, guidelines, and incentives and to the lack of control of the Ministry and Regional Agencies over the implementation. This has left FMGs and NCs free to develop on their own without having to be fully accountable to the Ministry or the Regional Agencies.

A recent report of the C. D. Howe Institute by Forget has come to similar conclusions [50]. It underlines several factors that have slowed implementation of FMG policies, notably inadequate development of teamwork, and low registration rate of patients.

Overall, the results reported in this paper indicate that reforms in PHC organizations initiated in 2003 have brought about some expected benefits but have yet to be completed. The reported impact of the new types of PHC organizations on accessibility and responsiveness partly fulfills the objectives set at onset of the reforms. Other results raise concerns, notably regarding continuity, use of services, and unmet needs.

5. Limitations

Our study has limitations. A population survey lends itself to possible recall bias by respondents reporting on their experience of care. If present, the bias should be equally distributed among respondents. Recall bias is more likely to occur when referring to single events taking place at a given point in time than with an overall experience extending over a certain period of time.

We did not have information on nonrespondents to the population surveys, which could also introduce a bias. However, the bias was minimized by applying a weighting procedure that adjusts samples’ data to the 2005 and 2010 populations’ data. In addition, our analyses control for individual characteristics of respondents.

Measures of experience of care are expressed from the patient’s points of view and perceptions. In that sense, these measures are subjective but still the most appropriate to describe experience of care. Our scales are largely derived from two validated instruments, PCAT and PCAS [24–26]. We constructed new scales of experience of care that were more adapted to our context. Following the classical theory of measure, we carried out factor analyses and calculated Cronbach’s alpha coefficients. The coefficient values were generally close to the commonly accepted level (.70) except for accessibility which was low (.30) [51]. We have kept accessibility in our analyses, considering it is a formative composite index in which the items composing the index are not necessarily correlated nor substitutable with each other, as is the case in reflective scales [29].

The imputation method we applied to nonrespondents of the PHC organization survey might have introduced a bias since it is based on respondent data. As pointed out by Haziza, the nonresponse bias may be greater with nonresponse than with imputation, particularly when basic information is available, as was the case here from calls to receptionists prior to the organization survey [52]. We tested the magnitude of a possible bias by carrying out statistical analyses with respondent organizations only. The results obtained were similar to those with grouped respondents and nonrespondents [16].

6. Conclusion

In spite of important resources devoted to PHC reforms initiated in Quebec in the early 2000s, results so far have been limited in terms of experience of care improvement, reduction of utilization of services, and unmet needs. This is the conclusion we can draw when we compare the evolution of PHC practices that have become FMGs, NCs, or FMG-NCs to that of PHC practices that have not acquired such status. This is not to say that the healthcare system has not improved as a whole. For example, we observed that continuity has increased at the system level but more so in practices that were not FMG, NC, or FMG-NC. Accessibility has decreased but much less than it would have without FMGs, NCs, and FMG-NCs. These examples highlight the fact that it is hazardous and perhaps unfair to attribute specific effects of a social policy to a single intervention, be it success or failure. Social systems are complex and their components are highly interactive. The effect of a change may induce other, often unexpected, changes. Finally, implementing policy changes takes time [53]. This is the conclusion of a recent article that assessed one of the earliest and largest medical home pilots implemented in the United States [54]. How long it takes to realistically and fairly assess sustainable benefits brought about by primary care reforms such as the ones initiated in Québec in the early 2000s remains difficult to ascertain.

Appendices

A. Organizational Variables

See Table 3.

Table 3: (Appendix A) Construction of organizational indicators.

Size of practice

    How many general practitioners, including those working part time, currently work at your clinic?

Presence of nurses

    How many nurses currently work at your clinic?

Information technologies used in the practice

    In your clinic, do you use…

    access to the health and social services telecommunications network?

    a Web-based appointment system for patients?

    computerized tools for continuing professional education?

    electronic medical records?

    electronic interface to diagnostic imaging laboratory services?

    computerized tools to aid medical decision-making?

Blood taking service available in the same building

    Are the following services available in the building where your clinic is located…

    Blood samples taking?

Radiology available in the same building

    Are the following services available in the building where your clinic is located…

    Radiology?

Collaboration with other PHC practices

    Does your clinic have formal or informal arrangements with other PHC clinics…

    to schedule services offered?

    to access to technical services?

    to exchange resources?

Collaboration with hospitals

    Does your clinic have formal or informal arrangements with hospitals…

    to schedule services offered?

    to access to technical services?

    to exchange resources?

Predominant type of visits in the practice

    What percentage of walk-in visits to all visits do you provide at your clinic?

    0% to 25% = By-appointment visits

    26 to 75% = Mixed

    51% or more = Walk-in visits

Quantity of diagnostic or therapeutic services available in the practice

    At your clinic, are the following services available…

    Strep-test?

    Skin biopsy?

    IUD insertion?

    Musculo-skeletal injection/aspiration?

    Suture/minor surgery?

    Cervical smear (Pap test)?

    Childhood vaccination?

    Influenza vaccination?

    3 or less among these = Few

    4 or 5 among these = A fair number

    6 or more among these = Several

B. Experience of Care Indices

See Table 4.

Table 4: (Appendix B) Construction of experience of care indices.

The following questions refer to the usual PHC source identified by the respondents

Accessibility of services

    if the doctor who is responsible for your care is not available, you can see another doctor?
0. Never/Sometimes/Often

1. Always

    how long does it take to see the doctor by appointment?
0. Two weeks or more

1. Less than two weeks

    how long does it usually take to get there?
0.15 minutes or more

1. Less than 15 minutes

    the office hours are convenient?
0. Not at all/A little/Somewhat agree

1. Strongly agree

    it is easy to reach someone by telephone to make an appointment?
0. Not at all/A little/Somewhat agree

1. Strongly agree

    it is easy to talk to a doctor or nurse by telephone?
0. Not at all/A little/Somewhat agree

1. Strongly agree

Continuity of care

    you see the same doctor?
0. Never/Sometimes/Often

1. Always

    how long have you been going there?
0. Five years or less

1. More than 5 years

    your medical history is known?
0. Not at all/A little/Somewhat agree

1. Strongly agree

    they are aware of all the prescribed drugs you take?
0. Not at all/A little/Somewhat agree

1. Strongly agree

    you can receive routine ongoing care for a chronic problem?
0. Not at all/A little/Somewhat agree

1. Strongly agree

Responsiveness

    how long do you have to wait between the scheduled time of appointment and the time you actually see the doctor?
0. Less than 60 minutes

1. 60 minutes or more

    the staff answer your questions clearly ?
0. Not at all/A little/Somewhat agree

1. Strongly agree

    you feel respected?
0. Not at all/A little/Somewhat agree

1. Strongly agree

    you are greeted courteously at the reception?
0. Not at all/A little/Somewhat agree

1. Strongly agree

    your physical privacy is respected?
0. Not at all/A little/Somewhat agree

1. Strongly agree

    the doctors spend enough time with you?
0. Not at all/A little/Somewhat agree

1. Strongly agree

    the local of the clinic are pleasant?
0. Not at all/A little/Somewhat agree

1. Strongly agree

Comprehensiveness

    all your health problems are taken care of whether they are physical or psychological?
0. Not at all/A little/Somewhat agree

1. Strongly agree

    the doctor takes the time to talk to you about prevention and asks you about your lifestyle habits?
0. Not at all/A little/Somewhat agree

1. Strongly agree

    they help you get all the health care services you need?
0. Not at all/A little/Somewhat agree

1. Strongly agree

    your opinion and what you want are taken into account in the care that you receive?
0. Not at all/A little/Somewhat agree

1. Strongly agree

    you are given help to weigh the pros and cons when you have to make decisions about your health?
0. Not at all/A little/Somewhat agree

1. Strongly agree

Outcome of care

    the services you get help you to better understand your health problems?
0. Not at all/A little/Somewhat agree

1. Strongly agree

    the services you get help you to prevent certain health problems before they appear?
0. Not at all/A little/Somewhat agree

1. Strongly agree

    the services you get help you to control your health problems?
0. Not at all/A little/Somewhat agree

1. Strongly agree

    the professionals you see encourage you to follow the treatments prescribed?
0. Not at all/A little/Somewhat agree

1. Strongly agree

    the professionals you see help motivate you to adopt good lifestyle habits?
0. Not at all/A little/Somewhat agree

1. Strongly agree

Unmet needs

    During the last 6 months, did you feel you needed to see a doctor for a health problem but didn’t see one?
0. No

1. Yes

Conflict of Interests

The authors declare that they have no competing interests.

Acknowledgments

The authors acknowledge the contribution of the following collaborators and researchers associated with the project: D. Roy, J. L. Denis, P. Lamarche, M. D. Beaulieu, J. Haggerty, D. Roberge, J. Côté, M. Hamel, F. Goulet, M. Drouin, J. Rodrigue, L. Côté, and O. Lemoine. The study benefited from the financial contributions of the Canadian Institutes of Health Research (no. PHE 101905), the Fonds de Recherche du Québec-Santé (FRQS) (no. 22032), the Agence de la Santé et des Services Sociaux de Montréal, and the Agence de la Santé et des Services Sociaux de la Montérégie. Isabelle Rioux provided editorial assistance and Sylvie Gauthier revised the text.

References

K. Davis, K. Stremikis, D. Squires, and C. Schoen, Mirror, Mirror on the Wall: How the Performance of The U.S. Healthcare System Compares Internationally, vol. 1755, The Commonwealth Fund, 2014.

The College of Family Physicians of Canada, A vision for Canada: Family practice—The Patient's Medical Home, 2011

V. J. Barr, S. Robinson, B. Marin-Link et al., “The expanded chronic care model: an integration of concept and strategies from population health promotion and the chronic care model,” Hospital Quarterly, vol. 7, no. 1, pp. 73–82, 2003.

B. Starfield, L. Shi, and J. Macinko, “Contribution of primary care to health systems and health,” Milbank Quarterly, vol. 83, no. 3, pp. 457–502, 2005.

J. L. Haggerty, J.-F. Lévesque, W. Hogg, and S. Wong, “The strength of primary care systems,” British Medical Journal, vol. 346, no. 7914, Article ID f3777, 2013.

D. McCarthy and S. Klein, “The triple aim journey: improving population health and patients' experience of care, while reducing costs,” The Commonwealth Fund, vol. 48, no. 1421, pp. 1–11, 2010.

Institute for Healthcare Improvement, “IHI Triple Aim Initiative,” 2015, http://www.ihi.org/

M.-P. Pomey, E. Martin, and P.-G. Forest, “Québec's family medicine groups: innovation and compromise in the reform of front-line care,” Canadian Political Science Review, vol. 3, no. 4, pp. 31–46, 2009.

M. Breton, J. F. Levesque, R. Pineault, and W. Hogg, “Primary care reform: can Québec's family medicine group model benefit from the experience of Ontario's family health teams?” Healthcare Policy, vol. 7, no. 2, pp. e122–e135, 2011.

T. Bodenheimer, E. H. Wagner, and K. Grumbach, “Improving primary care for patients with chronic illness,” The Journal of the American Medical Association, vol. 288, no. 14, pp. 1775–1779, 2002.

B. E. Landon, J. M. Gill, R. C. Antonelli, and E. C. Rich, “Prospects for rebuilding primary care using the patient-centered medical home,” Health Affairs, vol. 29, no. 5, pp. 827–834, 2010.

M. Breton, R. Pineault, J.-F. Levesque, D. Roberge, R. B. Da Silva, and A. Prud'Homme, “Reforming healthcare systems on a locally integrated basis: is there a potential for increasing collaborations in primary healthcare?” BMC Health Services Research, vol. 13, article 262, 2013.

B. Hutchison, J.-F. Levesque, E. Strumpf, and N. Coyle, “Primary health care in Canada: systems in motion,” The Milbank Quarterly, vol. 89, no. 2, pp. 256–288, 2011.

B. Hutchison and R. Glazier, “Ontario's primary care reforms have transformed the local care landscape, but a plan is needed for ongoing improvement,” Health Affairs, vol. 32, no. 4, pp. 695–703, 2013.

M. W. Friedberg, P. S. Hussey, and E. C. Schneider, “Primary care: a critical review of the evidence on quality and costs of health care,” Health Affairs, vol. 29, no. 5, pp. 766–772, 2010.

R. Pineault, R. Borgès Da Silva, A. Prud'homme et al., “Impact of Québec's healthcare reforms on the organization of primary healthcare (PHC): a 2003–2010 follow-up,” BMC Health Services Research, vol. 14, article 229, 2014.

M.-D. Beaulieu, J. Haggerty, P. Tousignant et al., “Characteristics of primary care practices associated with high quality of care,” Canadian Medical Association Journal, vol. 185, no. 12, pp. e590–e596, 2013.

L. Shi, “The impact of primary care: a focused Review,” Scientifica, vol. 2012, Article ID 432892, 22 pages, 2012.

J. L. Haggerty, R. Pineault, M.-D. Beaulieu et al., “Practice features associated with patient-reported accessibility, continuity, and coordination of primary health care,” Annals of Family Medicine, vol. 6, no. 2, pp. 116–123, 2008.

M. W. Friedberg, K. L. Coltin, D. G. Safran, M. Dresser, A. M. Zaslavsky, and E. C. Schneider, “Associations between structural capabilities of primary care practices and performance on selected quality measures,” Annals of Internal Medicine, vol. 151, no. 7, pp. 456–463, 2009.

R. H. Glazier, J. Klein-Geltink, A. Kopp, and L. M. Sibley, “Capitation and enhanced fee-for-service models for primary care reform: a population-based evaluation,” Canadian Medical Association Journal, vol. 180, no. 11, pp. e72–e81, 2009.

R. Pineault, J. F. Levesque, D. Roberge, M. Hamel, P. Lamarche, and J. Haggerty, Accessibility and Continuity of Care: A Study of Primary Healthcare in Québec, The Canadian Institutes of Health Research and the Canadian Health Services Research Foundation, 2008.

J.-F. Levesque, R. Pineault, S. Provost et al., “Assessing the evolution of primary healthcare organizations and their performance (2005–2010) in two regions of Québec province: montréal and Montérégie,” BMC Family Practice, vol. 11, article 95, 2010.

L. Shi, B. Starfield, and J. Xu, “Validating the adult primary care assessment tool,” Journal of Family Practice, vol. 50, no. 2, pp. 161–175, 2001.

D. G. Safran, M. Kosinski, A. R. Tarlov et al., “The primary care assessment survey: tests of data quality and measurement performance,” Medical Care, vol. 36, no. 5, pp. 728–739, 1998.

J. Haggerty, R. Pineault, M. D. Beaulieu et al., Accessibility and Continuity of Primary Care in Québec, The Canadian Health Services Research Foundation, 2004.

D. G. Safran, “Defining the future of primary care: what can we learn from patients?” Annals of Internal Medicine, vol. 138, no. 3, pp. 248–255, 2003.

K. A. Bollen and S. Bauldry, “Three Cs in measurement models: causal indicators, composite indicators, and covariates,” Psychological Methods, vol. 16, no. 3, pp. 265–284, 2011.

A. Diamantopoulos and H. M. Winklhofer, “Index construction with formative indicators: an alternative to scale development,” Journal of Marketing Research, vol. 38, no. 2, pp. 269–277, 2001.

J. D. Angrist and J. S. Pischke, “Parallel words: fixed effects, difference-in-differences, and panel date,” in Mostly Harmless Econometric, pp. 221–246, Princeton University Press, Princeton, NJ, USA, 2009.

B. D. Meyer, “Natural and quasi-experiments in economics,” Journal of Business & Economic Statistics, vol. 13, no. 2, pp. 151–161, 1995.

P. J. Gertler, S. Martinez, P. Premand, L. B. Rawlings, and C. M. J. Vermeersch, “Difference-in-differences,” in Impact Evaluation in Practice, P. J. Gertler, S. Martinez, P. Premand, L. B. Rawlings, and C. M. J. Vermeersch, Eds., pp. 95–105, International Bank for Reconstruction and Development, The World Bank, Washington, DC, USA, 2011.

B. A. Griffin and D. F. McCaffrey, “Nonparametric matching and propensity scores,” in Encyclopedia of Health Economics, pp. 370–374, Elsevier, Oxford, UK, 1st edition, 2014.

J. B. Dimick and A. M. Ryan, “Methods for evaluating changes in health care policy: the difference-in-differences approach,” The Journal of the American Medical Association, vol. 312, no. 22, pp. 2401–2402, 2014.

E. A. Stuart, “Matching methods for causal inference: a review and a look forward,” Statistical Science, vol. 25, no. 1, pp. 1–21, 2010.

P. C. Austin, “An introduction to propensity score methods for reducing the effects of confounding in observational studies,” Multivariate Behavioral Research, vol. 46, no. 3, pp. 399–424, 2011.

Régie de L'assurance-Maladie du Québec, “Services de médecine de famille de prise en charge et de suivi de la clientèle Inscription générale de la clientèle et autres changements. Communiqué aux professionnels de la santé et aux dispensateurs de soins assurés,” 2008, http://www.ramq.gouv.qc.ca/sitecollectiondocuments/professionnels/infolettres/2008/com145-8.pdf

A. Tourigny, M. Aubin, J. Haggerty et al., “Patients' perceptions of the quality of care after primary care reform: family medicine groups in Quebec,” Canadian Family Physician, vol. 56, no. 7, pp. e273–e282, 2010.

R. Pineault, R. Borgès Da Silva, S. Provost et al., “Primary healthcare solo practices: homogeneous or heterogeneous?” International Journal of Family Medicine, vol. 2014, Article ID 373725, 10 pages, 2014.

P. Tousignant, M. Diop, M. Fournier et al., “Validation of 2 new measures of continuity of care based on year-to-year follow-up with known providers of health care,” Annals of Family Medicine, vol. 12, no. 6, pp. 559–567, 2014.

J. L. Haggerty, D. Roberge, G. K. Freeman, C. Beaulieu, and M. Bréton, “Validation of a generic measure of continuity of care: when patients encounter several clinicians,” Annals of Family Medicine, vol. 10, no. 5, pp. 443–450, 2012.

C. Dunkley-Hickin, Effects of primary care reform in Quebec on access to primary health care services [M.S. thesis], McGill University, Department of Epidemiology and Biostatistics, 2013.

E. Strumpf, “L'évaluation des groupes de médecine de famille (GMF),” in L'Évaluation Économique et la Recherche sur les Services de Santé, Ministère des Affaires Sociales, de la Santé et des Droits des Femmes, Paris, France, December 2014.

J. Héroux, E. E. Moodie, E. Strumpf, N. Coyle, P. Tousignant, and M. Diop, “Marginal structural models for skewed outcomes: identifying causal relationships in health care utilization,” Statistics in Medicine, vol. 33, no. 7, pp. 1205–1221, 2014.

S. Dahrouge, W. Hogg, N. Ward et al., “Delivery of primary health care to persons who are socio-economically disadvantaged: does the organizational delivery model matter?” BMC Health Services Research, vol. 13, article 517, 2013.

A. Sweatman and G. Buckley, “Ontario's experiment with primary care reform,” The School of Public Policy, vol. 7, no. 11, 2014.

J. Kantarevic, B. Kralj, and D. Weinkauf, “Enhanced fee-for-service model and access to physician services evidence from family health groups in Ontario,” IZA Discussion Paper 4862, 2010.

Commissaire à la Santé et au Bien-Être du Québec, Perceptions et Expériences de Soins de la Population: Le Québec Comparé, Résultats de L'enquête Internationale sur les Politiques de Santé du Commonwealth Fund de 2013, Gouvernement du Québec, 2014.

Vérificateur Général du Québec, Groupe de médecine de famille et Cliniques-réseau, Services de première ligne dans le réseau de la santé et des services sociaux. Rapport du Vérificateur général du Québec à l’Assemblée nationale pour l’année 2015-2016. Vérification de l’optimisation des ressources Services de première ligne dans le réseau de la santé et des services sociaux. Gouvernement du Québec, 2015

C. Forget, The Case of the Vanishing Quebec Physicians: How Improve Access to Care, vol. 410 of Commentary, C.D. Howe Institute, 2014.

J. Bland and D. Altman, “Statistics notes: Cronbach's alpha,” British Medical Journal, vol. 314, no. 7080, article 572, 1997.

D. Haziza, “Imputation and inference in the presence of missing data,” in Handbook of Statistics, C. R. Rao and D. Pfeffermann, Eds., vol. 29 of Sample Surveys: Theory Methods and Inference, pp. 215–246, Elsevier, 2009.

B. Hutchison, “Reforming Canadian primary care—don't stop half-way,” Healthcare Policy, vol. 9, no. 1, pp. 12–25, 2013.

M. W. Friedberg, E. C. Schneider, M. B. Rosenthal, K. G. Volpp, and R. M. Werner, “Association between participation in a multipayer medical home intervention and changes in quality, utilization, and costs of care,” The Journal of the American Medical Association, vol. 311, no. 8, pp. 815–825, 2014.