Factors Explaining Program Sustainability: A Study of the Implementation of a Social Services Program in Sweden

,


Introduction
Evidence suggests that few implementation initiatives are sustained over time [1].Even if the initial implementation is deemed successful, there is a great risk that the organizational changes will not survive in the long term [2,3].Tus, it is common for organizations to recede back into old routines as new practices are forgotten or abandoned.Tis problem has been described as the "improvement evaporation efect" or "initiative decay" [4].
Despite extensive research on implementation in public organizations, there is a need for further investigation into how new programs can be sustained over time to ensure continued benefts for clients [5][6][7].Previous research has identifed a plethora of factors afecting program sustainability, but they have not been systematically compared.For example, Bodkin and Hakimi [8] identify 274 facilitators and barriers in a systematic review of health promotion programs.Despite identifcation, there remains a lack of research as to how these factors interact and intersect, as well as which factors are most important in diverse contexts.To evoke Matland [9], this vast collection of factors does not need more factors but structure.It is reasonable to assume that all factors are not equally important in each case.For example, their signifcance depends on the characteristics of the object of implementation and varies between organizational and institutional settings [10,11].Terefore, it is imperative to contextualize sustainability research and study which of the previously identifed factors best account for program sustainability.
Te aim of this study is therefore to evaluate the impact of sustainability factors identifed in previous research and to explain how these factors relate to each other.In this paper, we report fndings from a cross-sectional survey testing a model of sustainability factors in a case of implementation within the Swedish social services.We also elaborate on how the organizational setting may afect the factors' impact on program sustainability.
Te studied program is called IBIC (individens behov i centrum, or roughly translated, the individual's needs in focus) and was developed in 2016 by the National Board of Health and Welfare (NBHW).Te stated objective of the program has been to establish a more person-centered and needs-oriented social care, but an overarching goal has also been to promote evidence-based practice through structured documentation, standardized processes, and a common conceptual framework to assess individual needs, resources, goals, and results [12].Te implementation initiative included the NBHW at the national level and 173 out of the 290 Swedish municipalities.Our study focuses on the implementation at the municipal level.
Social services in Sweden are tax-fnanced and based on a decentralized model whereby oversight is allocated to 290 municipalities.Tis sector is structured by means of a purchaser-provider split, meaning that public providers, nonproft organizations, and private companies can be contracted as providers by municipalities.IBIC is intended to permeate both the purchaser and provider organizations.Terefore, its implementation not only involves numerous municipalities, but also a variety of organizations at each implementation site.Consequently, the recipient organizations are not single, unifed organizations, but rather clusters in which several organizational units interact.We identify this characteristic as organizational fragmentation and highlight its signifcance in the concluding discussion.
In the next section, sustainability is defned and related to the concepts of implementation and routinization.Ten, an overview of factors shown to afect sustainability in previous research is presented.In the subsequent methods section, the study object and setting, research process, questionnaire, respondents, measures, and analyses are described in detail.Tereafter, the fndings are reported, with the main fndings summarized in Table 1.In the concluding discussion, we elaborate on the key fndings and situate them within the fragmented organizational settings that characterize the context of implementation.
1.1.Defning Sustainability.Sustainability research is itself fragmented.Te feld is theoretically pluralistic and there is no standard or commonly accepted defnition of sustainability [6,7,13].In this study, we use implementation and sustainability as two distinct but related concepts that can be seen as concomitant processes [14].Implementation is defned as the process of putting an innovation, new method, or program to use within an organizational setting [15,16].Sustainability is defned in accordance with Fleiszer et al. [5] as a process whereby "improvements are maintained, new ways of working become routine, surrounding systems are transformed in support and the innovation may even be developed, over a period of time appropriate to a given situation." Tis defnition of sustainability combines three main elements recurring in the literature: (1) routinization, (2) program benefts, and (3) continued development or adaptation to dynamic contexts.In this paper, we focus mainly on routinization, which can be seen as the primary or fundamental process of sustainability [17,18].Routinization is the process of embedding a program into organizational structures and processes, such that the program becomes a part of the organization's core services [3,19,20].However, the defnition does not suggest that all programs are to be routinized, but rather that it is crucial to maintain efective programs that provide benefts, such as improvements for users and organizations [21,22].Furthermore, routinized programs should not be static, but continuously adapted to changes in the organizational and institutional environment, and eventually deimplemented when obsolete [7,23].

Factors Infuencing Sustainability.
In the sustainability literature, there is a theoretical tension between a rationalistic, top-down perspective emphasizing fdelity, planning, and control, and a dynamic, bottom-up perspective emphasizing local agency, adaptation, and continuous learning [9,24].In their seminal 1998 review, Shediac-Rizkallah and Bone [18] propose a three-level framework of factors infuencing sustainability.We use this framework to present an overview of factors recurring in the literature, both from top-down and bottom-up perspectives.Te framework includes factors related to (1) the implementation project, (2) the organizational setting, and (3) the broader community and institutional context.
(1) Factors related to the implementation project include the characteristics of the implementation object.For example, complex and resource-intensive programs are less likely to be sustained than simpler programs [22,25].Factors at the project level also include process factors such as strategic planning and early planning for sustainability [8,14], project structure, communication, and use of performance monitoring systems, especially evaluation and feedback [5,26,27].Several researchers emphasize that programs which are modifable and adaptable to local conditions are more likely to be sustained [3,6,28,29].Some researchers also claim that sustainability is more likely if the implementation strategy and process is adapted and developed over time based on ongoing evaluations and continuous learning [24,30,31].
(2) Factors related to the organizational setting include the characteristics of the recipient organization, such as its absorptive capacity [5,22], level of education Health & Social Care in the Community and stability of the workforce [1], and available resources to efectively manage the program [8,11,27].Te organizational setting also includes both management and staf commitment to the program.Managers may allocate sufcient resources, establish a shared vision, and build a widespread commitment to the program [8,[32][33][34].In addition, staf participation and shared decision-making can create a broad sense of ownership and support for the program [1,35,36].Some researchers emphasize that frst-line managers have a central role as they can motivate staf and lead the implementation at ground-level [32,37].(3) Factors in the broader community and institutional context include support from other organizations in the environment.For example, sustainability is more likely if there is collaboration and knowledge exchange with other organizations implementing the same program [31,38,39].Collaboration can enhance access to required resources, expertise, new perspectives, and political support [3,24,40].Sustainability may also be supported by the involvement of government agencies and universities [31], diversity of funding, and involvement of the principle funding body [33,34].Moreover, the institutional context includes the legislative and regulatory environment [23], and social norms, trends, and rationalized myths in the surrounding society [22,41].

Materials and Methods
Sustainability factors identifed in previous research were tested in a cross-sectional survey.First, we developed a questionnaire and recruited potential participants.Second, the data were processed in three main stages: (a) responses from the same organization were aggregated, (b) principal component analyses (PCA) were performed to identify latent patterns in the data, informing the construction of index variables, and (c) a multiple imputation was carried out to replace missing data.Tird, variables were analysed with descriptive statistics and linear regression.All analyses were conducted in SPSS 27.

Study Object and Setting.
Te social services in Sweden are organized based on a purchaser-provider split, and it follows that IBIC should be implemented and routinized in both purchaser and provider organizations.IBIC was developed by NBHW with the explicit goal of ensuring that the provision of social care services is based on the needs of each individual [12].IBIC is described as a needs-oriented, systematic, and structured working model, which can be applied to all adults with needs according to the Social Services Act (SFS 2001:453) or the Act Regulating Support and Service to Persons with Certain Functional Impairments (SFS 1993:387).IBIC is based on the International Classifcation of Functioning, Disability, and Health (ICF) [42], and introduces a standardized process and terminology to assess the individual's needs and resources in eleven "areas of life."Individual results and goal fulflment are to be measured and evaluated systematically.Data should be used to adjust the individual's treatment and services, but also as a basis for quality improvements [12].Overall, IBIC is a complex program, as it afects the whole process, from needs assessment to follow-up, and involves both purchasers and providers.Te challenges of these complexities were refected in a government report which concluded that the implementation is demanding and time-consuming [43].
NBHW has a government mandate to promote IBIC implementation, and the Swedish Association of Local Authorities and Regions (SALAR) also supports the implementation.When our data collection was carried out, 173/290 municipalities had implemented the program [44], and the implementation process had been ongoing for an average of 3.7 years (SD � 2.17) across the sites.

Questionnaire.
A questionnaire was developed from previous sustainability research and in close dialogue with IBIC experts at NBHW and SALAR.Te questions were informed by several instruments recurring in the literature.Sustainability was measured in terms of routinization through questions inspired by the Level of Institutionalization Scales [45], a questionnaire to assess organizational routines [46], and an instrument for measuring the sustainability of changed work practices [19].Sustainability determinants were derived from the literature review and informed by the Program Sustainability Index [47] and the Program Sustainability Assessment Tool [48].
Te questionnaire was then validated by four local implementers representing four municipalities.Tey performed a pilot test of the questionnaire, and the questions were adjusted based on their feedback.Te questionnaire was designed as a web survey and was sent out in November 2020.Tree reminders were sent at two-week intervals.

2.3.
Respondents.An e-mail with information about the study was sent to the head of administration in all municipal social services.Te study was also presented at two national IBIC conferences arranged by NBHW and SALAR.Potential participants were invited to register via an application form with questions about their role in the implementation.Te inclusion criteria were that the municipality had implemented the program and that the respondent had a role with high involvement in the implementation.186 participants signed up and met the inclusion criteria.Tey represented 125 municipalities, corresponding to 72.3% of those implementing IBIC.
Te number of responses received was 155 (response rate 83.3%).Te respondents can be described as street-level bureaucrats implementing the program in practice.A majority (72.3%) represented the purchaser organization or the central administration.Tey were mainly care managers, quality developers, or strategists.A smaller group (27.7%) represented the provider organizations and was mainly frstline managers, quality developers, or strategists (see Table 2).

4
Health & Social Care in the Community All participants indicated informed consent before flling out the questionnaire.Te Swedish Ethical Review Authority had no ethical objections to the project (Dnr 2020-01734).

Measures and Analyses.
All items are measured on fvepoint Likert scales (e.g., 1: strongly disagree and 5: strongly agree).Te data were processed in three main steps and then analysed with descriptive statistics and multiple linear regression.Tere is a debate about the use of parametric tests on Likert scale data, but the overall conclusion is that parametric tests are robust and thus can be used with Likert scales [49][50][51].
First, responses from the same organization were merged into one aggregate score, calculated as the mean of the individual scores.Te objective for this was to maintain the organization as the unit of analysis.We separated elder care, social services for people under 65, and support and services for people with disabilities into three diferent organizational units.Te data includes a total of 135 organizations with 1-3 respondents per organization.
Second, we conducted PCA with Varimax rotation according to the following procedures: (1) intercorrelations between items were checked to avoid multicollinearity.One of a pair of items with a very strong correlation (r > 0.90) and items with few correlations (r > 0.30) were removed [52].(2) We conducted the Kaiser-Meyer-Olkin measure of sampling adequacy (KMO > 0.50) and Bartlett's Test of Sphericity (p < 0.05) [52].(3) We looked at commonalities and removed items with low values (<0.30).( 4) Factors were extracted with Kaiser's criterion for eigenvalue (>1.0).( 5) Items with no high factor loading (>0.40) or equal loadings on more than one factor were removed [52].( 6) Reliability was tested with Cronbach's alpha (α) and we regarded α > 0.70 to be acceptable [52].If reliability was low but increased to an acceptable level without a certain item, that item was removed.( 7) Te extracted factors were interpreted and named based on the common theme of the included items.(8) Finally, index values were calculated as the mean of the included item scores.In accordance with these procedures, we conducted PCA on ten groups of items.Tis resulted in 15 index variables.Six items could not be included in any index due to low reliability and were analysed separately.However, one factor named "implementation in the provider organizations" was retained despite somewhat low reliability (α = 0.62).We judged this variable to have high face validity; it comprises four items that are theoretically connected and capture relevant aspects of implementation in the provider organizations.All variables, included items, and reliability are reported in Table 3.
Tird, a multiple imputation with fve iterations was carried out using the Markov chain Monte Carlo method [53].Te process included 26 variables, where the proportion of missing values ranged from 2.2% to 32.1%.In total, 14.2% of the values were missing, and 68.6% of all cases had missing values.
After these three initial steps to prepare the data, the variables were analysed with descriptive statistics and linear regression.Multicollinearity was assessed by checking pairwise correlations (r < 0.80) and the variance infation factor (VIF < 5.0) [52].Normal distribution of residuals was checked visually with histograms and normal probability plots.
Our outcome variable was routinization, which is the primary process of sustainability [14].However, routinization cannot be achieved without implementation, because there must be some program components to routinize [27].Terefore, we also measured the level of implementation, both in purchaser and provider organizations.Te predictors were arranged in four categories: (1) implementation strategy and process; (2) broad participation; (3) Health & Social Care in the Community  Te second and third categories belong to the organizational level, and the fourth category applies to the community level.However, we omitted factors pertaining to the implementation object, funding, and institutional context, since all study participants implemented the same program and were part of the same institutional context.Terefore, these factors did not vary between cases.Tree regression models were tested: a simple model, a full model, and a "thematic" model.Te simple model assessed the predictors as single variables.Te full model included all predictors.In the thematic model we divided the predictors into the four categories mentioned above.Regression analyses were performed on each category to determine the predictors' relative contribution to the model when controlled for the other variables in the same category.In addition, in all regression models, we adjusted for the level of implementation using two implementation variables.Tereby we could determine the predictors' efect on routinization when the variance explained by implementation was accounted for (see Figure 1).

Results
In Table 4, descriptive statistics are reported.Te results show that IBIC was implemented in the purchaser organization at 78 sites (57.8%) and in the provider organizations at 38 sites (28.1%) (score 4-5).Te program was implemented in both the purchaser and provider organizations at 29 sites (21.5%) (score 4-5 on both implementation variables).Overall, the program can be regarded as fully implemented routinized at 18 sites (13.3%) (score 4-5 on both implementation variables and routinization).Tus, after having been implemented for almost four years, the program still showed a low level of sustainability.
Tere is a fairly strong correlation between implementation and routinization; together the two implementation variables explain 15.4% of the variance in routinization (R 2 � 0.154, p < 0.001).However, our ambition is to explain routinization as the primary process of sustainability, and therefore the regression models adjusting for the level of implementation.
Te regression models are reported in Table 1.Model 1 is a simple model that shows signifcant correlations between routinization and most single predictors except adaptation to local conditions, provider collaboration, and the four predictors related to collaboration in the outer context.
Te column labelled "Model 2" shows the four "thematic" models, where all predictors in the same thematic category are included.Tus, we can determine the predictors' relative contribution to the model when controlled for the other predictors in the same category.Te frst category implementation strategy and process explains 39.0% of the variance in routinization (ΔR 2 = 0.390, p < 0.001), but open project strategy is the only predictor with a signifcant contribution to the model (β = 0.551, p � 0.000).Notably, rationally planned project (β = 0.063, p � 0.518) and efective project leaders (β = 0.065, p � 0.404) do not contribute any predictive value to the model.Te second category broad participation explains 22.3% of the variance in routinization (ΔR 2 = 0.223, p < 0.001) and three predictors make signifcant contributions to the model: frst-line manager commitment (β = 0.331, p � 0.006), user participation (β = 0.205, p � 0.010), and care manager participation (β = 0.177, p � 0.042).Te third category management support explains 39.2% of the variance in routinization (ΔR 2 = 0.392, p < 0.001) and three predictors make signifcant contributions to the model: management commitment (β = 0.416, p � 0.000), available resources (β = 0.210, p � 0.007), and informed decision to adopt the program (β = 0.155, p � 0.046).Finally, the fourth category collaboration in the outer context does not seem to have any efect on routinization (ΔR 2 = 0.030, p > 0.05).In summary, according to the four thematic models, open project strategy, management commitment, and frst-line manager commitment were the most important predictors of routinization (β > 0.30).
Overall, the regression models indicate that the most important factors were open project strategy, management commitment, user participation, frst-line manager commitment, and available resources.In the next section, we discuss those factors and refect on how the fragmented organizational setting may afect the factors' impact on sustainability.

Discussion
Tis paper focuses on routinization as the primary process of sustainability.However, we do not evaluate the program outcome and cannot draw conclusions about the potential benefts of IBIC.Tat said, the fndings indicate that routinization was positively related to factors pertaining to implementation strategy and process, broad participation, and management support, but not with variables pertaining to collaboration in the outer context.Our fndings provide empirical support to a dynamic, bottom-up approach to sustainability, but in combination with strong management support.

Implementation Strategy and Process. Te fndings
suggest that an open project strategy-where the implementation was coordinated with other change initiatives, had a long planning horizon, and was developed over time based on continuous feedback-was the most important factor for routinization.By far, it had the strongest explanatory value of the variables pertaining to implementation strategy and process.Other project-related factors, such as efective project leaders and rationally planned project, did not contribute with any additional explanatory value.Tese fndings are an important contribution to the sustainability research.A strong project leader can certainly be important by pushing for implementation, but our fndings suggest that a project leader plays less of a role in the routinization process (cf.[34,54]).
A relevant question is thus how an open project strategy should be understood.Te concept can be interpreted as three aspects of "openness" (cf.[55,56]).First, the implementation work is open in the sense that it is coordinated with other change initiatives in the organization.Tus, it is not narrowly limited to the specifc implementation task, but takes on a broader perspective on the organizational setting (cf.[57,58]).Second, a focus on long-term planning implies an openness towards the future after the initial  Health & Social Care in the Community implementation.Te implementation is not solely focused operational efciency and delivery of short-term goals, but also includes an early planning for sustaining new work practices (cf.[8,14,33]).Tird, the implementation work is open to adaptation through ongoing follow-ups and feedback.Tus, it is not tied to premade plans, but can be continuously modifed and developed [23,24,59].
We suggest that an open project strategy might be especially important in fragmented organizational settings.One reason for this is that fragmentation implies that the number of involved actors, possible relationships, and the level of complexity increases, which in turn may decrease the ability to plan and control the process.Tis motivates an open and adaptive strategy that enables the implementing actors to adjust and modify project activities and handle unforeseen events [56,60].
We found no connection, however, between routinization and adaptation to local conditions.Unlike open project strategy, this variable is not about adaptations of the implementation strategy and process, but about adapting the program itself to meet local needs and resources.In this respect, our fndings seem to deviate from previous research showing that programs that are adapted to local conditions are more likely to be sustained [6,28,29,61].One possible interpretation is that the adaptations made to IBIC were inadequate.Several researchers emphasize that new programs often need to be adapted to ft the mission, culture, and operating procedures of the organization [3,22], but these adaptations should not be ad hoc, but rather guided by systematic evaluations and feedback that provide reliable information about the program [8,23,62].However, our study shows that few sites (10.4%) used evaluation fndings to inform program adaptations (see Table 4).Terefore, implementing actors lacked systematic knowledge about the outcome and organizational ft of IBIC, which in turn created a weak foundation for well-informed adaptations of the program.

Broad Participation.
Te fndings suggest that user participation and frst-line manager commitment had a signifcant efect on routinization.Previous research shows that sustainability is more likely if relevant actors are involved in the implementation process, as it may provide valuable input and create a sense of ownership and commitment to the program [30,31,35].Te fact that user participation seemed to have such a strong impact on routinization is an interesting fnding.Tis means that the social service users were involved in the implementation work and that their experiences were used as inputs to the process.Previous studies often highlight participation from managers, staf, and community actors, but our fndings suggest that user participation can also be an important feature.Participation may make users more positive towards the program, and sustainability is more likely if changes are perceived as positive by users [1,3].User participation creates legitimacy for the program [22] and is likely to increase the motivation and commitment of both managers and staf.
In a fragmented organizational setting with numerous provider organizations, frst-line managers may hold key positions.As operations managers at the providers, they can potentially link top management strategies with staf experiences and perspectives.Tey can drive implementation and routinization forward and build commitment among their staf, but they can also work against changes they do not support [37].Aarons et al. [32] show that frst-line managers with a so-called transformational leadership style, which involves inspiring and motivating staf, predicted program sustainability.Passiveavoidant leadership predicted nonsustainability.

Management Support.
Management commitment and available resources were found to be two of the most important factors for routinization, but the factor labelled informed decision to adopt the program also had some explanatory value.Tis is in line with previous research, which shows that management commitment, ownership, and support are crucial for routinization and sustainability [5,8,33].
However, in a fragmented organizational setting it can be more demanding to create a shared sense of ownership and commitment to the program [30].IBIC was, for example, received diferently by purchaser and provider organizations.Te providers were not as involved and committed to the IBIC program as the purchasers, and both implementation and routinization were less successful on the provider side.Tis may indicate a need for strong management support, where managers can coordinate the implementation, establish a shared vision, and build widespread commitment to the program [8,32,34].Managers also have a responsibility to ensure that decisions to adopt new programs are based on actual and perceived needs, so that employees, users, and other stakeholders consider the implementation as meaningful [22,39].

Collaboration in the Outer Context.
Te fndings show that no variables connected to collaboration in the outer context were related to successful routinization.Active participation in national or local/regional IBIC networks had no predictive value, and collaboration between social service departments in the municipality even seemed to have a negative efect on routinization.
Tis is contradictory to previous studies indicating that collaborations and learning networks can facilitate sustainability [31,38,40].A possible explanation for the divergent fndings in this study is that collaboration and networks do not support sustainability per se.Networks need to be organized efectively because it is difcult to transfer the individual learning that takes place in networks back to the participants' home organizations.However, this transfer of knowledge from networks often tends to be ad hoc and relies on personal commitment [22,63,64].Nevertheless, all collaborations and networks require resources, increase complexity, and take time and focus from other activities.Terefore, if collaborations and networks do not afect the implementation in the home organizations, they can become a cost without added value.10 Health & Social Care in the Community 4.5.Limitations.Te study has some limitations.First, all data concerns the implementation of IBIC the Swedish social services and statistical generalizations beyond this case is not possible.However, we believe that the fndings are transferable to similar cases, but this transfer must be done with an understanding that both program attributes and context may afect sustainability.Second, the study is based on self-reported data.Literature reviews indicate that selfreports tend to overestimate sustainability [1].Nevertheless, deeply involved actors have unique insights that are important for understanding sustainability.Tird, the study is relatively small (n � 135) and had low representation from provider organizations (27.7%).However, the response rate was high (83.3%)and represented a large proportion of the municipalities implementing IBIC (72.3%).Fourth, for some variables, there were several missing values, especially on local politicians' commitment (32.1%), user participation (27.7%), and frst-line manager commitment (27.0%).Tese gaps were handled with multiple imputations, which van Ginkel et al. [65] argue is to be preferred over listwise and pairwise deletion.However, the fndings regarding these variables should be interpreted with caution.Lastly, the data as a whole was not suitable for PCA.Terefore, PCAs were made on subgroups of items related to the same theme.Te constructed index variables, however, had good face validity and reliability.

Conclusions
Te fndings suggest that fve factors-open project strategy, management commitment, user participation, frstline manager commitment, and available resources-were key factors for routinization as the primary process of sustainability.We argue that these factors may be especially important in fragmented organizational settings, where the recipient organizations are not single, unifed organizations, but are rather organizational clusters.Fragmentation results from organizational complexity and makes planning and control less predictable.Tus, in a fragmented setting, program sustainment is more likely if the implementation is guided by an open project strategy, where the implementing actors can adjust and modify project activities and handle unforeseen events based on continuous feedback and learning, long-term planning, and coordination with other change initiatives.Fragmentation also makes it harder to create a shared sense of ownership and commitment to the program.Tus, in a fragmented setting, sustainability may be promoted by broad participation throughout the process, including both frst-line managers and users.Tese fndings constitute empirical support for a dynamic, bottom-up approach.However, our fndings also suggest that this bottom-up approach should be combined with strong management support, whereby managers allocate sufcient resources, coordinate the fragmented implementation processes, establish a shared vision, and foster a widespread commitment to the program.Te fndings have practical implications for both local and national actors in planning, managing, and evaluating program implementation.Municipalities should consider adopting open project strategies, broad participation, and management support in their future implementation.Government agencies should design national programs that support these factors to increase the likelihood of sustaining new work practices.

Figure 1 :
Figure 1: Te tested model.Four predictor categories, routinization as the outcome variable, and two implementation variables as controls.