A challenge facing immunization registries is developing measures of childhood immunization coverage that contain more information for setting policy than present vaccine series up-to-date (UTD) rates. This study combined milestone analysis with provider encounter data to determine when children either do not receive indicated immunizations during medical encounters or fail to visit providers. Milestone analysis measures immunization status at key times between birth and age 2, when recommended immunizations first become late. The immunization status of a large population of children in the Oregon ALERT immunization registry and in the Oregon Health Plan was tracked across milestone ages. Findings indicate that the majority of children went back and forth with regard to having complete age-appropriate immunizations over time. We also found that immunization UTD rates when used alone are biased towards relating non-UTD status to a lack of visits to providers, instead of to provider visits on which recommended immunizations are not given.
A challenge to increasing early childhood immunizations on a state or local level is the limited ability of standard summary measures of up-to-date (UTD) rates to identify barriers to improvement. Typically, early childhood UTD rates are based on the number of doses of recommended vaccines, by individual antigen or in total, that a cohort of children receive by either a fixed age or a fixed date of assessment [
One alternative method of evaluation is to consider age-appropriate vaccinations. A substantial body of prior work exists comparing summary UTD measures against more specific assessments either for complete antigen series or combinations when actually due without considering late catch-up [
One perspective on age-appropriate immunizations is to track children’s progress through immunization milestones between birth and age 2 [
This study provides an example of using a milestone approach to assess a specific population and their progression through early childhood immunizations, where both payor-administrative and state-level immunization information system (IIS) data are available. Combining children’s healthcare encounter information from payer records with their immunization records gives a more accurate assessment at each milestone age of the effect of missed opportunities and missed visits on age-appropriate immunizations and overall immunization rates.
The study population consisted of a birth cohort of Oregon children enrolled in the Oregon Health Plan (OHP) and whose immunization records were in the Oregon ALERT Immunization Information System (ALERT IIS). ALERT IIS immunization records were merged with provider encounter records from the OHP for this population. The ALERT IIS is a statewide immunization registry which receives immunization records from 97% of Oregon private healthcare providers and 100% of the immunization records from public providers. The OHP is Oregon’s public healthcare plan that provides healthcare coverage for children in families living below 185% of the federal poverty level and covers both those with traditional Medicaid eligibility as well as an expanded State Children’s Health Insurance Program (SCHIP) population. The majority of OHP-enrolled children are placed in commercially available managed care plans. Immunization records for OHP children are available both from OHP collected records for billing and encounters as well as from direct provider record submissions to ALERT.
For this study, ALERT IIS records and OHP encounter records were selected from their respective data systems for children born in 2005. Immunization records and encounter data were restricted to those received through the child’s 24th month of age. Children’s records were merged across the two data systems based on the child’s name, date of birth, and county of residence. The matching process was based on the observation that within the 2005 Oregon birth cohort, a combination of name and date of birth was over 99.9% unique, with almost all exceptions resolving with the inclusion of residence. This high-probability matching process was selected over the usual process ALERT uses to incorporate OHP and other administrative data, wherein a hard-match is required also on additional information such as address or phone number.
Immunization records were selected for the six vaccines (including combination vaccines) included in the recommended “4:3:1:3:3:1” series. The 4:3:1:3:3:1 series consists of 4 diphtheria, tetanus toxoid, and acellular pertussis (DTaP); 3 poliovirus (IPV); 1 measles, mumps, and rubella (MMR), 3
To ensure that records of encounters would be available to compare with immunizations at all of the milestone ages, children with limited enrollment or nonenrollment at key ages were excluded from the final analysis dataset. The study population was restricted to children enrolled in OHP within 30 days of birth, with a cumulative total of at least 365 days of enrollment by age 2, and continuous enrollment across the key period of 15 to 18 months, when the 4th dose of DTaP is due. Because OHP enrollment generally occurs in 12-month blocks, the majority of children meeting the above requirements were also continuously enrolled through their second birthday. Children born outside Oregon were excluded since possibly both early encounters and immunizations would not be reported to ALERT or OHP. Children with only a birth dose of hepatitis
The OHP requires health plans and providers to submit detailed encounter records on enrolled children for all services received. For OHP-enrolled children, a subset of vaccination-eligible encounters was created from all encounters, based on a review of ICD-9 and CPT coding in OHP encounter records. A vaccination-eligible encounter was defined as an encounter occurring in a nonemergent or noninpatient setting with a medical provider, and with either a CPT procedure code indicating that routine care or evaluation was performed, or an ICD-9 diagnostic code indicating that the purpose of the encounter was consistent with routine care and immunization evaluation. These criteria were used to identify not only visits that providers would define as well-child visits, but also other visits during which immunizations could have been given, and include nonemergent “sick” visits. In a few cases, ALERT had a record of an immunization visit for which there was no matching OHP encounter record. This was usually found to reflect free vaccinations without administration fees at sites outside of those normally reporting to OHP, such as school clinics, and some public health departments. These visits were also counted as vaccination-eligible encounters. Also if CPT codes for vaccine administration were found in OHP data without other evidence of a shot-eligible encounter, these were taken as indicating that a vaccine encounter occurred.
The definitions of milestone periods were taken from Luman and Chu [
Immunization Encounter Classification.
Had immunizations by milestone? | |||
Had provider encounters? | Yes, UTD | Yes, Non-UTD | No |
Yes | Immunization Visit | Missed Opportunity | Missed Opportunity |
No | — | — | Missed Visit |
Where UTD is up to date status for ACIP recommended immunizations in the DTaP, HepB, Hib, Polio, and Varicella series by each milestone age.
At each milestone age, immunization status was compared with the immunization status at the previous milestone age to determine whether children had remained UTD, remained non-UTD, fallen behind due to a missed visit, fallen behind due to a missed opportunity, or caught up with the immunization schedule. Children remained UTD if they were UTD at the prior milestone age and UTD at the current milestone age. Children remained non-UTD if they were non-UTD at the prior milestone age and non-UTD at the current milestone age. This classification is presented in Table
Immunization Status Categorization.
At prior milestone | ||
At current milestone | UTD | Non-UTD |
Up to date (UTD) | Remain UTD | Catch Up |
Non-UTD | Fall Behind | Stay Non-UTD |
UTD is up to date status for ACIP recommended immunizations in the DTaP, HepB, Hib, Polio, and Varicella series.
Children fell behind due to a missed visit if they were UTD at the prior milestone age, non-UTD at the current milestone age, and had no record of a valid healthcare visit or immunization record during the period in between. Children fell behind due to a missed opportunity if they were UTD at the prior milestone age, non-UTD at the current milestone age, and had a vaccination-eligible healthcare visit during the period in between. Children caught up if they were non-UTD at the prior milestone age and UTD at the current milestone age. An exception to the missed opportunity calculation is for the 16-month milestone, which includes the first MMR vaccine. The MMR is not valid before 12 months of age, so encounters between 7 months and 12 months were not counted as potential missed opportunities.
As a check on the completeness of immunization visits represented by this merged dataset, a Lincoln-Peterson capture-recapture method was used to estimate the percentage of immunization visits for the study population not captured by the ALERT IIS either by provider records or by OHP administrative records. The total number of immunization visits, both captured and uncaptured, was estimated by
The principal assessment tool of this study is a time-based progression of young children across milestones and age-appropriate immunizations, presented in a novel form for easier depiction of change between milestones. The data by milestone are presented for whether children were complete on age-appropriate immunizations along with categories for catching up and falling behind by milestone, and by missed opportunities versus missed visits for non-UTD children. Finally, a comparison of age-appropriate milestone results is made to a summary UTD measure for the 4:3:1:3:3:1 immunization series assessed at 24 to 35 months of age.
Of 20,411 children born in 2005 who were enrolled in the Oregon Health Plan for some period of time, 13,199 met the requirements to be counted among the study population. The reasons and scope of exclusions are presented in Table
Population Size and Exclusions.
OHP children in ALERT: | 20,411 |
---|---|
Exclusions (in order applied) | |
Not born in Oregon | 1,004 |
1,123 | |
583 | |
Not enrolled within 30 days of birth | 345 |
Not enrolled after 1 year of age | 2,884 |
Not enrolled through 15–18-month period | 1,009 |
No immunizations reported post birth | 264 |
Total exclusions | 7,212 |
Total study population | 13,199 |
OHP is the Oregon Health Plan ALERT is the ALERT Immunization Information System.
The count of total immunization visits across both datasets was 74,919. The capture-recapture estimate of total immunization visits for the study population was 76,087. Thus the dataset appears relatively complete for all immunization visits of the study population, with the combination of ALERT IIS provider reports and OHP administrative data capturing 98.4% of estimated immunization visits among the study population.
As shown in Figure
Children by Count of Milestones with All Age-Appropriate Immunizations.
Rates of completeness of age-appropriate immunizations per the ACIP schedule varied among the milestone ages, from a high of 82.4% at 3 months of age to a low of 52.5% at 19 months of age. For the final milestone at 24-months, no further vaccinations were due, and the final 24 month completion rate for the study population was 68.6%, with 16.1% catching up from the prior 19-month milestone. The pattern of completion, falling behind, and catching up by milestone period is presented in Figure
Immunization Completeness At Milestone Ages (for Two Year Olds in 2007,
While 17.6% of the study population had fallen behind by the 3-month milestone, representing a late start on immunizations, the most salient episode of falling behind occurred at the 5-month milestone, where 21.0% of children fell behind. In this analysis, children fell behind at all milestone ages at which new immunizations were added. The 19-month milestone adds 4 antigens beyond the 16-month milestone, including the fourth DTaP and varicella; and the risk of falling behind between these milestones is calculated from Table
Another approach to interpreting the reasons for children falling behind is to further examine missed visits, missed opportunities, and vaccination visits at each milestone for those who were not complete. Figure
Provider Encounters by Milestones for Children Lacking Complete Age-Appropriate Immunizations.
The percentage of noncomplete children per milestone with provider encounters on which some shots were received (vaccination visits) ranged from a high of 50.8% prior to the 3-month milestone to a low of 15.2% before the 24-month milestone. The percentage of children with provider encounters and no shots, and without vaccination visits, in each period is a measure of the amount by which immunization-record-only data would misclassify missed opportunities as missed visits. This potential misclassified percentage of noncomplete children having vaccination-eligible encounters with no reported vaccinations ranged from a high of 44.8% at the 24-month milestone, to a low of 21.4% at the 5-month milestone. At the 3-month milestone, noncomplete children were evenly divided between those with no encounters on record during the period (31.4%), those with encounters but no vaccinations (34.1%), and those with encounters during which some vaccinations were given (34.5%).
As a final analysis, the results by milestone for encounters and completeness were stratified by children’s status on a summary UTD measure for having all shots in a 4:3:1:3:3:1 series by age 24 to 35 months, and using a fixed date of assessment. Overall 77.8% of the study population were UTD by 24 to 35 months for the 4:3:1:3:3:1 series. The comparison across milestones for children who were UTD is presented in Figure
Milestone Completeness and Provider Encounters for Children UTD for 4:3:1:3:3:1 Series by 24 to 35 Months.
Of the children who were UTD by the date of assessment, only 68% were complete for age-appropriate immunizations by the 19-month milestone, 72% were complete by the 16-month milestone, 67% were complete at the 7-month milestone, and 71% were complete at the 5-month milestone. The majority of those who were not complete at any milestone, with the exception of the 24-month milestone, also had encounters with providers on which some shots were given.
Figure
Milestone Completeness and Provider Encounters for Children Non-UTD for 4:3:1:3:3:1 Series by 24 to 35 Months.
Children not UTD at 24–35 months were also not complete for age-appropriate immunizations at 19 and 24 months by definition of which shots were required at these milestones. Overall in Figure
The reality of children’s immunizations in the present study population is a story of falling behind and catching up with recommended immunizations. This finding is consistent with prior application of a milestone age approach [
In this study, the majority of children who were not catching up at each milestone were having encounters with providers. These encounters were potentially ones in which missing vaccinations could be administered; however, converting these missed opportunities to vaccination visits may be difficult. Provider reluctance to administer vaccinations during sick or other nonroutine visits is a known barrier to improving immunization rates, and may be difficult to change [
A strength of the present study is the combination of administrative data on all encounters with immunization records reported separately to the ALERT IIS. This approach could potentially serve as a standard for the evaluation of immunizations given to health plan participants and public populations in areas that have strong immunization information systems such as ALERT. A similar approach by Dombkowski et al. [
From the perspective of a state immunization program with concerns for improving immunization rates, the development of a roadmap showing where and how children are falling behind is invaluable for setting policy. While national measurements such as the NIS can identify variations in rates between states, state-level programs have been left on their own to identify what in-state factors are affecting their rates [
The concept of milestone ages and the charting of children’s progress through the milestones, as advanced by Luman and Chu [
The present study is representative only of a single state population, and of children enrolled through the Oregon Health Plan. Because children in the OHP are generally enrolled in the same health plans, with the same networks of providers and benefits, as privately insured children, their encounters are potentially similar to the wider state population. Overall in 2007, the OIP estimated that among all OHP-enrolled 2-year-olds, the UTD rate for a 4:3:1:3:3:1 series was 75.2%, as compared to 72.9% among non-OHP-enrolled children. However, the present study population also reflects a group with stable, long-term enrollment in OHP. Children with short-term enrollment or who disenrolled after age 1 are not represented and may have substantially different patterns of falling behind and catching up to recommended immunizations. Also it is expected that individual health plans under the OHP are a significant factor in the receipt of age-appropriate immunizations; however, this information was not included in the present analysis dataset.
Another limitation on the present results is that the definition of encounters was deliberately set broadly, to reflect any encounters in which vaccinations could have been delivered as opposed to well-child visits, during which immunization screening should be routine. As such, the possibility of raising immunization rates by converting all missed opportunities here should be taken as an upper figure to what is possible. Institutional, scheduling, reimbursement and parental acceptance are all potential factors on what proportion of encounters without vaccinations could incorporate immunization screening.
Also, encounters were not stratified by type of provider or principal reason for each encounter. While some research suggests that provider type is not a key factor for immunization performance when the volume of well-child visits is taken account [
The milestone approach to evaluating early childhood immunizations provides a useful perspective for understanding the time-based progression of children through immunization periods. However, the results of this study warrant some caution in the use of immunization record data only in assessing failure to have age-appropriate immunizations because of the chance of misclassification of missed opportunities by providers as missed visits by parents. Nevertheless, for local assessment by public agencies or health plans, and for the design of interventions to improve immunization rates, looking at the patterns by which children fall behind or catch up on immunizations at milestone periods is a valuable next step beyond the count of vaccine doses received by age two.
This study was in part funded under CDC Grant no.280540/09. The authors also wish to thank Diana Bartlett, Laura Pabst, Jim Gaudino, Holly Groom, and Elizabeth Luman for their work in review of this research.