Assessing the Medicare Advantage Star Ratings

Every October since 2009, the Centers for Medicare and Medicaid Services (CMS) has released comprehensive data on Medicare Advantage (MA) health plan performance through its Star Ratings program. The program's goals are to incentivize health insurers to improve their MA plans and encourage consumers to enroll in high-quality plans. To investigate whether these goals are being met, we analyzed CMS's data for 2018 and previous years.

Our key findings:

• On average, the plans' scores have risen substantially, both overall and on the 22 individual quality measures that have been used consistently since the program's inception.¹Overall scores were considered only for MA plans that include prescription drug benefits. However, these plans account for the vast majority of all MA contracts.
• The percentage of MA members enrolled in high-performing plans (those with 4 or more Stars) has also increased significantly.
• Average Star Ratings have improved even though the standards used to assess plan performance have been tightened in recent years, which suggests that plan quality is rising faster than the numerical scores indicate.

In this paper, we present our analyses of MA plan performance on Star Ratings. In addition, we briefly compare the financial performance of MA plans against fee-for-service (FFS) coverage, and discuss two factors that may be confounding Star scores. We then offer recommendations for how insurers can achieve additional improvements in Star Ratings and refine their strategies before the next ratings release in Fall 2019.

Overall ratings have risen

In 2011, CMS introduced overall scores for MA Prescription Drug (MA-PD) plans, which account for the majority of all MA contracts. Between 2011and 2018, the national average of the enrollment-weighted Star Ratingsforall MA-PD plans increased from 3.18 to 4.06, which reflects the plans' overall improvement in performance even after membership shifts and changes to scoring criteria were accounted for (Exhibit 1).²Because of data reporting constraints, the ratings calculated in a given year reflect the performance of MA plans two years earlier (e.g., 2018 Star Ratings, published in the fall of 2017, reflect data from 2016).

Since 2011, overall Medicare enrollment has grown by 17%,³CMS. Medicare Enrollment Dashboard. Data as of January 2018. and MA penetration⁴The calculation of MA penetration includes MA, MA-PD, special needs plans, and other MA plans, such as cost plans and demonstration plans. as a share of that enrollment has increased from 23% to 33%.⁵Jacobson G et al. Medicare Advantage 2017 spotlight: Enrollment market update. Kaiser Family Foundation. June 6, 2017. In addition, the percentage of MA-PD members in plans with 4 or more Stars has risen from 24% to 73% (Exhibit 2). MA program enrollment reached 18.5 million in 2017, compared with 9.7 million in 2008,⁶MedPAC. A Data Book: Health Care Spending and the Medicare Program. Chapter 9. June 2017. and the number of enrollees receiving the extra benefits associated with quality bonuses (e.g., reduced premiums, additional vision or dental coverage, and other additional benefits) swelled from 2.8 million people in 2010 to 13.2 million in 2017.

The overall scores are based on the scores assigned to specific measures, many of which have been modified since the program's inception.⁷The appendix contains a complete list of the 2018 metrics by category. However, 22 of the measures have been used consistently since 2009. (We call this set the consistent measures.) After calculating the enrollment-weighted average scores for each of the consistent measures, we found that all but two have risen over time (Exhibit 3).

Digging deeper into the ratings

Since the Star Ratings program's inception, the financial incentives offered to MA carriers have varied, and CMS has periodically altered its ratings methodology (e.g., by raising the cut points for various scores). To investigate how these factors may have influenced Star scores, we conducted three sets of analyses, using data for the 22 consistent measures.

Impact of financial incentives

To investigate the possible impact of financial incentives on plan performance, we looked at the average scores for the consistent measures before, during, and after CMS introduced its quality bonus payment (QBP) demonstration. (For more details about the financial incentives, see the sidebar below.) We used the following time periods:

• Years 2009–11, before financial incentives for quality were announced
• Years 2012–13, after the financial incentives were announced, until the end of the QBP demonstration (i.e., during the second and third years of thedemonstration)
• Years 2014–18, after the demonstration ended and the QBPs defined in the Affordable Care Act were reinstated

Our results show that average overall scores for the consistent measures have risen 16.7% since 2009. The rate of improvement differed among the three periods, however (Exhibit 4).

Impact on measure categories

We dug even deeper to analyze the rateof improvement among the four categories of measures (as defined by CMS). We found that, between 2009 and 2018, average aggregate raw scores for the consistent measures rose in three categories: outcomes, process, and access (Exhibit 5).⁸Based on changes in enrollment-weighted average raw performance scores. However, there was little change in scoring for patient experience.

In the three categories with increased scores, the rate of improvement differed by time period. The greatest improvement in access metrics occurred in 2012 and 2013. The largest increases in outcome and process scores occurred between 2014 and 2018. These findings suggest that investments in performance may take time to pay off.

Impact of cut-point changes

Throughout the program's history, CMS has periodically updated the performance levels required to achieve the rating for each measure.⁹CMS. Medicare health & drug plan quality and plan performance ratings 2015: Part C & Part D technical notes. April 10, 2015.¹⁰MedPAC. Report to the Congress: Medicare Payment Policy. Chapter 12. March 2016. As a result, performance that might have achieved 4 Stars one year might be given only 3 Stars the next year.¹¹For most measures, CMS now defines the Star cut points based on the natural gaps that arise among clusters of plans based on the distribution of scores. (CMS. Trends in Part C & D Star Rating measure cut points. November 8,2016.) We therefore sought to understand the extent to which cut-point changes such as these were affecting Star Ratings.

We found that the cut points needed to achieve 4 Stars had increased for 8 of the 14 consistent measures that are scored using a single metric. Despite the increasing difficulty of getting a score of 4 Stars on those measures, the enrollment-weighted average Star Ratings increased for all but "improving or maintaining physical health" and "reducing the risk of falling" (Exhibit 6).

To further explore the impact of tightening standards, we applied the 2009 cut points for all of the 22 consistent measures to the contracts covered by the 2018 ratings to see how many MA enrollees would be in plans with at least 4 Stars if none of the cut points had changed. The result: about 1.2 million more beneficiaries (about 6.5% of total MA beneficiaries) would have been covered by plans with 4 or more Stars.¹²This analysis calculated the total number of beneficiaries in 4+ Star plans on a given metric using 2018 raw data and 2009 cut points. For a view of how this analysis worked for one example measure, see the appendix.

These results indicate that many MA enrollees are receiving better care today. For example, approximately 76% of beneficiaries aged 50–75 received an appropriate screening for colorectal cancer in 2016, whereas only 54% of such enrollees were screened in 2007. It is possible, however, that some of the increases may simply reflect better reporting and documentation. (Again, more details can be found in the sidebar below.) Furthermore, whether increased screening rates are indicative of higher quality of care is a matter of some debate.

MA plans have also become more efficient

In a separate analysis, we found that MA plans have become more efficient than FFS coverage at providing Parts A and B benefits (Exhibit 7). Currently, the MA plans deliver Part A and B benefits at 90% of the costs of FFS coverage (or 92% if differences in risk coding are fully accounted for), compared with 102% in 2009.

Even though changes in bidding benchmarks were phased in from 2012 to 2016, MA plans continued to manage costs and bid below the declining benchmarks.¹³Congressional Research Service. Medicare Provisions in the Patient Protection and Affordable Care Act (PPafford): Summary and Timeline. June 30, 2010. This finding indicates that, in addition to the quality improvements previously discussed, MA plans have also improved their efficiency in recent years.

Confounding factors

The Medicare Payment Advisory Commission and others have identified several possible isues with the Star Ratings program, including the potential for plan consolidation to "erode the validity of the Star Ratings system as a measure of plan performance in a given area."¹⁴MedPAC. Report to the Congress: Medicare Payment Policy. Chapter13. March 2017. The quality of data collection has also been raised as a concern. We address both issues below.

Plan consolidation

Star Ratings are calculated at the contract level and apply to all plans that fall under the contract, regardless of quality differences between the individual plans. A plan that gets consolidated into an existing contract receives a Star Rating and quality bonus based on the score of the existing contract, even if its own score had been lower.

To understand the impact of plan consolidation, we looked at the 271 plans with less than 4-Star scores that were consolidated into other contracts between 2017 and 2018. A total of 1.8 million people had been enrolled in these lower-quality plans; 76% of them were covered by 4-Star plans in the 2018 ratings simply because of consolidation (Exhibit 8).

Thus, plan consolidation does have an impact on average enrollment-weighted Star Ratings, but it is unlikely to be the only factor contributing to the rising scores. (For more details on why this is the case, see the sidebar below.) We analyzed the seven contracts from the 2015; ratings year that had plans from the previous year folded into them. Of the seven, four were given 4 Stars in 2015; the remainder received 3 or 3.5 Stars. By 2018, three of the seven contracts had improved their ratings, three maintained their ratings, and one experienced a decline (Exhibit 9). This finding suggests that consolidation will not provide a long-term Star Ratings boost unless the underlying performance of the consolidated plans also improves—investments in quality must continue if the quality of the underlying plans is to continue to rise.

Caveat: Given the increased rate of consolidation in the past few years and the two-year gap between plan performance and ratings assignments, it remains to be seen if the same pattern will hold true for plans that have been consolidated since 2016. Furthermore, CMS has noted that the practice of "masking low quality plans under higher rated surviving contracts" confuses beneficiaries, and it recently proposed a new rule, under which Star Ratings for surviving contracts would be calculated based on the average enrollment-weighted scores of the consolidated contracts.¹⁵CMS. Medicare program; Contract year 2019 policy and technical changes to the Medicare Advantage, Medicare Cost Plan, Medicare Fee-for-service, the Medicare Prescription Drug Benefit programs, and the PACE program. November 28, 2017.

Better data collection

For some measures, the raw scores may now be higher than they were historically because many MA plans have improved their ability to measure and track the factors that influence those scores (as well as their overall payments and Star Ratings). However, our research suggests that the recent score improvements are likely to be the result of both better documentation and better performance—e.g., actual increases in the rate of screenings or other interventions. (For more information about changes in data collection, see the sidebar below.)

Options for further improvement

Overall, these results show that, since the advent of the QBP system, MA plan quality (as measured by Star Ratings) has improved while controlling or reducing costs. Experience with Star Ratings indicates that health insurers can adjust and improve their performance over time based on incentives built into CMS-determined metrics.

Going forward, health insurers will need to continue investing in their Stars programs to maintain their current standard of quality and meet ongoing changes in cut points across measures. Insurers can improve their scores in two key areas:

Outcome and process measures. Insurers should continue to strive for excellence in these metrics—as cut points in these areas evolve, plans will have to keep up to stay competitive. Health Effectiveness Data and Information (HEDIS) scores affect both categories and thus should be an ongoing focal point for improvement. The improvements could require long-term investments and accountability across several areas of the enterprise, but insurers should see payoffs from increased performance within three to five years.¹⁶A rule finalized by CMS in April 2018 establishes that substantive changes to the Star Ratings program will go through rule-making rather than subregulatory updates in the annual call letter. This change underscores the importance of considering long-term investments, given that Stars program changes could take longer to implement under the new regulation.

Patient experience and access scores. Many insurers will also need to increase their focus on patient-experience and access measures, since scores for these measures have seen less improvement than other domains have. Scores from the Consumer Assessment of Healthcare Providers and Systems (CAHPS) surveys could be a good place to start—many are in the patient-experience category and have historically been less of a focus for insurers. Concerted efforts to improve in this area could enable insurers to outperform competitors and provide a source of product differentiation in the form of enhanced member experience.

Some measures will remain difficult to achieve 4 Stars on, given cut-point evolutions, but continual improvements over time through dedicated investments, resources, and accountability could keep high-performing plans ahead of the competition. Health insurers can achieve such improvements in a few ways (see sidebar, "Background: Evolution of the MA Star Ratings program).

Improved digitization and analytics

Improvements in these areas could help health insurers engage beneficiaries and physicians, reduce the administrative burden of managing care, and strengthen underlying program performance. For example, insurers could develop and promote use of these tools:

• Better consumer digital interfaces could provide pricing transparency and scheduling information or allow consumers to more directly and easily access care.
• Advanced analytics could identify and proactively close member care gaps, including better physician web portals and digital interfaces that provide strategies to address these issues.

Better provider engagement

By engaging more effectively with providers, health insurers could further improve patient experience and outcomes. Investments in patient-experience improvements have traditionally been low, given the difficulty of influencing physicians. Nevertheless, investments in this area represent an opportunity for further improvements in underlying Stars performance. Payers could achieve this in a few ways:

• Invest in opportunities to change physician opinion and incentives to encourage and empower behaviors that could improve quality and Stars performance. For example, physicians could be included in programs that provide competitive incentives to close the care gaps.
• Host educational events to educate physicians on the Stars program, including roundtables in which best-practice physicians share their experiences.
• Develop physician- and office-based materials to encourage performance on Stars measures. For example, insurers could provide physicians with buttons that say, "Ask me about your flu shot" or provide training for office staff to encourage patients to inquire about critical prevention measures.

More effective data collection and coding

Improvements here could help insurers identify new sources of value within the existing population:

• Natural language processing or other artificial intelligence mechanisms could help minimize coding errors.
• Better tracking of Stars performance could help identify and address areas of underperformance, which could also provide incremental benefits by improving risk revenue.