Details regarding the survey methodology
June 20, 2011
In February 2011, McKinsey & Company commissioned a survey of 1,329 U.S. private sector employers to measure their attitudes about healthcare reform. The opinion survey was paid for entirely by McKinsey as part of its routine, proprietary research.
We stand by the integrity and methodology of the survey.
The survey was not intended as a predictive economic analysis of the impact of the Affordable Care Act. Rather, it captured the attitudes of employers and provided an understanding of the factors that could influence decision making related to employee health benefits.
As such, our survey results are not comparable to the healthcare research and analysis conducted by others such as the Congressional Budget Office, RAND and the Urban Institute. Each of those studies employed economic modeling, not opinion surveys, and focused on the impact of healthcare reform on individuals, not employer attitudes.
Comparing the McKinsey survey to economic estimates, such as the CBO’s, is comparing apples to oranges. While the McKinsey Quarterly article about the survey cited CBO estimates, any comparison is not apt. We understand how the language in the article could lead the reader to think the research was a prediction, but it is not.
As the survey results reported, 30 percent of respondents who said their companies offered employer sponsored health insurance said they would “definitely” or “probably” drop coverage in the years following 2014, the year the Affordable Care Act takes full effect. (Nine percent said “definitely,” and 21 percent said “probably.”)
McKinsey commissioned Ipsos, the third largest market and opinion research company in the world, to field the online survey based on a questionnaire developed by McKinsey. Respondents were drawn from a panel of nearly 600,000 people maintained by Ipsos, not from McKinsey clients. The survey instrument was administered directly by Ipsos using their wholly-owned online panel.
Ipsos fully adheres to the CASRO Code of Standards and Ethics for Survey Research, the ESOMAR International Code of Marketing and Social Research and the American Association of Public Opinion Research (AAPOR). Ipsos has three different ISO certifications—ISO 9001 for Quality management systems, ISO 20252 for Market, opinion and social research and ISO 27001 for Information technology, security techniques and Information security management systems.
The respondents worked for companies that ranged in size from under 20 employees to more than 10,000 employees. They represent a cross section of employer size segments, industries and geographies. As is standard industry practice, the names of the participants and their companies were blinded. All results were weighted based on the U.S. Census data of firms by industry.
To qualify as a respondent, the individuals were asked a number of screener questions to ascertain that they played roles in choosing which benefits their companies provide to employees either as primary decision makers or having influence in the decision-making process. All sample surveys and polls may be subject to sources of error, including, but not limited to coverage error, self-selection bias and measurement error.
All respondents indicated they had either primary decision making authority (51%) over medical benefits or influence over the decision-making process regarding employee benefits at their company. The top five reported respondent occupations were: owner, head of human resources, head of procurement, CEO/president, vice president of compensation or benefits director/manager.
The first portion of the survey focused on the level of benefits offered by the employer today and any recent changes to benefits or intended changes pre-2014. Respondents were then asked their level of awareness of nine key provisions of the Affordable Care Act such as the individual and employer mandate, subsidies and guaranteed issue.
Next, the survey explored what respondents said they would do in 2014 and beyond. To make answers to those questions more informed, respondents received, in keeping with standard survey practice, factual information on the major provisions of the Affordable Care Act (e.g., exchange based subsidy levels by household income, guarantee issue provisions, medical underwriting and rating restrictions, and employer penalty levels). No information was offered on whether a particular provision or particular action would benefit or have an adverse impact on the employer or the employee. Nor was information provided on existing benefit rules, regulation, and taxes.
Respondents were then asked a set of questions on whether they thought they would maintain current coverage, offer alternative models such as defined contribution, offer mechanisms to provide employees the choice of accessing the future exchange market or cease provision of coverage. Respondents were also asked about their biggest concerns in ceasing coverage as well as how they would compensate employees if they did cease to provide coverage.
As noted, the survey only captured current attitudes. Employers’ future actions will be determined by many considerations. Among them: Medical cost inflation, the details of new state health insurance exchanges, employee attitudes toward compensation and benefits, a company’s ability to attract and retain talent, actions taken by competitors and the state of the economy. These are just some of the many factors likely to influence employer behavior in the future.
To reiterate, the survey reported in the McKinsey Quarterly was not an economic forecast, but rather a measure of attitudes intended to understand the factors involved in employer decision making regarding employee benefits.
Please email comments or questions to: US_employer_healthcare_survey@mckinsey.com