Statement on Healthcare Checklists Issue
The American Society for Quality takes exception to a recent ruling by the Office for Human Research Protections that is having a potentially chilling effect on healthcare quality improvement and patient safety initiatives being carried out at healthcare institutions across the country.
The ruling halted data collection on a highly successful Johns Hopkins-initiated program at use in Michigan hospitals that involves the use of simple checklists aimed at reducing infections in hospital intensive care units. OHRP characterized the initiative as research on human subjects that is subject to the same safeguards taken when patients are given experimental drugs in clinical trials or subjected to experimental procedures. Those safeguards include obtaining written permission from each patient and getting approval from the institutional review board (IRB) that oversees clinical research in the institution.
Because we feel strongly that the ruling reflects a basic misunderstanding of the nature of contemporary healthcare quality improvement practice, we offer the following recommendations for actions that we believe the Department of Health and Human Services must take in order to rectify a situation that is causing consternation among the nation’s healthcare institutions:
- Develop a clear set of guidelines to determine when the OHRP’s regulatory protections are required and when they are not.
OHRP has acknowledged that it does not have such guidelines but is working to develop them. ASQ is willing to assist OHRP in that task, and we believe that healthcare quality practitioners and their institutions must have input into the development of that guidance.
ASQ believes that OHRP has mischaracterized quality improvement efforts such as the Johns Hopkins and Michigan programs, in terms of the level of patient protection required and in terms of who is the subject of these initiatives. Having a clear set of guidelines will be a great help to the many healthcare institutions that are carrying out quality improvement and patient safety initiatives every day. And it will help allay their apprehensions that have been raised as a result of this ruling.
- OHRP must understand and acknowledge the difference between healthcare process improvement initiatives and experimental research on patients.
While caution in human research is a good thing, and IRB oversight of such research is a good thing, confusing process improvement or other patient care matters with the type of human research necessitating an IRB oversight approach is incorrect. An organized and systematic approach to the use of checklists, as in the Michigan hospitals case, may be a type of research, but it is not human research of the type warranting IRB oversight.
Healthcare is a service to human beings, and patients experience healthcare individually. But healthcare today can be characterized as a complex, open, interdependent system that involves the interaction of multiple caregivers and systems. We know from the human factors discipline that human memory is fallible and the likelihood of error is dependent on variables such as complexity. Highly complex systems such as modern healthcare are prone to error, which is why checklists can be such helpful tools to ensure and promote best practice. The use of checklists in healthcare is fundamental and should become more widespread. Human research means you are doing something to someone. But assessing the efficacy of checklists does nothing to anyone other than the caregivers employing the checklists. It is a process improvement initiative, and it deserves to be quantified. OHRP, which has acknowledged it has no way of judging checklist-based research, has erroneously placed these efforts within the domain of experimental research on patients.
Consider the following example that illustrates how a typical process improvement initiative might be carried out.
If we count how many patients go through a particular hospital operating room in a month and how many incur infections, we get a ratio. If we do the same count for a month after using a check list of recommended practices and we get a significantly reduced ratio, and we get that reduced infection ratio consistently over time, then we have evidence that this intervention on the process helps reduce defects in the process that cause harm to patients. The research is done on measuring the total outcomes of the process—not on measuring individual patients. The results of this initiative are reduced variation, delivering safe and effective care, and having a more efficient organizational process that yields decreased cost and improved safety for the participating hospitals.
When measurement of the outcomes for implementing a process is linked to human subject research, before imposing the rigor of an IRB, it should be established that there is actual involvement of and risk to the patient.
Heuristics and so-called rules of thumb are also checklists that play an important role in our perception of risk and safety. Organizationally, the challenge remains how to transfer individual learning into organizational learning. One prevalent observation about healthcare is that we rarely learn from past experience on the organizational level. Quality improvement initiatives carried out across multiple institutions with collaboration and sharing of data (such as the Michigan hospitals checklists initiative) are an attempt to capture this learning and overcome this shortcoming.
- Existing initiatives that have been adversely affected by the ruling must be allowed to proceed.
While guidelines are being developed, the Michigan hospitals initiative is at a standstill, and other institutions are left to wonder whether their efforts might also be at risk. ASQ asks the Department of Health and Human Services to prevail upon OHRP to suspend or reverse its ruling and permit the Johns Hopkins and Michigan initiative—and others like it—to proceed unencumbered by the regulatory requirements that the ruling imposes.
We urge the Department of Health and Human Services to act on these recommendations in order to forestall adverse effects of the OHRP ruling on quality improvement activities, which include:
- inappropriately forcing quality improvement activities into an experimental clinical research mode;
- erecting unnecessary hurdles that will slow down the pace of quality improvement activity;
- deterring observational studies and collaborations that are routinely used to improve care and enhance patient safety.
After studying the OHRP ruling, the ASQ Healthcare Division came to this conclusion:
“Applying regulations that appropriately protect human subjects in true experimental research is ethical and proper. Applying them to the observational research and process improvement methods forming the foundation of hospital or healthcare epidemiology effectively stifles increasing the effectiveness of infection surveillance and control … threatens operation of regional and national surveillance collaborations … and sets bureaucratic paperwork burdens incompatible with the efficient, adaptive, rapid cycle protocol change pace common in a continuous quality improvement process.”
We have no doubt that the OHRP ruling was made in good faith in order to protect patients, although it was made using an inappropriate set of criteria. In the interest of avoiding future misunderstandings, ASQ is ready to offer its expertise and insights in the areas of measurement and data-driven decision making in the development of criteria that will clarify the distinctions between process improvement efforts and experimental research on patients.
