«Executive Summary Prepared for the Patient Safety Programme of the World Health Organization Donna O. Farley, PhD, MPH Evaluation Consultant August ...»
Evaluation of the WHO Patient Safety
Solutions Aides Memoir
Prepared for the Patient Safety Programme
of the World Health Organization
Donna O. Farley, PhD, MPH
August 31, 2011
Table of Contents
Evaluation of the WHO Patient Safety Solutions Aides Memoir
Table of Contents
List of Tables
The Inaugural Set of Patient Safety Solutions Aides Memoir
Evaluation Aims and Approach
Data Collection Methods
Analysis of Data Collected
Highlights of Findings: the Patient Safety Solutions Products
Highlights of Findings: Dissemination and Impacts of The Patient Safety Solutions Aides Memoir
Summary and Suggestions for Action
i List of Figures Figure 1. Logic Model of the WHO Patient Safety Solutions Development and Dissemination
List of Tables Table 1 Methods for Collecting Data To Address Each Set of Evaluation Questions.................. 4 Table 2 Distribution of Stakeholder Survey Respondents by WHO Region
Table 3 Distribution of Stakeholder Survey Respondents by Type of Organization
Table 4 Suggestions by the Survey Respondents for Improvements to the Patient Safety Solutions
ii Introduction As described on the World Health Organization (WHO) patient safety website,1 “Patient Safety Solutions Aides memoir describe “…system designs or interventions that demonstrate the ability to prevent or mitigate patient harm stemming from the processes of health care.” WHO established the goal to establish a set of Patient Safety Solutions Aides memoir (hereinafter called Solutions) that were evidence-based, presented in a standard format, and described in simple terms the actions to take to address risks associated with a particular patient safety problem. Each written Solution was to present the relevant problem and characterize the strength of evidence supporting actions to address it, including potential barriers to adoption, risks of unintended consequences created by the solution, and patient and family roles in the solution. In addition, each Solution was to cite relevant references and other resources.
In 2005, the World Health Organization (WHO) designated the Joint Commission and Joint Commission International as the WHO Collaborating Centre for Patient Safety Solutions.
Working with WHO, and with guidance from an International Steering Committee, the Collaborating Centre developed the first set of nine Patient Safety Solutions.
The Inaugural Set of Patient Safety Solutions Aides Memoir In April 2007, the inaugural set of Solutions was approved by the International Steering Committee for publication and dissemination, for use by WHO Member States as of May 2007.
The Solutions were posted on the websites of both WHO and the Collaborating Centre. 2 The
following are brief descriptions of topics for the nine Solutions:
Look-Alike, Sound-Alike Medication Names. Confusing drug names is one of the most common causes of medication errors and is a worldwide concern. With tens of thousands of drugs currently on the market, the potential for error created by confusing brand or generic drug names and packaging is significant Patient Identification. The widespread and continuing failures to correctly identify patients often leads to medication, transfusion and testing errors; wrong person procedures; and the discharge of infants to the wrong families.
Communication During Patient Hand-Overs. Gaps in hand-over (or hand-off) communication between patient care units, and between and among care teams, can cause serious breakdowns in the continuity of care, inappropriate treatment, and potential harm for the patient.
Performance of Correct Procedure at Correct Body Site. Considered totally preventable, cases of wrong procedure or wrong site surgery are largely the result of miscommunication and unavailable, or incorrect, information. A major contributing factor to these types of errors is the lack of a standardized preoperative process.
Control of Concentrated Electrolyte Solutions. While all drugs, biologics, vaccines and contrast media have a defined risk profile, concentrated electrolyte solutions that are used for injection are especially dangerous.
Assuring Medication Accuracy at Transitions in Care. Medication errors occur most commonly at transitions. Medication reconciliation is a process designed to prevent medication errors at patient transition points.
Avoiding Catheter and Tubing Mis-Connections. The design of tubing, catheters, and syringes currently in use is such that it is possible to inadvertently cause patient harm through connecting the wrong syringes and tubing and then delivering medication or fluids through an unintended wrong route.
Single Use of Injection Devices. One of the biggest global concerns is the spread of Human Immunodeficiency Virus (HIV), the Hepatitis B Virus (HBV), and the Hepatitis C Virus (HCV) because of the reuse of injection needles.
Improved Hand Hygiene to Prevent Health Care-Associated Infections. It is estimated that at any point in time more than 1.4 million people worldwide are suffering from infections acquired in hospitals. Effective hand hygiene is the primary preventive measure for avoiding this problem.
Evaluation Aims and Approach
The aims of this evaluation were to perform a comprehensive evaluation of the following:
The concept of the patient safety solutions The value of the Aide memoir as a product/tool for generating awareness of patient safety solutions and the patient safety risk areas that they address, as well as for framing evidence-based policies and mechanisms to improve patient safety by WHO Member States.
The process of development, quality and dissemination of the nine patient safety solutions Aide memoirs;
The impact in terms of relevance and usability of the developed patient safety solutions Aide memoirs by both developing and developed Member States.
The evaluation was a retrospective assessment of the nine Patient Safety Solutions Aides memoir established in 2007. It was designed to obtain a mix of perspectives from a variety of stakeholders, including those who are not affiliated with WHO or the Collaborating Centre and those who are.
The following questions were addressed in the evaluation:
1. How effective is the concept of patient safety solutions Aide memoirs as the means to generate awareness of patient safety problems and available solutions, and to inform and aide governments as a policy development tool?
2. What is the quality or scientific integrity of the developed Aide memoirs solutions in terms
3. How valuable are the Aide memoirs in terms of the product?
4. What impact have the patient safety solutions Aide memoirs had in terms of aiding Member
States achieve the objectives of improving patient safety:
5. How can the concept, value, quality and impact of the product be improved? Some of the
critical questions are:
Logic Model That Guided the Evaluation The evaluation was guided by a logic model of the system within which the Solutions had been developed and implemented. Within this system are numerous stakeholder groups, each with unique needs and priorities, which should be understood to assess how well the first set of Solutions have functioned and to best determine if and how to structure and manage this program in the future.
This logic model is shown in Figure 1, which shows the World Health Organization regional structure (on the left) and the parallel regional clustering of its Member States (on the right). The work of the WHO Collaborating Center on Patient Safety Solutions is shown in the center, including each of the key steps undertaken in the selection, development and dissemination process. Throughout the development process, the Collaborating Centre interacted with staff and others involved with WHO, as well as with individuals in many of the Member States.
All of these organizations, and the individuals who are part of them, are stakeholders in this process. Further, we understood that each set of stakeholders provided its own unique perspectives and priorities as the solutions were being developed, and likewise, had a variety of views and reactions to the completed Solutions products.
Analysis of Data Collected Three analytic methods were used, one for each type of data collected, the results of which were synthesized together to generate overall findings and recommendations. The analyses addressed factual information on the development and dissemination of the Solutions, the perceptions of stakeholders interviewed about the Solutions program, and results of the stakeholder survey.
Description of the Development and Dissemination Processes The primary sources of information about the development and dissemination processes were documents provided by the staff at WHO and the Collaborating Centre, along with telephone discussions with them to verify facts and gather additional detail. Using this information, the steps in the Solutions development process were summarized, including the structure established for the collaborative process, decisions made about topics for the Solutions, and engagement of stakeholders to get their feedback on draft products. The process for disseminating the Solutions, once released, also was summarized, including the dissemination plan established, actual dissemination activities, and feedback from stakeholders on how the Solutions were being used in their countries. These descriptions are located at the start of each of the relevant chapters in this report.
Analysis of Stakeholder Survey Results The questionnaire was designed so that the broadest response could be obtained for questions regarding the patient safety needs and priorities in the WHO Member States, as well as stakeholder perceptions of how well the Patient Safety Solutions responded to those needs. Then a subset of respondents who reported they had been aware of the Solutions before the survey were the subjects for most of the remaining questions, which required familiarity with the Solutions to answer effectively. Finally, a smaller set of respondents who had actually used the Solutions in local patient safety improvement initiatives were asked questions about how they used the Solutions, and how useful they were to the improvement efforts.
A total of 956 surveys were completed by stakeholders in the six WHO regions. Of these, 75 individuals completed only the first three questions (what is your country, what type of organization are you in, and what is your current health- care role). Therefore, the effective sample is 881 completed surveys. This sample size was sufficiently large to generate useful estimates of stakeholders views and experiences with the Solutions. It was possible to examine regional differences in patient safety needs and responsiveness of the Solutions to those needs.
For other topics, however, only overall estimates could be developed due to small sample sizes.
The counts of respondents for are reported with each set of survey results.
Highlights of Findings: the Patient Safety Solutions Products The results presented in this Section address evaluation questions related to the actual products of the Patient Safety Solutions Aides memoir. These include the effectiveness of the underlying concept of the Solutions, the value of the Solutions to users, the evidence-base assessment, and the effectiveness of the written products actually produced. For each topic, the pertinent evaluation questions being addressed are listed, followed by a presentation of evaluation results.
The Section ends with a discussion of findings based on these results.
The results for each topic are a combination of results from the stakeholder survey, individual interviews, and review of written materials and resources. Through this approach, it is possible to synthesize the diverse perspectives of different stakeholders. It also allows the analysis to combine quantitative data from the survey with qualitative data from the individual interviews, to “tell a cohesive story” with the interview information helping to interpret the survey results and capturing the dynamics of stakeholders’ experiences with the Patient Safety Solutions.
The concept of the Patient Safety Solutions appears to have been well accepted. Most stakeholders interviewed felt that its primary role was to generate awareness, and that it should be used in conjunction with other WHO patient safety products and tools. They sought further guidance from WHO regarding its vision for the Alliance for Patient Safety and how the Solutions fit into that larger vision and package.
Variations in local patient safety need and priorities were found across countries. Although some differences between developing and developed countries were reported, some issues were shared across many countries, which offered an opportunity for the Solutions to address issues of importance to a large number of them. Indeed, the topics selected for the first set of Solutions were generally accepted as being of high priority and broadly applicable across countries of differing developmental status. This assessment emerged in results of the Field Review of the 2007 Solutions as well as in the stakeholder interviews and survey.
Stakeholders generally perceived that the key audience that WHO had designated for the Solutions was policy makers at the national level of countries. They also tended to agree that this target was too narrow, and that the audiences should be expanded to include health care providers and patient populations, as other important stakeholders for patient safety. This view was supported by the survey results, in which three groups were identified as target audiences by larger percentages of respondents than were the ministries or departments of health.