One of the needs identified was a means of providing quantitative feedback on how well current practice meets the needs of patients and their families and of professionals.  The Organizational Assessment project (1999) was undertaken to fill this gap.  Guided by the messages of the Living and Healing study, existing quality of life and patient satisfaction measures were reviewed.  None fully addressed needs identified by study participants.  The tools which came closest to meeting criteria were the City of Hope Quality of Life patient and family tools, which, with significant modifications and additions, became the basis of the Organizational Assessment  Patient,  Family Caregivers, and Bereaved Family questionnaires.  A parallel Health Professional questionnaire was also developed using data from the Living and Healing study.

AREAS	Patient Perspective	Family Caregiver Perspective	Bereaved Family Perspective	Professional Perspective	Professional Competency Perspective	Organization Perspective
Physical
Emotional
Spiritual
Relationships
Health Care Experiences

 
 
 
 
 
 
 
  
  
  
  
  
  
 
  
  
  
  
  
  
An organization or program can choose to assess all areas from a single perspective, e.g. from the patient perspective, or can select a single area, such as the spiritual area, and assess it from the perspective of patient, family caregiver, bereaved family, professional, competency, and organization. If the concern and interventions were planned to address a single area and feedback desired from only one perspective, the organization could assess that area and perspective only, e.g. examine health care experiences from Bereaved Family perspective only.  In other words, an organization could use one sub scale and ask only one group to fill it out.  Additionally, a clinician could use the tools over time to determine how well the patient and family are doing.

LEARNINGS

Important lessons learned were learned in the pilot phase of this project which can guide other organizations intending to use any of the tools for assessment of the quality of their end of life care. They are listed as DO and DON'T statements. In addition, common questions and answers are presented at the conclusion of this section.

Do identify the gatekeepers and powerful voices within your organization. Having a physician champion for this project is essential to its success. The physician champion is able to discuss the possible benefits of obtaining this information with peers and to enlist their cooperation as their patients participate in the project.
 
Do assure physicians that this data will not identify individual practitioners, but will provide guidance as the organization seeks to focus on specific quality improvement initiatives to improve care for those affected by life threatening illnesses.
 
Don't talk about patients who are dying.  The focus of this work is to improve care for those who are living with or affected by a chronic progressive or life threatening illness.  Patients may ultimately die from their condition, but right now they are living and the goal is to help them live fully.
 
Do be clear about who to include and who to exclude from the population studied.  Plan for multiple education sessions with physicians and staff about this work.  Health professionals often think of palliative care as applying only to persons in the active phase of dying.  Help them think in terms of people who are approaching the last stage or phase of their life.
 
Do use questions such as, " For which patients would you NOT be surprised if they died of this condition within the next 12 months."  However, we found that even that language caused some difficulty when identifying persons who could provide feedback.  Professionals reported that there was concern about taking away hope for those so identified. Another phrase that might help is, "For which patients are you concerned about the seriousness of their illness or symptoms and concerned about their survival?"
 
Do determine the number of people who have died while receiving care in your facility during the prior six months and compare your referrals to the project as a percent of that number. This will give you an approximation of the possible population of concern and alert you to problems in referrals.
 
Don't assume that those who refer participants to the project are comfortable identifying the possible participants. Provide multiple opportunities for them to discuss the project, their concerns about the project, and to be specific about what it would take for them to be active participants and champions for the project.
 
Do have a plan to collect stories of excellent care as well as care that did not meet your expectations for end of life care as you gather the questionnaire data.  This will assist you to understand examples of care that contributed to the outcomes or questionnaire numbers.  Numbers alone don't tell the story because one cannot translate numbers to specific understanding of care actions.  Also, stories without numbers do not allow you to set improvement targets or test your effectiveness when you have implemented new strategies to improve care. Using both approaches provides the most useful information to guide practice changes.
 
Do modify your approach as you learn what works and what does not work within your own organization. No single approach works for all organizations or populations.
 
Do be sensitive to this as a vulnerable time for patients and families.  Be prepared to offer support and services during and after the survey. Some family members will not want these issues raised for their loved ones.  Make sure that the importance of the data gathered outweighs the burden for patients and families.
 
Do provide opportunities for those who are gathering the information to meet often and discuss learnings and concerns.
 
Don't seek the information on the day of discharge or immediately following a significant change.
 
Do use letters, presentations, and one-on-one communication to keep the importance of your project visible. Commit to a time when feedback will be available and commit to planning interventions based on the feedback.
 
Do use the tools as you design quality improvement initiatives. Use only the sub scale that applies to your area of concern.  Ask for feedback only from the population that is affected by your interventions. 

Questions and answers that should be considered as you begin using these tools.

Is this a questionnaire or an interview form?

It can be used either way. Interviews take longer. However, they may be the best method of gathering information from participants after they have gone home.

Do I have to use the whole questionnaire?

NO. Use only the subsections of the questionnaire that suits your objectives. We do suggest that using the entire questionnaire to establish a baseline prior to implementing practice improvements is very useful. Although the tools seem long, it takes only 30 minutes on average for someone to complete the questionnaire.

Does the person have to fill the questionnaire out all at once?

NO. However, we suggest it be completed within a 2-hour period. You are more likely to have participants respond to their current status when they complete the form within that time frame. People took between 30 minutes and 90 minutes to complete the questionnaire.

Do I have to use all versions?

NO. You may use only one version or as many as you wish. These are designed to allow you to compare perspectives if that is useful to you. They are also designed as stand alone questionnaires.

Won't these questions raise some sensitive issues?

They may do that.  A follow-up phone call may be advisable if answers on the questionnaires cause concern and, of course, if the person requests a follow-up phone call.  Do this as soon as you receive and review the completed questionnaire. It is a good idea to review our cover letter from the pilot phase and create one that addresses similar content. Our goal is always to support the patient and family. Some of the data collectors expressed concerns they were raising sensitive issues at a time when the patients and family were vulnerable. Also, data collectors were concerned about entering the patient's space only for the purpose of collecting information. Patients and families are often pleased to be asked to assist in improving care. However, that may not be true for ALL patients. Be sensitive to verbal and non verbal cues.

Why is this so complicated? Can't we just measure number of advance care plans or pain scores?

That is always a choice. However, the tools in this project developed from the Living and Healing During Life-Threatening Illness study and reflect the perspective of what matters to patients and families affected by life-threatening illess.  They are comprehensive and flexible.  They allow the focus to move beyond assumptions made by health care organizations to the actual experience of patients, families, and caregivers.

Contributing to a National Data Base

For many organizations, it is helpful to compare their own data to those of other organizations. If this might be helpful for you, we invite you to participate in a voluntary national data base development project. Using the Modified City of Hope questionnaires as revised, the Coalition will include your data with the national data base and provide for you a report of your outcomes compared to the national combined data. This service is free to organizations whose systems are members of the Coalition. Others will be charged a fee to defray the expenses involved.

How do I participate?

Submit at least 30 participant responses for each desired focus. Collect the data over a quarter or other specified timeframe. Use all or a subsection of any of the questionnaires. Your data will be combined with data for that questionnaire or subsection. Data must be submitted on disk or electronically using an Excel file that is obtained from the Coalition office. Specific information from you about your organization and the population surveyed will also be requested.

What do I get?

You will receive a report comparing your data to data submitted by other organizations. At this time, we are unable to compare your results to like size or location organizations.

What will it cost?

There is no cost for this service for Coalition member organizations. Non member organizations will be charged a fee of $150 / data set. A data set is defined as data submitted at one time for a single population. In other words, you could submit data from 90 respondents with life threatening illnesses of cancer, renal disease, respiratory disease, and / or cardiac disease and they would be considered a single population. They are all people with life threatening illnesses and the data was submitted as a single file. Family caregiver data for this same group would be considered a second data set.

Where will it go?

Data is to be sent to Supportive Care Coaltion: Pursuing Excellence in Palliative Care in Portland, Oregon. The office has contracted with researchers and statisticians working for Center for Outcomes Research and Education (CORE) at Providence Health System for statistical support and report generation.

     Karin Dufault, SP, RN, PhD, Executive Director
     Supportive Care Coaltion: Pursuing Excellence in Palliative Care
     c/o Providence Health & Services
     4805 NE Glisan 2E07
     Portland, OR 97213-2967 

     Phone (503) 215-5053
     FAX  (503-215-5054
     Email: karin.dufault@providence.org


When should I do this?

It is recommended that you participate by providing baseline data prior to implementing quality improvement initiatives, that you participate during your follow-up evaluation to the interventions, and as an ongoing annual feedback mechanism. More or less frequent participation is at your discretion and should be supportive of your overall goals and objectives.

When will I get my reports?

Reports will be sent to you 2-4 weeks after we receive the data files in a format including analysis, graphs, and interpretation of the results.

How do I find out more about this project?

The contact person for this effort is Karin Dufault, SP, RN, PhD, Executive Director of the Coalition.  She should be contacted before beginning the assessment and is available for phone consultation during the project.  Data entry files and other required forms should be obtained from the Coalition before initiating the project.

CONCLUSIONS

While it is important to learn how well we are providing care to those at the end of life and their families, we have learned that one of the hardest tasks is to identify the population from whom feedback is needed to help us modify our services and practices. This reluctance to actually identify persons with a life threatening illness or facing the possible end of their life is not a new problem. Our experience during the pilot phase of this project reinforced for us that professionals of all disciplines, as well as the public, are not ready to identify those who are approaching death. Unless we can systematically identify those, we will be unable to provide systemic interventions to improve care. Our challenge is to develop the language that will help us overcome this barrier.

We learned that administering the questionnaire had the potential to raise issues for patients and their families, and some declined to participate. This may be partly due to the fact that they have not dealt with the approaching death and it may be due to their current sense of grief and vulnerability. Any approach to gathering information should be sensitively undertaken with support systems in place. For example, perhaps a spiritual care visit should be planned following the completion of a questionnaire just as we would address physical concerns as soon as they are identified.

We also learned that the responses to the questionnaire have all the biases of any questionnaires. We do not know about the experience of those who chose not to respond, we may only get the information asked for on the questionnaire, and people may want to give us socially acceptable positive ratings and not share their true concerns. We reiterate that the answer to the real question about how well an organization is meeting the end of life care needs of people and their families lies within the responses to the questionnaires, the stories shared by families and patients, and stories shared by professionals. No one perspective will provide a comprehensive view of the state of care.

In spite of the difficulties, there is value in conducting the surveys. There is learning about your organization's readiness to address end of life care issues as you prepare to engage in the surveys. There is opportunity to identify educational needs and barriers to meeting needs of people at the end of life as you structure the environment to participate in the process. There are personal professional growth opportunities as your physicians and staff suspend assumptions about end of life care from their professional perspective and are more open to feedback from those living with and affected by life threatening illness.