CQI’s methodology consists of a set of interrelated processes that at a minimum: 1) develop or adapt surveys based on consumer expressed preferences, 2) conduct personal interviews or focus groups with clients/consumers, 3) analyze data and write reports with recommendations, 4) present reports to relevant stakeholders, and 5) develop action plan(s). Since the goal is to maximize the consumer voice in survey research, CQI’s methodology requires that it lead and participate in every step of the research process. CQI’s general mode is thus to lead a survey research project and act collaboratively with other research partners.

CQI’s research can be divided into four categories:

Program Evaluation/Research
Community-Based Participatory Action Research
Consumer Satisfaction and Assessment of Quality
Service planning and needs assessments

CQI personal interviews and focus groups form the foundation of our work [In person interviews are critical to obtaining the best and most reliable information.] Personal interviews are conducted in private, and information that would identify the person interviewed is not shared with anyone . The interviews are conducted by individuals who have had experience with mental health services either as consumers or family members so that a rapport can be developed Respondents then have an unusual opportunity to open up. Thus, CQI interviewers are there to prompt them in a way to think thoughtfully and critically about the quality of their care.

The sophisticated nature of personal interview survey research requires that CQI staff, made up of consumers and family members, be experienced in a variety of research areas (ie, consumer researchers and evaluators). Two of the five main staff have advanced degrees in applied research. Over time, staff have developed a significant expertise in survey design, coordinating interviews, conducting personal interviews, and qualitative review. Where CQI staff does not have expertise, board members, friends, or consultants have been helpful. CQI’s excellent track record as evaluators has brought us loyal clients and excellent feedback on the readability, comprehensiveness, and utility of the reports.

Program Evaluation/Research
CQI leads evaluations of new and/or unique programs, in particular aspects that involve personal interviews and focus groups. In this case, CQI collaborates with consumers and other stakeholders to develop an interview guide, conduct data analysis, and prepare the data for review, publication, or both. As part of this process, CQI conducts a series of interviews with the new program’s clients upon initiation of the program, and at regular intervals thereafter.

CQI Process Evaluation: The goal of a CQI process evaluation is to assess whether a program is operating in an efficient and effective manner, often with fidelity to a newly developed model. CQI helps to refine key program components, including program assumptions, intervention characteristics, and outcomes for the intervention or program being developed.

CQI Outcomes Evaluation: The primary intent of the CQI outcomes evaluation is to determine the effects of the program on the intended clients or consumers. The evaluation will also examine the effects the clients’ participation in the program has on the mental health of them and their families.

More often than not, CQI conducts simultaneous process and outcomes evaluations.

In 2002, the Boston Medical Center obtained a SAMHSA grant to initiate and research a 8-bed Safe Haven shelter, which provides housing and services in a low-demand setting for people who are chronically homeless, actively substance abusing and have severe mental illness. CQI staff are members of a project steering committee, and they are part of the research team, having helped to develop the research design. More specifically, CQI is leading the effort to learn about resident experiences, working with the evaluation committee to develop the interview guide, conduct interviews, code data and conduct data analysis.

Also, CQI is using this methodology to assist the University of Massachusetts Center for Health Care Policy and Research to evaluate an innovative self-directed care program. CQI has also been working with the state of Massachusetts to evaluate an innovative comprehensive system of care approach to meeting the special needs of families with a child(ren) with a serious emotional condition, through interviews with parents who are current and past clients.

Community Based Participatory Action Research
CQI works with funders to construct research projects that are designed to influence policy and practice by involving consumers in the project early on. Community Based Participatory Action Research (CBPR) provides a framework for accomplishing these goals.

CQI's strategy of participatory action research engages the mental health consumers and family members in dialogue about issues that are of concern to them. Participatory action research is a partnership between the affected community and researcher where the community is actively involved in all phases of the research project: defining the problem, designing the research methodology, collecting data, and analyzing and disseminating results. For example, CQI has used CBPR as a capacity building and prevention approach for young adults with psychiatric conditions (The Young Adults in Transition Citizenship Project). Through our CBPR approach, CQI has also developed culturally specific training materials and has co-founded the Boston Mental Health Community Academic Partnership.

Consumer Satisfaction and Assessment of Quality
CQI is frequently called upon to survey clients of behavioral health services to help the system or program improve the quality of its care. CQI reports on consumer opinion of the quality of services through semi-structured personal interviews with individuals and families who receive such services. Quantitative questions are designed to be actionable, and interviewers ask for comments when a respondent provides a negative response. In addition, CQI asks several open-ended questions to give overall context to the hard data. Demographic information is collected, and often used for comparative purposes.

Data-driven reports are written that include recommendations.  CQI staff meet with both provider staff and/or the funder (egs, DMH, MBHP) to attempt to discuss the report. Thus, when CQI surveyed at several day programs several years ago, some clients said that they had not been provided with clear mandate to call a specific crisis line if they were in crisis. In all cases, providers made improvements in response to CQI recommendations, as evidenced by follow-up CQI focus groups and surveys.

After interviewing at several provider programs of a specific type, we are able to write aggregate reports based on interviews with hundreds to thousands of people that identify large scale systems trends.

Service Planning and Needs Assessments
All stakeholders are beginning to think more about designing systems that are directly responsive to the needs and desires of consumers and family members. CQI conducts research on the needs and desires of people who use or might use services in order to assist in planning consumer-driven systems of care. After interview guides are developed and the interviews are conducted, a data-driven report that includes recommendations for systems improvement is written and presented. . Recent examples of such reports include Voices of Youth in Transition: The Experience of Aging out of Public Mental Health Services in Massachusetts: Practice and Policy Implications, DMH Clients and Employment in Western Massachusetts: The Consumer Perspective, and the MassHealth Quality of Life report.