"Evaluation for Excellence" Program Makes Excellence More Measurable

In October 2000, the AIDS Foundation of Chicago launched a new initiative entitled "Evaluation for Excellence." Modeled after the very successful Prevention Evaluation Demonstration Project (1997-1999), Evaluation for Excellence matched eight doctoral-level researchers with eight community-based agencies to conduct process and outcome evaluation activities. As public and private funders increasingly require community-based agencies to measure both the quality and outcomes of their HIV prevention and care services, this initiative enabled AIDS service organizations to determine and demonstrate the effectiveness of their programs and to make necessary changes to enhance program design and delivery and ultimately client outcomes.

AFC provided direct funding to eight evaluators/researchers to work with eight community-based agencies to evaluate an existing HIV care or prevention program. AFC staff actively recruited independent and university-based evaluators and researchers with experience working with community-based agencies. Agencies were chosen based on their response to a Request for Proposals.

Project Objectives:

• To improve HIV prevention and care services through process and/or outcome evaluation
• To identify, replicate, and share strategies that make such programs successful
• To enhance the capacity of community-based agencies in the areas of program design, implementation, and evaluation
• To establish ongoing collaborative relationships between community-based organizations, local universities, and researchers/evaluators

Highlights:

The Evaluation for Excellence program provided significant value to the agencies. Each agency grappled with many issues related to the provision and quality of client services, including:

• Client satisfaction
• Client perceptions of services
• Issues of confidentiality and consent
• Literacy and educational levels of clients
• Fidelity of program activities to a model or curriculum
• Description of program services in a "logic model" format

Many agencies were able to develop and "reality check" outcome measures for their respective programs and services, and make changes in programs or staffing to address any discrepancies between the intent of the programs and the actual outcomes as demonstrated by the evaluation activities. All the agencies reported an increased understanding of evaluation tools and the importance of program evaluation to their planning and consideration of client services.

The Future:

A portion of the program has been extended for two additional years to support evaluation activities at The Children's Place, Vital Bridges, and Vision House. The AIDS Foundation of Chicago will continue to support these efforts, and seek additional funding to support evaluation activities that are independent of other grants and contracts.

Participating Agencies and Project Description:

AIDS Legal Council
Project: Outcome evaluation and client satisfaction for legal services

AIDS Pastoral Care Network
Project: Outcome and process evaluation for three congregation-level educational interventions.

Horizons Community Services
Project: Process evaluation for drop-in program for gay, lesbian, bisexual youth

Howard Area Community Center
Project: Process evaluation for HIV/AIDS prevention program for English- and Spanish-speaking women

Interfaith House
Project: Evaluation of "success" and length of stay in transitional residential health program for homeless HIV-positive people

Open Hand Chicago (now Vital Bridges)
Project: Outcomes evaluation of grocery center program to determine factors that impact client utilization of food services. Outcomes evaluation for nutritionist services.

Regional Care Association
Project: To examine which treatment variables (including frequency and length of visits) contribute to what level of outcome in the areas of medical status, mental health status, and quality of life.

South Side Help Center
Project: Identify reasons that clients leave the Mental Health Counseling and Support Service Program