Evaluation

Evaluation provides a systematic process to document the impact of programmatic work, identify needs for quality improvement, and maximize intervention outcomes. The Center for Excellence in Health Innovation (CEHI) provides comprehensive evaluation services and supports external evaluators to provide programmatic quality improvement/quality assurance. CEHI staff uses a variety of methods to conduct evaluation, with a purpose to share lessons learned and inform continued development of programmatic work and interventions. Evaluation projects come from federal agencies, state agencies, private foundations, and community organizations.

Evaluation Projects

Chronic Disease Evaluation Services

Funder: Maine Center for Disease and Control's Chronic Disease Prevention and Control Program (CDPCP)
Project Period: November 2018–September 2023

CEHI staff provides evaluation services for federally funded project 1815 — Improving the Health of Americans through Prevention and Management of Diabetes and Heart Disease and Stroke. The team collects and analyzes data to track progress for interventions and activities across Maine. Evaluation reports focus on identifying challenges, highlighting outcomes, and providing continuous quality improvement for the Maine CDC's CDPCP.

The five-year evaluation (2018–2023) will capture the results of the project in terms of progress toward four areas of focus:

  • Improve access to and participation in evidence-based programs related to the prevention and management of diabetes, hypertension, and cardiovascular disease including National Diabetes Prevention Program (National DPP), Diabetes Self-Management Education and Support (DSMES), and Self-Measured Blood Pressure programs.
  • Assist health systems with the implementation of evidence-based systems for improving patient care including bi-directional electronic referral systems and team-based care.
  • Promote the adoption and utilization of electronic health records, health information technology, and clinical quality measures (i.e., National Quality Forum) reporting to improve patient care and monitor health disparities.
  • Increase public and private health plan coverage for National DPP and strengthen the utilization of DSMES as a covered benefit by eligible health plan beneficiaries.

Clinical Interprofessional Curriculum

Funder: Josiah Macy Jr. Foundation
Project Period: February 2015–May 2019

CEHI evaluates the expansion of clinical interprofessional (IP) learning opportunities and UNE’s Clinical Interprofessional Curriculum (CIPC) learning activities, with an initial focus on sharing lessons learned, informing the continued development of the CIPC, and expanding it to other clinical practices. While CEHI evaluates the impact of each of its clinical partner's IP education activities on student knowledge of and attitudes toward team-based care, it also recommends several assessment tools and resources to assist practice sites to measure the impact of IP care on practice-specific measures or further develop and improve IP practice.

Collaborative SBIRT Training for Maine's Future Health Profession Leaders (SBIRT – Student Training)

Funder: Substance Abuse and Mental Health Services Administration
Project Period: 2016–2019

University of New England's Westbrook College of Health Professions was awarded a SBIRT (Screening, Brief Intervention, and Referral to Treatment) Student Training Grant from the federal Substance Abuse and Mental Health Services Administration. The grant's primary goals are to:

  • Develop and implement a student SBIRT training program.
  • Advance the adoption and practice of SBIRT throughout Maine’s system of care.
  • Reduce behavioral health disparities among racial, ethnic, sexual, and gender minorities by strategies to decrease differences in access, service use, and outcomes.

CEHI staff provide program evaluation services for UNE SBIRT including overseeing the longitudinal evaluation of SBIRT Leaders program and assessing and evaluating substance use training needs among Maine’s practicing professionals.

The Interprofessional Education Collaborative 

CEHI has been conducting process and outcome evaluation studies for UNE’s Interprofessional Education Collaborative since 2014. Key activities include:

  • Annual Surveys of All Health Professions Students in Westbrook College of Health Professions, College of Dental Medicine, College of Pharmacy and College of Osteopathic Medicine.
    • August 2014 to August 2018: Surveyed 827 incoming (new) students about IPE Past Experience, Influence of IPE programs on selecting UNE, and Communication Preferences
    • May 2015 to May 2018: Surveyed 721 enrolled students about IPE Experiences at UNE, Preferences, and IPEC Competencies
  • ​​Evaluation of the Interprofessional Team Immersion (Offered twice a year, 2014–2018)
    • Student pre-post surveys re: knowledge, attitudes, and skills (e.g. ISVS, ICCAS)
    • Qualitative reactions (e.g. Influence on Future Practice)
    • Faculty Observation of team skills (ICAR) and Rapid Cycle QI Process Feedback
  • Evaluation of Parallel Processes Case-Based Learning (Offered Summers, 2017 and 2018)
    • Student pre-post surveys on IPE skills (Communication Scale and ICCAS)
    • Qualitative reactions e.g. Influence on Future Practice
    • Faculty debrief sessions

The Maine Advanced Nursing Education Sexual Assualt Nurse Examiners Program (Maine ANE SANE Program)

Funder: Bureau of Health Workforce, Health Resources, and Services Administration
Project Period: September 2018–September 2021

​​The Maine ANE SANE Program is a collaborative effort of the UNE Department of Nursing and the Sexual Assault Forensic Examiner (SAFE) Program at the Maine Office of Child and Family Services, Maine Department of Health and Human Services. This unique private/public collaboration will strengthen the state’s existing innovative SANE training program by increasing its reach and impact through simulation and pedagogical training, ECHO technology to reach rural participants, and the use of geographic information systems (GIS) to identify "hot-spots" where the existence of SANEs is absent or sparse. GIS technology will also be used to identify the locations and density of special populations in Maine in which sexual assault is high (e.g., the elderly, disabled, immigrant and refugee, and Native American populations). The program will particularly target SANE trainings in these areas and for these populations. CEHI staff will be providing evaluation services and technical assistance to ensure that SANE trainings are delivered as intended and achieve the planned learning objectives.

The Maine Area Health Education Center Network (AHEC)

Funder: Bureau of Health Workforce, Health Resources, and Services Administration
Project Period: September 2017–Aigust 2022

CEHI evaluates the Maine AHEC Network through the use of mixed methods, including both quantitative (e.g., survey scales for quality and outcome measures) and qualitative (e.g, open-ended questions, reflective debriefs, interviews) data collection. The formative (or process) evaluation of the Maine AHEC network and quality improvement activities will rely heavily on the off-site program office and CEHI staff for ongoing monitoring and reporting into the Maine AHEC database for tracking programs and participants. For the summative (or outcome) evaluation, many tools and processes are in place to measure intended objectives and outcomes, based on Maine AHEC’s evaluation history to date. However, new programs like the Care for the Underserved Pathway (CUP) AHEC Scholars will involve new tools and processes to be developed and the piloting of tools and measures. Documentation of quality improvement changes made in response to information gathered through the evaluation will use a rapid feedback approach including the use of dashboards, brief memos, and debriefing. 

The Maine Leadership Education in Neurodevelopment and Related Disabilities (LEND) Program

Funder: Maternal and Child Health Bureau, Health Resources and Services Administration
Project Period: July 2016–June 2021

The Westbrook College of Health Professions was awarded a grant from the federal Maternal and Child Health Bureau to develop the Maine LEND program. The program's goal is to leverage the rich clinical, policy, and research resources of UNE and Maine Medical Center to train promising professionals in interdisciplinary approaches to assessment, treatment, and systems leaderships. CEHI staff provide program evaluation services for Maine LEND including monitoring and QI for didactic and experiential trainings of students and practicing health professionals, data coordination to ensure timely federal reporting, and measuring longitudinal outcomes for long-term LEND trainees.

Maine Prevention Services: Substance Use Prevention Services

Funder: Maine Center for Disease Control and Prevention
Project Period: October 2016–September 2021

CEHI evaluation staff provides quality assurance and continuous quality improvement to managing the statewide substance use prevention grant. Staff monitor a progress report that includes data regarding community interventions and activities across Maine. This monitoring tool provides the opportunity to track progress, identify challenges, and highlight accomplishments in a timely manner. CEHI evaluation staff coordinates its quality assurance processes with the state contracted evaluator, Public Consulting Group.

Maine Supplement Nutrition Assistance Program Education (SNAP-Ed):

Funder: Food and Nutrition Services and National Institute for Food and Agriculture, United States Department of Agriculture
Project Period: October 2018–September 2023

CEHI staff lead the internal evaluation of UNE’s SNAP-Ed statewide implementation plan. The multi-disciplinary team tracks and reports outcomes and provides continuous quality improvement to support successful program interventions at the individual, environmental, and sector levels. The evaluation team monitors and reports monthly data to track local progress for community interventions and activities. The data allow the implementing team to identify challenges and highlight accomplishments at the local and state levels. Outcomes related to nutrition education, participant satisfaction, and implementation of policy, systems, and environmental changes are shared annually with partners and funders who broadly impact the program by making sector-level changes in collaboration with the implementing team. The Maine SNAP-Ed evaluation plan is guided by the United States Department of Agriculture’s SNAP-Ed evaluation framework and utilizes the indicators and tools designed to facilitate aggregate reporting on national priorities. 

Primary Care Training and Enhancement - Bangor

Funder: Bureau of Health Workforce, Health Resources, and Services Administration
Project Period: July 2016–June 2021

CEHI was awarded a five-year grant from the Health Resources and Services Administration in July 2016 to continue interprofessional education from the classroom into the clinical setting, with an emphasis on rural health. UNE partners with Penobscot Community Health Care (PCHC), Maine’s largest federally qualified health center, to train UNE osteopathic medical, pharmacy, and physician assistant students, and PCHC clinicians, on social determinants of health, shared decision-making, oral health in primary care, interprofessional practice, and health literacy. Evaluation of the project includes the following:

  • Number and types of activities students were involved in at PCHC
  • Knowledge, attitudes, and beliefs among students and clinicians on the impact of the project
  • Patient satisfaction of having students involved in their care

​​Primary Care Training and Enhancement - Medication Assisted Treatment

Funder: Bureau of Health Workforce, Health Resources and Services Administration
Project Period: July 2017–June 2019

CEHI was awarded a one-year grant from the Health Resources and Services Administration in July 2017 to address the epidemic of opioid use disorder (OUD) in Maine, with an emphasis on medication-assisted treatment (MAT). Specific objectives include training UNE health professions students and community-based providers on OUD and MAT, and increasing the number of MAT prescribers at Penobscot Community Health Care. Evaluation of the project includes the following:

  • Number and types of attendees at various trainings
  • Knowledge, attitudes, and beliefs among training attendees on the impact of activities
  • Number of new MAT prescribers and patients on MAT at PCHC

Primary Care Training and Enhancement - Champions

Funder: Bureau of Health Workforce, Health Resources and Services Administration
Project Period: October 2018–September 2023

CEHI was awarded a five-year grant from the Health Resources and Services Administration in October 2018 to create a fellowship for physicians and physician assistants (PAs) practicing in federally qualified health centers in Maine, New Hampshire, and Vermont. The fellows will receive training and mentorship on healthcare leadership as it relates to opioid use disorder (OUD) and medication-assisted treatment (MAT). They will then apply what they’ve learned by completing a healthcare transformation project related to OUD and MAT, and precepting UNE osteopathic medicine and PA students. Evaluation of the project includes the following:

  • Knowledge, attitudes, and beliefs among fellows on the impact of trainings and mentorship
  • Outcomes of healthcare transformation projects
  • Number and types of activities involving students

Project Alliance, Drug-Free Communities

Funder: Office of National Drug Control Policy
Project Period: October 2015–September 2020

Coastal Healthy Communities Coalition (CHCC) at the University of New England’s College of Osteopathic Medicine was awarded a Drug-Free Communities grant from the federal Substance Abuse and Mental Health Services Administration. Project Alliance’s goal is to prevent and reduce substance use among youth in nine towns in Northern York County. CHCC contracts with the CEHI for program evaluation services, including quantitative and qualitative data collection, analysis and display, and environmental scanning.

This website uses cookies to understand how you use the website and to improve your experience. By continuing to use the website, you accept the University of New England’s use of cookies and similar technologies. To learn more about our use of cookies and how to manage your browser cookie settings, please review our Privacy Notice.

Accept