Program evaluation

Ten Steps

• Clarify what is to be evaluated: you need a clearly-defined goal or goals, population, strategies, activities, outputs, and outcomes
• Engage stakeholders, to build support
• Assess resources and evaluability
  • Is it clear why you are doing an evaluation?
  • Will it be useful?
  • Is there buy-in from leadership?
  • Are adequate resources available?
  • Is the timing right?
• Determine your evaluation questions, involving as many stakeholders as possible
• Determine appropriate methods of measurement and procedures, using the most feasible and credible methods
• Develop and evaluation plant, identifying specific evaluation activities, tasks, roles, resource allocations, and deadline
• Collect the data, using data collection tools and procedures
  • May need to pilot test the tools and procedures, and monitor collected data closely
• Process the data and analyze the results
• Interpret and disseminate the results
• Apply the evaluation findings

Logic Model

• See https://www.publichealthontario.ca/-/media/Documents/F/2016/focus-on-logic-model.pdf
• Formal way to design and plan an evaluation
• Can describe a program, initiative, policy, or organization
• Vary in specific components depending on the context

Component	Description	Examples
Common Components
Goal	Overarching long-term outcome that you hope to achieve	To prevent diet-related health problems in adulthood
Inputs	The resources invested into the program or initiative	Staff, funding, time, materials, supplies
Activities	The specific things that the program or organisation will do	PSAs on the risk of STIs; workshops on healthy eating
Audience	The target population for the program; may be primary (the main population who receive the intervention) and secondary (other groups who are impacted)	A teacher-training course impacts teachers (primary audience) but also students (secondary audience)
Outputs	Things that are produced by the program (quantitative)	Number of PSAs run; number of workshops run
Outcomes	The changes expected; may be short-term, medium-term, or long-term	Increased awareness of STIs, decreased incidence of STIs; increased knowledge of health foods and proper food handling
Additional Components
Situation	The context that lead to the creation of the program	Increased incidence of gonorrhea in 14-24 year olds; increased obesity amongst teenagers
Assumptions	Underlying theories and beliefs about the program and its context which can influence the development of a program and which activities are implemented. Transparency around assumptions makes explicit the beliefs that underlie chosen actions.	Teenagers are interested in physical activity; young adults listen to the radio and will hear the PSAs
External Factors	Factors that impact the program but are beyond the control of program planners and overseers. Factors may be positive or negative and are likely to influence program success.	Cultural norms, political climate, social policies, environmental factors
Strategies	A broad approach or plan to facilitate change	The Ottawa Charter for Health Promotion identifies five strategies to consider when developing health promotion programs: Build healthy public policy Create supportive environments Strengthen community action Develop personal skills Re-orient health services