• who is the target population
• whats the nature of the problem being solved
• how does the service fit the environment

• clear sense of target population
• clear sense of need program will fill
• clear articulation of program benefits
• clear sense of alternatives

• are services being delivered
• can the same services be delivered at lower costs
• are the services reaching the right population
• are the clients satisfied with service

impact evaluation

 The program happened, how did it change lives? 

 Project impact evaluation studies the effect of an 

intervention on final welfare outcomes, rather than 

the project outputs or the project implementation 

process.  

 More generally, project impact evaluation 

establishes whether the intervention had a welfare 

effect on individuals, households, and communities, 

and whether this effect can be attributed to the 

concerned intervention. 

 When there is an important question you want/need to know the answer to 

 Common program with not much evidence 

 Uncertainty about which alternative strategy to use 

 Key question that underlies a lot of different programs 

 About to roll out a big new program, important design questions 

 Timing--not too early and not too late: 

 Test once basic kinks have been taken out 

 Before rolled out on a major scale 

 Program is representative not gold plated  

 Time, expertise, and money to do it right 

 Develop an evaluation plan to prioritize 

 Determine key questions for your organization. 

 How many can you answer  

 from existing rigorous impact evaluations? 

 from improved process evaluation? 

 Select top priority questions for impact evaluation. 

 Establish plan for answering them 

 taking advantage of roll out of new programs 

 set up experimental sites

 What would have happened in the absence of the 

program? 

 Since counterfactual is not observable, the key goal of all impact evaluation methods is to construct or “mimic” the counterfactual. 

 Constructing the counterfactual 

 Counterfactual is often constructed by selecting a group that is not affected by the program. 

 Randomized:  use random assignment of the program to create a control group which mimics the counterfactual. 

 Non-randomized: argue  that a certain excluded group mimics the counterfactual.

 lottery design 

 phase-in design 

 Rotation design 

 encouragement design