Evaluations on a Shoestring

Having more demands on a program than a budget will allow was common well _before _the market went into decline. Today, programs have even fewer resources than several years ago, but they are still expected to maintain all documented aspects of their programs, including evaluation, which makes the need for efficiency highly critical. How do you carry out an evaluation on a tightly constrained budget?

A common approach is to first have a discussion with the evaluator about the various evaluation designs that may be appropriate for this particular program. Part of that discussion will likely involve the strengths and weaknesses of each methodology and the costs associated with each. Oftentimes, when designing a budget for an evaluation, we propose "phases of evaluation", such as logic model development (Phase 1), evaluation design (Phase 2), and evaluation implementation (Phase 3). For example, in Phase 1 we would lay out the theory of change and the outcomes that would be expected from the program. In Phase 2 we would discuss the data that needs to be collected, how it will be collected, and what method would be used for collection. Phase 3 would be when the data would be collected, analyzed, and reported. All three phases would be proposed and outlined in the original budget and the program director could choose how many of the phases they choose to fund. In some cases, a program may utilize an external evaluator solely for Phase 1. This approach allows flexibility in the evaluation funding. You can request this budgeting format from an evaluator.

Depending on the level of control you have over your program's expenses, an initial exercise may be to conduct a budget review, to ensure, before starting the evaluation, that your program's finances are being spent as efficiently as possible.

Even if you don't control the budget, you can still determine exactly how much can be dedicated to the evaluation. To do this, review the initial evaluation plan (e.g., the one in the grant that ultimately funded your program) with your evaluator.

Before you meet with the evaluator though, consider the outcomes of the evaluation that are most important, as they may drive (up or down) the costs of the evaluation. Do you need to focus on the program's implementation to obtain the additional funding that will allow you to focus on several future outcomes? Or is your program far enough along that you need only be concerned with one or two key outcomes? Have aspects of the program changed or will they change as a result of the limited budget?

Another consideration may be a cost-benefit (costs of the evaluation relative to benefits derived from the outcomes, to which you assign a monetary value) or cost-effectiveness (costs of this program relative to other programs) analysis to determine if your program is worthy of the budget it is receiving.

There are multiple ways to conduct a budget-conscious evaluation; the ideas listed here are just a few. What are some other approaches that you would recommend?

‍