by Sheila Matano An article by Lehn Benjamin in the September 2012 issue of the American Journal of Evaluation explored the extent to which existing outcome measurement frameworks are aligned with the actual activities performed by nonprofit staff to ensure positive outcomes for their clients. Benjamins analysis of numerous measurement guides revealed that existing outcome measurement frameworks focus primarily on program activities completed and the changes in the users as a result of those program activities.

Use of an evaluations findings (i.e., lessons learned) and process use (i.e., evaluation use that takes place before lessons learned are generated and feedback initiated) are two of the clearest, simplest examples of the uses for evaluations. (Fleischer and Christie (2009) offer other examples, but recognizing they don't have clear definitions, they wont be discussed here.) By now there is much agreement that there is a great deal of useful information generated during the evaluation process itself, information that could increase involvement and learning.

I must admit Im excited about todays post. Not because it gives us an excuse to indulge ourselves in a lot of unfocused social media (e.g. facebook, LinkedIn, or Twitter) fun, but because of the opportunities and uses these tools can provide program evaluators. Not only have these platforms have provided us, as evaluators, with greater ease in gleaning resources (such as through the American Evaluation Associations facebook page) and communicating with clients and colleagues (via Twitter and our local evaluators LinkedIn group), but we've begun to see programs use of these platforms as an important piece of their evaluation stories.

A May 2012 New York Times Opinionator article reviewed the success of the Nurse-Family Partnership (NFP) home visiting program. NFP is a program in which registered nurses visit with first time, high-risk pregnant women throughout their pregnancy and early motherhood. These nurses teach the women the importance of prenatal care, talk with them about the childcare and child development, and work with the mothers on appropriate parenting behaviors until the child is 2 years old.

Experimental design, a major component of pure (i.e. basic) research is considered the gold standard for research. The premise of experimental design is that a group of participants are randomly assigned to treatment or intervention groups. This random assignment is intended to limit differences between groups. Additionally, the participant and/or experimenter are often blind to which group the participant belongs. With this type of design, you can effectively compare the outcomes across groups at the end of a program.

Typically our blogs focus on evaluation techniques that are specific to program evaluations. But what about the organizations executing the programs? Is there a way to evaluate an organization with the goal of improving how it functions? Coghlan and colleagues (2003) suggest that the appreciative inquiry method can be a constructive approach to evaluating the function of an organization. Appreciative inquiry is used more often in the private sector, but is being seen more and more as an evaluation approach with applications in the public sector as well.

The concept of evidence-based policy was examined in a recent article by Pawson and colleagues (2011).The authors discussed the current trend of “evidence-based everything” and the impact this approach can have on policy making. They examined the example of proposing a policy banning smoking in a car when there are children present and the difficulty in providing conclusive evidence to support the policy. Pawson and colleagues highlight the ongoing theme of their article in the following Donald Rumsfeld quote:

If you want to know if your program’s participants mastered the objectives of the program, the Success Case Method might be for you. (See this report for a summary of this method). This approach involves focusing on those individuals who were either particularly successful or particularly unsuccessful at learning your program’s objectives. The approach is very purposeful, in that you don’t select a random sample of participants; you go to participants at both ends of the learner spectrum to gather information.

Having more demands on a program than a budget will allow was common well _before _the market went into decline. Today, programs have even fewer resources than several years ago, but they are still expected to maintain all documented aspects of their programs, including evaluation, which makes the need for efficiency highly critical. How do you carry out an evaluation on a tightly constrained budget? A common approach is to first have a discussion with the evaluator about the various evaluation designs that may be appropriate for this particular program.