Successful Use of Mixed-Method Design for Project Evaluation

A number of our evaluation projects are community based, and at times grants are funded to unite community agencies, so they can work more closely together to achieve their goals. How do you determine how well organizations are collaborating? How do you improve their collaboration? As a result, we're always looking for evaluation tools that are straightforward and provide complete, easily interpretable results.

In their 2009 study, Cross and colleagues 1 evaluated interagency collaboration using a mixed-method design, which is not an easy task. They approached it from a variety of perspectives, and incorporated qualitative and quantitative data that included network analysis. To completely evaluate this mixed-use approach, they:

• Held focus groups (qualitative) to determine agency classifications and linkages
• Collected ratings of linkage (quantitative): networking, alliance, partnership, coalition, collaboration, and no contact.
• Combined the information using network analysis to create diagrams of the connections with varying thicknesses to represent the linkage ratings
  

I found this to be an excellent approach. Not only did they design an evaluation that would answer their questions, but they found a way to present that data in an easily understandable way. Their final product was a useful set of visual depictions of interagency networks that allowed a range of stakeholders to quickly identify areas of strong and weak collaboration. These figures also gives the evaluator a clear way to provide recommendations for improving partnerships when necessary.

When people think of mixed-method approaches, most assume there will be a section for quantitative results and one for qualitative results. Or they think that the qualitative data will be used to drive the type of quantitative data gathered (e.g., if you're developing a questionnaire and you first hold focus groups to decide what questions to ask), instead of one that brings the two types of information together.

In the Cross et al.study, each type of data contributed equally to the finished product and the result was greater than the individual parts. I believe that maintaining this perspective in mixed-method designs will improve the continued effort to advocate the benefits of both qualitative and quantitative data.

Resources: Cross, J.E., Dickmann, E., Newman-Gonchar, R., & Fagan, J.M. (2009).Using mixed-method design and network analysis to measure development of interagency collaboration._American Journal of Evaluation, 30, 310-329._‍