Understanding the "evidence" in "evidence-based" home visiting programs

A May 2012 New York Times Opinionator article reviewed the success of the Nurse-Family Partnership (NFP) home visiting program. NFP is a program in which registered nurses visit with first time, high-risk pregnant women throughout their pregnancy and early motherhood. These nurses teach the women the importance of prenatal care, talk with them about the childcare and child development, and work with the mothers on appropriate parenting behaviors until the child is 2 years old.

The Opinionator article listed many of the impressive evaluation findings from years of research on NFP, including follow up studies showing that children whose mothers had gone through the NFP program were 58% less likely to be convicted of a crime at age 19 than those whose mothers had not been in the program. Reading about the sizeable, long-term impacts of a social policy was fascinating, yet its not what prompted this post.

We are the local evaluator for home visiting programs in Baltimore City. In this role, we worked with non-evidence-based programming (prior to transition due to funding) from the Maternal, Infant, and Early Childhood Home Visiting program (MIECHV) through the Patient Protection and Affordable Care Act. MIECHV will provide $1.5 billion from fiscal year 2010 to fiscal year 2014 to jurisdictions for the implementation of evidence-based home visiting models. This requirement for evidence-based models prompted the Home Visiting Evidence of Effectiveness study (HomVEE); research literature on 22 programs was reviewed to determine which models were deemed effective by statutory standards. Nine home visiting models met this standard.

In our role as evaluators in Baltimore City, we were able to document the process Baltimore used to select NFP and Healthy Families America (HFA) as their two MIECHV-funded evidence-based programs. NFP was the nurse-based program that would be used for highest risk women in the City, and HFA was the paraprofessional-led model that was designated for women with lower risk. Both of these models met the criteria to be considered evidence-based per the HomVEE study.

What peaked our interest in the Opinionator article was the reference to a study (review the study here done by the Coalition for Evidence Based Policy (CEBP). Despite the findings in the HomVEE report, the CEBP study showed that not all evidence-based programs are effective, even with their designation in the HomVEE study. The discrepancy between the two reports is that HomVEE focused on statistically significant findings, where the CEBP report required a further additional specification on the practical importance of the findings, such as sizeable and sustained effects. In other words, the CEBP study further explored eight of the nine evidence based home visiting programs from the HomVEE study and found that only one program (NFP) has the potential to produce important long-term life improvements for both the mothers and children involved.

Given that jurisdictions used the HomVEE results to choose the programs to implement for MIECHV funding, what do you think this means for the support of evidence based programming? How can we be sure that the designation of an evidence-based program is valid? Let us know your thoughts and reactions by leaving a comment below.