There is clear evidence that Head Start had a statistically significant impact on children’s language and literacy development while children were in Head Start. These effects, albeit modest in magnitude, were found for both age cohorts during their first year of admission to the Head Start program. However, these early effects dissipated in elementary school, with only a single impact remaining at the end of 3rd grade for children in each age cohort: a favorable impact for the 4-year-old cohort (ECLS-K Reading) and an unfavorable impact for the 3-year-old cohort (grade promotion).
They easily defined a control group (though this probably biased the sample toward intensely needy areas):
Selected Head Start grantees and centers had to have a sufficient number of applicants for the 2002-2003 program year to allow for the creation of a control group without requiring Head Start slots to go unfilled. As a consequence, the study was conducted in communities that had more children eligible for Head Start than could be served with the existing number of funded slots.
So what were the results?
Consider 3'rd graders who had been in the program in 4'th grade. Compare them to a similarly selected group of kids (same kind of family condition, race, family income, etc) who hadn't been in Head Start. They say:
4-Year-Old Cohort
- There were no significant differences between the Head Start group and the control group on any measures of social-emotional development during the Head Start year or during kindergarten.
- At the end of 1st grade, impacts on social-emotional development were few and mixed.
- There were two unfavorable findings based on teacher reports of children’s behavior: (1) children in the Head Start group demonstrated moderate evidence of more socially reticent behavior (i.e., shy and hesitant behavior) as reported by teachers, and (2) there was suggestive evidence of more problematic student-teacher interactions.
- In contrast, there was suggestive evidence of less withdrawn behavior for children in the Head Start group as reported by their parents.
- At the end of 3rd grade, parents reported less aggressive and total problem behaviors for the Head Start group children. However, teachers reported unfavorable impacts with a higher incidence of children’s emotional symptoms, less closeness, and a less positive relationship with the Head Start children. Finally, Head Start children in the 4-year-old cohort reported less positive peer relations at school compared to the control group.
Hey, they found that it improved at least in one category of social skills, right?
I took the liberty of changing the sign of the statistical "effect" to positive for good trends and negative for bad ones--their tables are a little hard to interpret otherwise.
There's a slight shift, but nothing to write home about, and the effects look pretty random. Some will be extra high, some extra low--that's the luck of the draw. Nothing much to see here.
Let me emphasize that. If you pick enough different things to compare, you will wind up with some random difference which looks "statistically signficant." But it isn't. My advisor used to say that if you look at 100 histograms, one of them will have a 3-σ peak in it.
OK, how about scholastic measures? Consider 3'rd graders who'd been in the program at age 3, and compare them to a control group. They say:
3-Year-Old Cohort
- At the end of the Head Start year, children in the Head Start group showed strong evidence of less hyperactive behavior and fewer overall problem behaviors as reported by their parents.
- At the end of the age 4 year and the end of kindergarten, children in the Head Start group demonstrated suggestive evidence of better social skills and positive approaches to learning as reported by their parents. Further, children in the Head Start group also continued to show moderate evidence of less hyperactive behavior at the end of kindergarten.
- By the end of 1st grade, parents of Head Start group children reported moderate evidence of a closer relationship with their child than parents of control group children. At the same time, parents of Head Start group children reported (suggestive evidence) a more positive overall relationship with their child than parents of children in the control group.
- There were no impacts on teacher-reported measures of social-emotional development for the 3-year-old cohort in either the kindergarten or 1st grade year.
- For this age cohort, there was only a single statistically significant social-emotional impact at the end of 3rd grade. Children in the Head Start group demonstrated better social skills and positive approaches to learning as reported by their parents, compared with the non-Head Start group.
Zero, zilch, nada. No sign of anything good or bad.
Everybody loves the idea of Head Start, but the HHS numbers suggest that it isn't doing anything except wasting money and talent.
Does anybody have any ideas for something that does work for these kids? If so, let's do a U-turn at Head Start. If not, let's wrap it up as a bad job.
"Effect" is taken from their tables; I didn't try to recalculate it.
No comments:
Post a Comment