A grant-funded program’s purpose was to help students who are not proficient in math pass a required algebra course so they could graduate on time. Edstar was to evaluate the effectiveness of this program towards this objective.
The program enrolled students whose last names sounded as if they are minorities, because minorities are known to be at risk of not being able to pass math courses. The program director had to set up the program and needed to hire site coordinators. He didn’t know how many students would need to be in this program because the list of names was so long, and he could not be sure all minority-sounding names were actually minority. For example, he thought the name Williams may be either Black or White. So, he didn’t think it is possible to know. He selected the kids whose names really sound minority for sure. He didn’t check that the kids he selected at random had low math scores or possibly had even already passed algebra. He was confident in the research that says that minority students are at risk of failing math. Even if they were currently passing, because they are at risk, they could start failing at any time, he reported to Edstar. (He had to report to Edstar how he selected students to serve and how he aligned services. This was part of what we had to document.)
He used grant funds to allow the girls in the program to have their photos taken at Glamour Shots. He reported to us that the photos would boost their confidence, and confident people pass math more than people with no confidence. (This was a 21st Century Community Learning Center Grant in a high school.)
Beliefs and Skills of the Program Manager
Knowing What Can Be Known: He did not know that he could have known exactly who had already failed algebra, and/or whose math standardized scores were below grade level. This would have told him about how many kids needed this service.
Assumptions about Cause and Effect: He thought creating confidence with glamour shots would raise algebra scores. There is no evidence that these two things are connected.
Evidence vs. Experts: He thought his expert opinion about this innovative service was better than looking at research on programs that had evidence of helping kids stay in school, or helping them pass algebra.
What At-Risk Means: He thought all minority students are at risk of dropping out, and of failing algebra–even if they were already successful in school.
How to Identify Kids to Align Services: He thought he could identify who is likely to drop out by identifying minority students, and that he could identify them by reading a list of last names.
We pulled pre-and post-academic data for the students who were served. More than half of them had already taken algebra and passed it. Of the students who had not already passed algebra, two-thirds had always scored above grade level in math and had not been at-risk of failing algebra. The remaining one-third failed algebra. The glamour shots had no affect at all on passing algebra. The Program Director argued that the glamour shots were effective because nearly everyone in the program passed algebra even though they were minorities. He went on to run another grant-funded program that we evaluated and he enrolled students the same way in it.
Students in the program were viewed as “at-risk” simply because they were in the program. They became less successful in school.