(continuing from part 1 introduction)

Current Program Evaluation for Tennessee

               The purpose of any pilot program always includes the idea that a new concept can be tested at a small scale to see if it works before investing much larger amounts of money and effort into a larger scale endeavor.  In other words, one tests the waters before diving in so you can either pull back from bad ideas or adjust imperfect initial plans to increase the chances of success at the larger scale.  To make this process successful, an evaluation of some sort is required.  Jumping in the water without at least sticking a toe in could be a problem if the water is the wrong temperature.  For a governmental program spending millions of dollars, a pilot program allows this type of smaller scale test run before putting 100’s of millions of taxpayer money into a potentially unsuccessful program.  Once the pilot program has its chance to operate for a period of time, then we must take some measurements to decide if it worked.  If we stick our toes in and the water is too hot, we can change our original plunge plan.

               Before jumping into the larger scaled program, we should have reliable criteria that can be confidently measured which tells us whether we can expect to get our return on investment out of a program.  In regard to school choice, given the critical importance of our children’s future success in life and the hundreds of millions such a full scale program could cost (just look at other states in a prior article on site), we need to evaluate the small pilot program that occurred in Tennessee since the Tennessee Investment in Student Achievement (TISA) legislation was passed in 2019 giving first Nashville and Memphis opportunities for school choice before adding Chattanooga.  Of course, we can also look at other states where similar programs have been running even longer, but if we have our own program and plan to inflate it to full statewide scale, we should evaluate if we are implementing a school choice program well or not.  Minimal criteria should include whether the student recipients of our state’s vouchers or scholarships demonstrated improvements in education versus children who remained in our public system.  Parental satisfaction and student mental health outcomes are less objective but still worth considering.  Once the program has run for several years, we could look at whether more students graduate, more go to college, or whether they earn higher incomes.  Tennessee’s program needs a few more years for these last criteria to be measurable, but some earlier short-term criteria could be evaluated if we were given the data.

               So, let’s look at the outcome studies which compare students who received scholarships versus those who remained in the public school assigned to them. To be blunt, we have silence at this point as I can’t find any studies in Tennessee that make that comparison.  The only thing that the state reports is that 91% of recipient parents are satisfied.  That is a very subjective number.  Realistically, a few years is probably needed to see if a student will learn more in a new school such that it can be measured and considered statistically and educationally significant.  Beyond that, some method of comparing the students on an equal “apples to apples” basis is needed.  In most other states, this standard comparison comes in the form of standardized testing, in which all students take the same test and see who scores better.  This seems like a level playing field at first glance, but such a Tennessee comparison has not been done so far. At a second glance, even it has some problems.

               Besides having no testing comparison in Tennessee, three problems arise out of trying to use such a standardized testing comparison.  First, private schools in Tennessee are not required to administer a standardized test to all their students.  Therefore, students moving from public to private schools may not get tested unless a new law requires it.  Second, even when schools do test their students or if they are forced to do so, there are several options for standardized testing.  If one group of students takes one test, and another takes a different test, the results are not technically comparable, still leaving us without an adequate comparison of apples to apples.  Finally, even if all the students are compelled to take the same test, a school which does not employ a curriculum focused on that specific test will find itself at a disadvantage.  Just like a sprinter training for the 100 yard dash will not fare as well in the 2 mile run having trained differently, the student who spends the year in one curriculum may not do quite as well on a test which did not match their recent curriculum.  Unless the public-school group and the private school group utilize the same curriculum for the year, the private school students could be at a disadvantage.  The private schools could therefore be incentivized to change their curriculum to match the test used for comparisons with the public school.

               Without a fairly implemented objective comparison in Tennessee that extends over several years including sufficient numbers of students, we cannot predict whether expanding the program to a statewide form is a good idea or not.  We are left with the one statistic and its hopeful logic.  The statistic promoted by the state on their website sounds encouraging that 91% of parents are satisfied.  What does that mean?  Are they simply satisfied that they qualified for $7000?  Are they satisfied that their child escaped a bully at school or school violence dangers?  Are they satisfied that their child has a better teacher or better friends at school now?  On the other hand, it seems logical that giving parents money to move their children out of a public school with subpar testing scores where they are bullied daily to a private school with higher academic standards would result in better outcomes for the child? If only logic always worked out in the real world.  There are so many factors intervening between the money and the outcome that simple logic is often too simple.  By looking at many other states as I did in another article, the outcomes for the school choice students have not always been as clearly successful as proponents have promised or reasoned.

               Therefore, in regard to objective outcome criteria for Tennessee’s pilot school choice program, we do not have data on which to base a decision.

               Without outcome data, we must turn to other statistics to better understand what the program is doing to or for our future generation.  These numbers are mostly available, but a few gaps exist which could help Tennesseans determine whether or not to take the plunge on universal school choice.  From Tennessee’s website, it looks like over 2400 students have been awarded educational scholarships out of over 3400 applicants.  That should mean that around 1.7 million dollars have been awarded.  The program stipulates income limits for recipients, so we can safely assume that these have been awarded to lower income families that probably would not have been able to afford the private schools.  From there, I don’t see any data on where these children went in regards to private or charter schools.  At least in North Carolina, there is a way to see how many children went to which private schools as some form of accountability and tracking North Carolina State Education Assistance Association. Before addressing the money question next, we return again to the one emphasized statistic, parents are 91% satisfied.  I still wondered what made them satisfied and why are 9% not satisfied?

               Now we come to an important data point, the cost of such a program.  Scaling such a program involves multiplication obviously, but also a conscious decision on how big do we want the program to be.  Even if we want to theoretically multiply and cover the state, at some point we have to draw a line and say we can’t afford to multiply by a bigger number.  At this point, Tennessee reportedly spends about $11,600 (according to Federal report U.S. Public Education Spending Statistics from Education Data Initiative per student in the public school system.  I assume that this number includes TISA’s 2019 increase in spending where 9 billion more was promised towards the education of Tennessee children.  According to TISA’s law codes, 70% of a given school’s funding comes from the state while 30% must come from the local government over the school system.  One of TISA’s guiding principles is that the money follows the child such that a school gets state money based on how many students are enrolled and additional factors regarding the child and certain school district characteristics.  High risk or special needs children mean more money for the school district while school districts with challenging conditions also get a higher funding.  How much beyond this TISA calculation are we going with this education scholarship money?

               For the current school choice pilot program beyond TISA’s foundation, the state publicizes a number of rules to be followed for the student and the schools receiving the funds.  As noted above, income limits apply to the student’s parents.  The private schools which the student wishes to move to  must accept the state money and its requirements.  Currently independent homeschoolers who are not under an umbrella program cannot receive this funding.  Beyond the private school tuition, the sate provides a list of approved spending.  Once those stipulations are fulfilled and a child is awarded the scholarship., $7075 is available to their parents for school choice. 

               Before looking at the proposed scaling of this program, we look back at what we have available in terms of this pilot program.  We are trying to decide if we want to go beyond sticking our toes in the water.  Do we have enough data on the costs, success, and impact of this pilot program to decide if we want to take a larger plunge and how much further of a plunge do we want to take?  Proponents seem to be basing their arguments more on three things.  First, we get an emotional appeal that the situation for our children is desperate, and we must do something before it is too late.  Second, we get an appeal to the logic that of course it will work to move children into supposedly better private schools.  Third, while we don’t have real outcome measures for our pilot program, we can assume that our bigger program will succeed “like” other state programs even though their outcome measures were not consistently improved.  So far, I am not convinced by the current information that is available.

Return Wednesday for Part 3 – Future Proposed Universal Program

Bibliography: