Call to Order and Roll Call
The November meeting of the Education Assessment and Accountability Review Subcommittee was held on Tuesday, November 15, 2016, at 10:00 a.m., in Room 129 of the Capitol Annex. Senator Mike Wilson, Chair, called the meeting to order, and the secretary called the roll.
Present were:
Members: Senator Mike Wilson, Co-Chair; Representative James Kay, Co-Chair; Senators Alice Forgy Kerr, Gerald A. Neal, and Max Wise; Representatives Linda Belcher, and Mary Lou Marzian.
Legislative Guests: Representative Derrick Graham.
Guests: Erin Klarer, Vice President of Government Relations, KHEAA; Eric Kennedy, Director of Governmental Relations, KSBA; and Wayne Young, Executive Director and General Counsel, KASA.
LRC Staff: Joshua Collins, Yvette Perry, Janet Stevens, Avery Young, and Maurya Allen.
Representative Kay moved to approve the minutes, seconded by Representative Belcher. The motion passed by voice vote.
Presentation: 2015-2016 Unbridled Learning Assessment and Accountability Results
Commissioner Stephen Pruitt, Kentucky Department of Education (KDE), said that this would likely be the last time that assessment and accountability results would be presented in this form because of the phasing out of “Unbridled Learning” and the forthcoming new accountability system in compliance with the federal Every Student Succeeds Act (ESSA). The new system will return focus to students and quality instruction, with less emphasis on testing. The data shows from the 2015-2016 Unbridled Learning assessments shows is a continuation of an appalling achievement gap, likening the assessments to an autopsy which only shows where things have been going wrong after the fact. The achievement gap represents an opportunity gap that his office is dedicated to finding ways to close.
Ms. Rhonda Sims, Associate Commissioner, Office of Assessment and Accountability, KDE, said that all the data presented is also available in more detail on the KDE website. The website also has breakdowns of the data by school district and comparisons to state-wide data. Some key observations from the data were improvements in graduation rates and college and career readiness rates. Achievement in math scores also has increased. She noted continued concerns with the achievement gap. Also, while the addition of novice reduction calculations impacted overall scores, and the calculation was sensitive to small counts of students, there was no reflection of a starting point regarding percentage of novices and there were similar number of schools with increases and decreases in scores.
Commissioner Pruitt said that his negotiations with the U.S. Department of Education regarding the implementation timeline for the new assessment and accountability system stemmed from some of these observations. In his opinion, the department needs more time to populate databases that can accurately reflect change over time. He wants to make sure that the data they collect is valid and measures what they are attempting to measure in terms of student success. In particular, the current data collection methods addressing the achievement gap have been problematic because they combine so many different areas, making it hard to pinpoint the real source of problems.
Ms. Sims said that the data they have is sensitive to small counts in regard to novices and that rankings are no longer reported because it was an inaccurate depiction of student and school performance. She presented the Annual Measurable Objective (AMO) scores and participation rates which are federally required statistical measures. These showed a higher than required participation rate of 99.7, largely because of high participation among elementary school students, but a slightly below target Graduation Rate of 88.6 (goal of 89.6). She then illustrated how these were calculated using the federal standard formula. Accountability calculations use a four-year adjusted cohort rate. So the number of students (cohort) who earned a regular high school diploma by the end of the current school year is divided by the number of students entering grade nine (starting cohort) four academic years earlier, adjusting for students moving into and out of the cohort in the past four years. The four-year adjusted cohort rate is used to determine whether federal graduation rate goals are met. She showed a graph illustrating a slight increase in the four-year adjusted cohort graduation rate from 86.1 in 2013 to 88.6 in 2016. While these are encouraging trends, the disaggregated rates for minority, free/reduced price lunch, and disabled students are still lagging and are indicators of a continuing achievement gap for these groups of students.
Ms. Sims discussed the Next-Generation Learners categories, starting with college and career readiness scores. Under the Unbridled Learning accountability system, high schools receive points based on graduates meeting college ready and/or career ready indicators. Schools receive one point for each student qualifying as college ready by achieving benchmark scores on the ACT, COMPASS, or KYOTE tests. The COMPASS test is being phased out and will no longer be available on the market. To receive one point for a student as career ready, the student must meet benchmark scores on both a career-ready academic exam (WorkKeys or ASVAB) and on a career-ready technical exam (KOSSA or industry certificate). Students who achieve benchmarks in both college ready and career ready categories earn the school an additional half point. Commissioner Pruitt pointed out that this was an area which has received a great deal of discussion in the ongoing process of developing a new accountability system. There are indications that it would be more helpful to eliminate the qualifiers of “college” and “career” and simply consider students “ready.” This would allow for greater flexibility for students to learn necessary skills in different pathways than the traditional pathway to graduation. Ms. Sims said that the tests have been a very static measure and a more dynamic understanding is currently being sought. To be included in the college and career ready scores, a student must be a graduate first. While the points system has been a motivator for schools to consider career ready pathways as valuable as college ready ones, it may no longer be necessary.
Achievement in the categories of Proficient and Distinguished have also been steady. Schools earn one point for Proficient/Distinguished, a half point for Apprentice, and no points for Novice scores. A bonus half point is also available for more distinguished than novice students. Ms. Sims said that the Gifted and Talented community has been supportive of this because it allows for the acknowledgement of distinguished level achievements. Commissioner Pruitt said that the current desire for students to “test when ready” will likely have an influence on these scores. If a 5th grade student is excelling in mathematics, he may be ready for the 7th grade math test. In that situation, however, he may only receive a Proficient score, where he could have received a Distinguished score on a 5th grade test. He said this is an interesting aspect to consider when viewing student and school scores in the future.
Ms. Sims illustrated the encouraging improvements in math scores among all grade levels and said this likely reflects better understanding of new math standards and implementation. She discussed the achievement gap, which is calculated using two measures. The first involves a “super group” where African American, Hispanic, American Indian, limited English proficiency, free/reduced price lunch (FRPL), and disabled students are counted in a non-duplicated aggregate, meaning that while a student may be both African American and receiving FRPL, they are only counted once in the total. The percentage of the super group scoring Proficient or higher in all content areas is the first measure. The second is the novice reduction targets for reading and mathematics, which was new for 2016. The annual novice reduction targets are set for individual student groups and for the super group, if those groups had 10 or more novice students in 2015. The annual novice decrease was set at 10 percent of that number. Points were awarded based on the percent of the target met, and this process allowed for very fine detail calculation.
Commissioner Pruitt said that the focus on closing the gaps began at KDE before his tenure, but it continues to be his top priority. The goal of education should be to meet students where they are and real innovations are necessary to do that. Novice reduction is largely a focus on improved instruction and he strives to see even more improvement in this area. Ms. Sims said that in looking at data on the super group for signs of improvement, there have been gains in several areas. Among them have been improvements in the number of Proficient/Distinguished scores on reading and math tests among elementary school students; on reading, math, social studies, and writing tests among middle school students; and on reading, math, social studies, and language mechanics tests among high school students. However, the reduction of novice scores in reading, especially among high school students, has largely not met goals. This is prompting questions regarding the instruction these students receive prior to their high school careers. Tables were presented which illustrated the state wide gap regarding percent of novice reduction target met for individual groups as well as the super group. Some groups showed improvement, but the super group did not meet the novice reduction target in elementary reading, middle school reading, or middle school math. High school scores in reading and math had improvement, but did not reach the target for novice reduction.
Ms. Sims said the overall score for Unbridled Learning combines Next-Generation Learners scores and Program Review scores. The weighted score for Next-Generation Learners was 44.7 and for Program Reviews it was 23, for an overall score of 67.7. The combined overall score target of 66.2 was set and locked in March 2016 to provide a target for fall 2016 reporting. Ms. Sims reminded the committee that the accountability classifications of Distinguished, Proficient and Needs Improving are state labels. Federal considerations include the AMO, which is an improvement goal, and the Participation Rate and Graduation Rate, which are data requirements. Federal labels of Distinction, High Performing, High Progress, Priority, and Focus are used for the assignment of federal rewards and assistance. The percent of all schools that are at Proficient/Distinguished level has risen from 56 percent in 2014-15 to 63 percent in 2015-16. The number for high schools has been even higher, influenced by many factors including going to a cohort rate over the last several years.
Commissioner Pruitt said that the first step has been acknowledging that it existed and taking ownership of the problem. This was not a situation that occurred suddenly but is the result of years of ignoring the problem. Now is the time to take any steps necessary to help all the students across the Commonwealth rise and meet their full potential.
Chairman Wilson commended the commissioner for making the achievement gaps his top priority. He said that the General Assembly also needed to take ownership of the problem and reflected on how many students must have fallen through the gaps even during his six-year tenure as a legislator. He committed to helping in any way possible to address and resolve this serious problem. He said that one-third of Kentucky students are not ready for postsecondary education, which is discouraging. Universities no longer consider ACT scores as sufficient indication of college readiness, looking more at persistence, school attendance, and involvement in extracurricular activities. The state’s accountability system should similarly take a whole child approach.
In response to a question from Representative Belcher, Ms. Sims said that the approximately one percent of students with severe disability are counted in the college and career ready statistics if they meet the benchmarks in the Transition Attainment Record which is a checklist designed specifically for their needs. However, under No Child Left Behind federal guidelines, they were not counted for graduation rates and they do not receive the same diploma as their non-disabled peers. This provision should change under the ESSA guidelines and this will greatly improve hiring prospects for these students. One of the initiatives currently underway at the KDE special education department is enhancing workplace networking opportunities for the one percent of students with severe disabilities to assist them in the transition from school to work. Commissioner Pruitt reminded the committee that there is a difference in graduation rates used for accountability and graduation requirements. He does not want to tie Kentucky graduation requirements too closely to the federal ESSA requirements so that Kentucky can retain full control over any future changes.
Responding to another question from Representative Belcher, Commissioner Pruitt said that opting out of assessment is not an option for any Kentucky parent or student, regardless of disability status. He encourages this because he does not want to place any restrictions on what is expected of students because often, when they are allowed the appropriate support, they can excel. Sometimes to the surprise of even their own parents. He stressed that it should be the goal of the department to educate teachers in the best ways to prepare students and through that we should see the best outcomes for all students.
Representative Kay said that the committee heard a report on the achievement gap in their last meeting and that he has been trying to best understand the various suggested approaches to address it. In his opinion, the best positive change comes from the leadership on the ground – teachers. However, teachers in the Commonwealth today are worried about their pensions and spending money out-of-pocket because of lack of resources. These things negatively impact their ability to do the best job they can for their students. Legislators need to do what they can to invest more in educators and the department needs to focus on enhancing and supporting the leadership of teachers. Kentucky has a “pursuit” culture, not an “achievement” culture. The goal should not be to reach a milestone, but to use milestones as stepping stones to lifelong learning.
Representative Graham said that he appreciated the tenacity of the commissioner. He challenged the General Assembly to recognize that the commissioner and KDE cannot do all of the work alone, but will need resources and support that the General Assembly can provide. Investment in education, especially early childhood education, is an investment in closing the opportunity gaps at their source. He encouraged members to continue listening to KDE and to teachers because they are on the ground and in the schools every day so they will be best equipped to know how to move education forward.
Office of Education Accountability’s Proposed 2017 Research Agenda
Mr. David Wickersham, Director, Office of Education Accountability (OEA), and Dr. Bart Liguori, Division Director of Research, OEA, came forward to present the proposed 2017 research agenda. Mr. Wickersham said that the enabling legislation for OEA states that the committee must adopt an annual research agenda for OEA on or before December 1st each year. The annual agenda may include studies, research, and investigations considered to be significant by the committee.
For 2017, OEA proposes preparing the annual district Data Profiles, which are always well received. OEA proposes that the Compendium of State Rankings, previously published biennially, next be published in 2020 and then on a five-year cycle. Any significant changes in state rankings between 2016 and 2010 will be brought to the committee’s attention in the form of memoranda. He highlighted three proposed study topics.
The first study topic is an examination of a study of high school indicators and their association with postsecondary success. High school indicators will include course grades, school attendance, mobility, and test scores on the ACT and K-PREP end of course exams. Measures of postsecondary success would include both postsecondary education data (attendance at a Kentucky college or university, college persistence, credit attainment, college GPA, and graduation) and workforce participation data (wages and participation). This report would also include regional and demographic differences in the relationship between these measures.
The next topic is a study to examine school attendance in Kentucky at the state, district, and school levels, compared to attendance rates in surrounding states. The study will also examine the relationship between chronic absenteeism and school mobility on educational outcomes at the school and student level. School and district level factors will include accountability scores and components, as well as possible impact of attendance on funding. Efforts will be made to measure differences based on factors such as geography, race/ethnicity, special education, free/reduced price lunch status, and English language learner status. The study would also analyze efforts to improve attendance at the school and district levels. Mr. Wickersham noted that, at the suggestion of legislators, the report would examine the impact of compulsory attendance laws, such as those made by Senate Bill 97 of the 2013 Regular Session, on graduation rates, test scores, student discipline, and enrollment in homeschooling or alternative school settings. Additionally, the study will examine the impact of Senate Bill 200 of the 2014 Regular Session, known as the Juvenile Justice Bill, on school attendance and school discipline. Because both of these pieces of legislation are relatively recently enacted, the study may address these topics using school and district level site visits and/or surveys.
The third study is of funding, enrollment, and characteristics of preschool and kindergarten programs at the state and district levels, including the number of districts that are implementing full day kindergarten. In regards to preschool, the study will analyze variation among districts in the percentage of eligible student that enroll in preschool; preschool attendance rates; preschool revenue versus expenditures; preschool students eligibility based on family income versus disability; and kindergarten readiness based on the Brigance screener. The study will also compare Kentucky preschool data to national preschool data on enrollment, funding, student/teacher ratios, teacher qualifications, and other program characteristics and quality measures. Based on data availability, comparisons will be made between state funded preschool and other preschool options available throughout the Commonwealth.
Additionally, this third study will examine full-day kindergarten regarding the costs districts incur above the half-day SEEK funding districts receive per student. A review will be conducted to determine how Kentucky compares to surrounding states in regards to kindergarten funding and instructional time required. Based on available data, this study will explore the relationship between enrollment in full-day kindergarten and academic achievement. Mr. Wickersham noted that this is a difficult data point, but one that OEA will look into and provide what feedback they can.
Chairman Wilson noted that the third study should address early childhood concerns that were referenced by Representative Graham. He also is interested in what impacts Senate Bill 200 has had on school attendance and discipline. Regarding high school indicators associated with postsecondary success, Chairman Wilson asked if OEA intended for the workforce data, including where graduates are working and the wages they are receiving, to be linked with school indicators to determine if school success would have predicted workforce success. Mr. Wickersham said that they would attempt to present that information to the extent that they can capture data.
There being no further questions, Representative Belcher made a motion to adopt the research agenda as presented. Representative Kay seconded, and the motion passed by voice vote.
Representative Kay thanked the OEA staff for their work. With no other business, Chairman Wilson adjourned the meeting at 11:15 a.m.