The sixth meeting of the Education Assessment and Accountability Review Subcommittee was held on Tuesday, October 14, 2008, at 10:00 AM, in Room 131 of the Capitol Annex. Senator Jack Westwood, Co-Chair, called the meeting to order, and the secretary called the roll.
Present were:
Members: Senator Jack Westwood, Co-Chair; Representative Harry Moberly Jr., Co-Chair; Senators Dan Kelly and Ken Winters; Representative Mary Lou Marzian.
Guests: Larry Taylor, Kentucky Department of Education and Mary Ann Blankenship, Kentucky Education Association.
LRC Staff: Sandy Deaton, Audrey Carr, Janet Stevens, and Lisa Moore.
Senator Westwood asked for a motion to approve the minutes of the August 12, 2008 meeting. Representative Moberly made the motion to accept the minutes and Senator Winters seconded the motion. The motion was approved by voice vote.
Senator Westwood introduced Ms. Elaine Farris, Deputy Commissioner, Bureau of Learning and Results Services, Kentucky Department of Education (KDE), Mr. Ken Draut, Associate Commissioner, Office of Assessment and Accountability, KDE, and Ms. Rhonda Sims, Director, Division of Assessment Support, KDE, who reported on the 2008 tests scores for the Commonwealth Accountability Testing System (CATS), No Child Left Behind (NCLB), and ACT.
Mr. Draut began the PowerPoint presentation by defining the state and federal accountability systems in Kentucky. He noted that the federal NCLB accountability requirement has annual measurable objectives that include the percentage of proficient and distinguished students in reading and mathematics and targets for all students and populations of sufficient size. He said other academic indicators in NCLB include CATS classifications in elementary and middle school and graduation rates in high school. He also said NCLB requires a participation rate to test 95 percent of all students and populations of sufficient size and Kentucky meets that goal every year.
Mr. Draut said the CATS state testing system has an academic index that includes the percentage of novice, apprentice, proficient and distinguished students in seven content areas. The nonacademic indexes in CATS includes attendance, graduation rates, including retention and dropouts, and transition to adult life. The ACT high school index also requires administering the PLAN in grade 10 and the ACT in grade 11 for all students with each accounting for 2.5 percent in the accountability index.
Mr. Draut discussed the NCLB annual measurable objectives and the starting points and yearly targets of the percent of students enrolled for a full academic year and scoring proficient and distinguished. He noted Kentucky met 20 out of 25 target goals in 2008, which equals 80 percent of its target goals. A detailed table is included in the meeting folder located in the Legislative Research Commission (LRC) library.
Mr. Draut said based on the NCLB public release on August 5, 2008, 820 schools met 100 percent of NCLB adequate yearly progress goals, which is 70.9 percent of all schools. Other figures reported included: 89.0 percent of all elementary schools met 100 percent of goals; 44.4 percent of middle schools met 100 percent of goals; 33.3 percent of high schools met 100 percent of its goals; and 103 out of 175 school districts, 58.9 percent, met 100 percent of NCLB adequate yearly progress goals. He also discussed the summary of tiers and consequences for Title I schools and districts.
Mr. Draut discussed concordance in CATS reporting in Kentucky. He said multiple changes to CATS in 2007 required a statistical process to link data from the old system used in 1998-2006 to the new one established in 2007-2008. He said concordance is a statistical process used to establish the link. The concordance table relates the scores on similar assessments by lining up the percentile ranks. A school’s adjusted score does depend on the rank order of other schools in Kentucky. The concordance table set in 2007 established the link between old and new CATS accountability index scores. He said the same table was used for the last time in 2008, and content area and individual student scores were not adjusted.
Ms. Sims explained the growth chart for Kentucky. She said schools have different starting points, but all schools have a goal to reach 100 in the accountability index. She noted 594 schools are meeting their goals, 537 schools are progressing toward their goals, and 26 schools are in the assistance category. A detailed adjusted accountability index calculated with the concordance table at each level of elementary, middle, and high school is available in the meeting folder in the LRC library.
Ms. Sims said elementary school students score higher on the CATS test with middle and high school students performing lower on the CATS assessment across the entire Commonwealth. She also said female students tend to score higher than males and African-American and Hispanic students score the lowest in the academic index. She noted that the KDE will continue to work to close the achievement gaps in the test scores among these different sub-groups of students.
Ms. Sims said other items in the 2008 disaggregated academic index show that free and reduced lunch students score lower on the CATS than students who are not approved for free and reduced lunch. Students with English proficiency score higher than students identified with limited English proficiency. Finally, students with no disabilities score higher than students identified with a disability, while students receiving accommodations for their disability score higher on the test than students who receive no accommodations for their disability.
Mr. Draut explained the distribution of scores for the PLAN and ACT tests in 2007 and 2008. He said the scores are placed in ranges of novice low, novice medium, proficient and distinguished and then a formula is used that creates the index for the ACT and PLAN. He said the score range percentages establish a starting point for the high school ACT academic index. Kentucky has an average ACT index of 73.8 in 2008.
Mr. Draut said the composite score for Kentucky juniors completing the ACT is 18.3. He said the ACT provides college ready benchmark scores for the core subject areas of English, mathematics, reading, and science. He said 46% of Kentucky students are meeting the ACT benchmark score in English; 20% in mathematics; 33% in reading; 15% in science; and 10% of Kentucky students are meeting all four ACT benchmarks. African-American and Hispanic students are scoring lower than Caucasian and Asian Americans on the ACT, which is following the scoring trend in the CATS testing.
Mr. Draut said transitioning from 2008 to 2014 will include a data review and appeals will be processed and revised Kentucky Performance Reports will be posted in the fall of 2008. A new growth chart will be created from the nonadjusted 2007 and 2008 accountability indices and the new growth charts will be released in late 2008.
Senator Westwood asked about the dropout rates declining from 5.34 percent to 2.22 percent in 2008. He questioned the validity of the numbers as there has been speculation as to how the dropout rates are calculated in Kentucky.
Mr. Draut said the dropout number used was based on a one-year calculation of how many students dropped out of a school. He said graduation rates are calculated differently so often dropout and graduation rates differ. He noted NCLB uses a graduation rate in its accountability system and Kentucky uses the dropout rate in its state accountability numbers.
Senator Westwood said it is problematic that Kentucky uses a different formula to calculate dropout rates while trying to be in compliance with NCLB. He said this creates problems because it does not provide a clear picture to the federal government of exactly how many students are dropping out in the state. He would like Kentucky to use a formula that mirrors the NCLB formula so the information is useful as to how the state is performing with the nonacademic standard.
Senator Westwood said the increase in the high school writing on-demand score from 55.8 in 2007 to 72.0 in 2008 seems extremely high even using the new standards. He said most of the nonadjusted academic index scores declined in all subject areas except for small increases in the writing portfolio and practical living and vocational studies scores.
Mr. Draut said the writing test that was administered in the fall of 2006 was scored in 2007 using the old model and the old holistic model scoring standards. In the fall of 2007, the students completed the new CATS writing on-demand test and it was scored analytically and posted in the 2008 school year.
Senator Westwood asked if the new scoring standards are lower than in the past or if students are just performing that much better on the test.
Mr. Draut said it is a combination of switching from a holistic scoring model to an analytical scoring model as well as implementing the new standards that resulted in the numbers increasing at that level. Ms. Farris added that the standards on the new test were not lowered.
Senator Kelly praised the KDE staff for their work on the concordance charts. He said only ten percent of Kentucky junior students are prepared to go to college according to the chart showing the percent of Kentucky students meeting college readiness benchmarks on the ACT. He said that he has a hard time understanding why the test is performing so well and the students are performing so poorly. He finds the information non-useful and asked if only two percent of Kentucky schools are in trouble and need assistance.
Ms. Farris said only two percent of the schools needed assistance based on the accountability model that KDE used.
Senator Kelly said he would like to see more useful information, such as how schools are doing that are not performing well, instead of just meaningless indices numbers. He also does not understand why Kentucky is not focusing on the 26 schools that are in need of assistance to get them up to standard although it is well-known that Kentucky’s education problems are much broader than that. He said the type of information presented in the KDE report is keeping Kentucky from really focusing on where the real problems are. He said the KDE data does show the achievement gaps that exist in Kentucky’s educational system, but focus has to be placed on getting more than ten percent of Kentucky students ready for college or the workforce.
Mr. Draut noted that 594 Kentucky schools met their goals, but this was only ten more schools than in the 2005-2006 school year. He said many schools listed in the progressing category are headed for the assistance category and this is a cause of concern.
Senator Kelly said his concern is that the schools are meeting the goals with the accountability system but the students are not prepared for college.
Ms. Farris said it is a school accountability model that analyzes schools across the state, but school districts are looking at individual student performance. She said the KDE is asking schools to look at the core classes that students are taking as this can affect how they perform on the ACT test. She noted some students do not get the core classes needed to perform well on the test because there is not equal opportunity and access for students to get the classes they need in schools and some of it is the student’s choice of class selection. The KDE wants to ensure that all students receive the core classes they need in middle and high school so that access is provided to the aligned curriculum of core content, program of studies, and the ACT standards.
Senator Kelly said Kentucky schools need to analyze their intervention programs with the goal of helping students to perform better and not with the goal of moving the indices up the chart. He said demographics play a part in how many novice students are enrolled in a school and the focus should not be placed on necessarily shrinking the number of novice students.
Senator Winters said he is concerned that half of the high schools in Kentucky regressed in 2007-2008 on their CATS test scores. He asked the KDE their thoughts on why this regression occurred.
Ms. Farris said high schools are a large concern across the nation. She said assistance teams are being assembled in the high schools in Kentucky to look at individual student’s course selections and make interventions when necessary.
Senator Winters discussed the importance of students taking core content classes versus having a high performance teacher in the classroom. He asked Ms. Farris which one was more important for student success.
Ms. Farris said core content and highly qualified teachers go hand-in-hand. She said the core content needs to be delivered to students in an effective manner. She said some students are not getting the core content classes needed because of course selection.
Senator Winters asked if Ms. Farris was comfortable that the core content is adequate in all subject areas. Ms. Farris said she was. Senator Winters questioned the adequacy of mathematics classes as more than half of all students need math remediation in college. Ms. Farris believes it is more of an access issue in that all students are not receiving the core content for whatever the reason.
Senator Winters asked what the legislature can do to facilitate all students getting the core content classes needed to perform well on the ACT and upon graduation of high school.
Ms. Farris said the KDE is continuing conversations with Kentucky middle and high schools to encourage and to communicate how important it is for all students to have access to the content that they are going to be assessed on. She said the students need access to the appropriate content to be ready for postsecondary education and this depends upon availability and rigor of the courses being taken.
Representative Moberly is concerned that the test results are not showing good results and Kentucky is not making the progress that was intended. He said the committee would like ideas on what they can do to provide assistance. He also asked if the KDE feels that declining resources in Kentucky schools are attributing to the problem.
Ms. Farris said declining resources is a problem and does impact student performance and success. She said the KDE is aware that problems exist in middle and high schools across the Commonwealth. She said that 56 assistance teams have been assembled and are visiting schools across the state and having conversations with school districts, superintendents, and principals in an effort to build partnerships to help schools improve and to have an impact on student learning. She noted the KDE is providing assistance so that local school districts do not have to use local funds to receive these services.
Representative Moberly said Kentucky has been having this conversation about improving high schools for the last two or three years. He said he is tired of talking about it and not getting any results. He noted the legislature wants to help the problem and would like guidance or thoughts on what they can do to provide assistance.
Ms. Farris said a team at KDE is looking at secondary education. She said they will have discussions with the committee on what the team is doing and how the committee can support their efforts.
Representative Moberly said the committee agrees on interventions for math and reading and he believes that was reflected in the test results. He said the committee had previously discussed having fewer versions of the CATS test to make the results more reliable and wondered if that had ever been implemented into the system.
Mr. Draut said common items were implemented into all versions of the test; however, the CATS test is a matrix designed test and there are still six different forms that are administered to students. He said this type of matrix test is designed to get a broad picture of the schools’ accountability and curriculum program. He noted that the reliability and validity at the individual student level meets all the required standards that other tests meet.
Senator Kelly asked if students who are having trouble getting access into a class is due to the availability of teachers with the skills to teach the course, particularly in the areas of math and science.
Ms. Farris said it is largely due to the availability of courses based on the size of the school and courses offered, the selection of classes by particular students, including students with disabilities that are not in core content classes on a regular day basis, and expectations of students and the classes they take.
Senator Winters asked if accessibility to courses depends upon the student just as much as the school. Ms. Farris said that was correct. Senator Winters said his concern is about basic skill development in students. He is encouraged that the KDE believes the core content is adequate and that student choice is the reason why students are not receiving the core content courses needed to succeed.
Senator Winters asked if the 537 schools that are progressing is equivalent to meeting the goals of 2014, or are those schools just doing better than they were last year.
Mr. Draut said the 537 schools that are progressing are of concern as they are heading towards entering in the assist zone in later years, while the schools that are moving into the meeting goals zone are flattening out.
Senator Winters asked if enough responsibility is leveraged on students to have them try to perform well on the CATS test. He is concerned that students do not try hard on the test because they see it as a benefit to the school and not a personal benefit.
Ms. Farris said a lot of school districts are doing a better job motivating students to do well on the CATS test, and this motivation is intrinsic. She said motivation is probably not as big a problem across the state as thought to be, but student accountability needs to be communicated.
Mr. Draut said he feels comfortable that the students in the elementary schools are motivated to do well. He is not so sure that there is not a motivation problem for middle and high school students. He said some school districts are beginning to use CATS scores to allow students to move to the next level and also providing consequences or letting students use the scores in some meaningful way.
Senator Winters said the students need other incentives besides motivation to be prepared mentally to take the CATS test. He said his only concern is for the young people of the state and he wants Kentucky students to be pacesetters and not followers.
Representative Moberly said he is concerned about students being motivated to perform their best on the CATS assessment. He noted that students from Madison Central High School responded to questions from legislators that they felt the CATS test was important to the school, and the ACT test was important to them. He feels that the Kentucky Educational Excellence Scholarship (KEES) funding should be tied to students’ CATS scores in an effort to motivate them to do better on the test.
Senator Kelly said he had asked Commissioner Draud to look at intervention programs implemented in Kentucky schools and determine if they are working and can be replicated. He said the summary he received from the KDE was helpful but it did not contain a future plan for next steps. He thinks the literacy intervention has been very successful, but would like to see all students have access to the intervention strategies and would like this topic to be discussed in the near future.
Senator Westwood introduced Ms. Marcia Ford Seiler, Director, Dr. Ken Chilton, Director of Research, and Dr. Deborah Nelson, Research Analyst, Office of Education Accountability (OEA), to present a review of Special Education in Kentucky. Ms. Seiler said her staff would describe the funding, assessment results, and future challenges and issues regarding the special education program. She also asked the committee members for suggested study topics that they would like for OEA to address in 2009. She said the first study identified for 2009 is a mathematics study which will include a review of the core content, intervention strategies for struggling students, assessment results, and teacher quality issues.
Dr. Chilton began the PowerPoint presentation by citing the main data sources for the report as the Individuals with Disabilities Education Act (IDEA) and finance reports and Gifted and Talented (G&T) information produced from the KDE. Some KDE reports that were used included the district annual finance reports, the KDE federal and state grant allocations, and the Support Educational Excellence in Kentucky (SEEK) transportation funding calculation report.
Dr. Chilton said IDEA has a focus on changing the culture of special education with an emphasis on mainstreaming special education students. He also said IDEA believes special education students should be assessed like regular education students.
Dr. Chilton said the process of identification is governed by the Admissions and Release Committee (ARC). Students are referred by teachers, parents, professionals, or state agency personnel staff for evaluation. He said the eligibility is then determined at the school level by the ARC. The identification criteria are more specific in some categories than in others and not all referrals result in special education identification. He said membership on the ARC is governed by regulations, and includes special education teachers, administrators and parents, and can include the child and diagnostic and evaluative staff as well. He noted that research shows that not all school districts employ professional diagnostic staff.
Dr. Chilton said the disability categories include those with a high incidence such as speech and learning disabilities; those with a moderate incidence such as developmental delays, and hearing and visual impairments; and those with a low incidence such as emotional behavioral disorders, multiple disability, deaf-blindness, autism, and traumatic brain injury.
Dr. Chilton said the preschool through 12th grade special education enrollment has grown moderately at 3.2 percent since 2000. He noted the different cohorts of special education however, broken down by age groups, have outpaced the regular education population by growing 16.1 percent over the last 7 years. He said Kentucky ranks about 12 percent higher than the national average in enrollment of special education students. He discussed specific identification trends for each age cohort which is in the meeting materials located in the LRC library.
Dr. Chilton said the total special education revenue in 2007 was $539,366,526. He said the funding formula has SEEK add-on weights based on a per pupil basis with each student identified as associated with an increase in revenue. He noted the more severe the disability the higher the SEEK add-on weight because of the need for more intense services.
Dr. Chilton summarized by explaining the general trends in special education from 2000-2007. He said special education revenues have grown by 32.3 percent; expenditures have grown by 41.8 percent; and K-12 special education enrollment has grown by 10 percent. He noted that some school districts may have coded special education expenditures to the wrong account code so there could be discrepancy in the figures. He said expenditures have outgrown revenues steadily since 2003 and there was a $38 million gap in 2007.
Ms. Nelson explained the student assessment data. She said the topic of assessment and accountability for students with disabilities is extremely controversial. She said many concerns stem from changes associated with federal regulations in the NCLB Act.
Ms. Nelson said CATS and IDEA believe special education students should have participation in assessments, inclusion in accountability systems, and appropriate testing accommodations. She said a testing accommodation can be technology or a process that is intended to allow students with disabilities to demonstrate their knowledge of content without regard to their disability.
Ms. Nelson explained changes in assessment and accountability primarily with the NCLB Act. She said changes were made because federal policymakers believed there were low expectations for students with disabilities and that they did not receive core content in regular classroom settings. She noted the CATS system is a compensatory system and all schools face state sanctions. She said CATS also permits alternate assessments for about one percent of all students. NCLB uses subgroup accountability and only Title I schools face sanctions. She noted NCLB administers an alternate assessment that must be on grade level.
Ms. Nelson said NCLB is criticized as having unfair or unrealistic expectations because the changes were implemented so quickly. It has provided a shock to the system because of insufficient attention to improving teacher skills and giving them access to the type of resources that would allow them to adapt to advanced content.
Ms. Nelson said elementary special education students made the greatest gains overall. She said the data suggests that at least some students with disabilities are capable of performing at higher levels than in the past. She said middle and high school special education students tend to have more trouble reaching proficient or distinguished as content and subject matter on the assessments is more advanced. She discussed specific reading and math proficiency rates on the 2007 Kentucky Core Content Test (KCCT) and these detailed tables are located in the meeting folder in the LRC library.
Ms. Nelson said the KCCT includes regular and alternate assessments. She said many students with disabilities receive accommodations, but the accommodations raise validity and reliability issues. She also noted that the reader, paraphrase, and scribe accommodations are administered by individuals therefore making it difficult to ensure that accommodation practices are standardized in administration across districts and schools, thus affecting the reliability of the assessment results.
Ms. Nelson said Kentucky has far exceeded the federal requirements for participation in assessments. She said NCLB requires a 95 student participation rate, and Kentucky has a 98 percent participation rate. She said 65 percent of students with disabilities take the regular assessment with accommodations; 25 percent take the regular assessment with no accommodations; 8 percent use an alternate assessment; and 3 percent of students with disabilities are excluded.
Ms. Nelson said there has been a steady increase in graduation rates for special education students, although still far below the graduation rates of all students. She noted that dropout rates have steadily declined, but there is a concern noted in the full report on how the dropout rates are calculated.
Ms. Nelson said a major goal of IDEA is to prepare students for employment and independent living. She said national studies are showing transition progress for students with disabilities in employment and in postsecondary enrollment. She said Kentucky does not have trend data on post-goal outcomes for special education students, but the report does report the first wave of the data collection. She said concerns are raised within the report whether all special education students are receiving the transition services, which are required by law.
Ms. Nelson briefly discussed the G&T program. She said 17 percent of all K-12 students are enrolled in the exceptional children program, but the program does not receive direct federal funding. In Kentucky, the G&T program is funded at significantly lower levels than the special education students and many districts supplement with local funds. She said the program was funded at $7.2 million in 2007, which amounts to an average of $62 per pupil identified.
Ms. Nelson said advocates for the G&T students have raised concerns that during an era of accountability focusing on minimum proficiency targets for all students, the G&T students are overlooked in classrooms while performing at levels far beyond proficiency. She said not all districts are able to meet regulatory requirements with the limited funding for the program that is available. Services can vary widely among districts and schools, and data suggests that not all student populations are identified at representative levels in the G&T program. Specifically, disadvantaged students, African-American, and Hispanic students are identified at lower rates in the program than other students.
Ms. Nelson said appropriate identification is essential in referring students for special education. She noted that not all school districts and schools employ those people qualified to make technical diagnostic decisions. She said accurate identification raises important and unique concerns at the preschool level because identification for special education can determine preschool eligibility. She said the biggest qualifier for preschool eligibility is a student who has a developmental delay, but the definitions of this are broad and subject to interpretation. She also said it is difficult to use standardized instruments at this level so it is important to use transparency and fairness in preschool eligibility.
Ms. Nelson said it is important that appropriate services are provided to students who may be struggling in school, but do not have a disability. She said the federal government has highlighted the concern that too many students are referred to special education because of reasons other than having a disability. She said when students are incorrectly placed in special education it can hurt the efficiency and effectiveness of program.
Ms. Nelson said the Response to Intervention (RTI) is a major, federal initiative that is part of the latest reauthorization of IDEA. She said the RTI is a major strategy for addressing the issue of appropriate identification and KDE is utilizing RTI. It is a systematic assessment and intervention made in a regular classroom prior to special education identification. She said challenges to implementation include: district and school resources; access to students at preschool; and financial disincentives in some districts.
Ms. Nelson said expenditures are outpacing revenues at the state level. She said the trend is more pronounced in higher wealth districts and there are no clear explanations why.
Ms. Nelson said that data shows improvement is possible for students with disabilities. There is a discrepancy between expectations and performance. She said there are potential unintended consequences such as the misuse of accommodations and tensions between academic and nonacademic goals.
Senator Kelly asked about consequences of not meeting the NCLB expectations for next year. He asked if standards would be lowered for all students or if Kentucky would have to forfeit the revenue if no changes are made in the federal legislation.
Ms. Nelson said NCLB is in the process of being reauthorized. She said there will more than likely be some changes made in the federal legislation as this situation is not unique to Kentucky. The federal government is also allowing some states to pilot alternative intervention models if they are not on track to meet their goals in 2014.
Senator Westwood thanked OEA for the report and said individual questions may be posed to staff as legislators peruse the report. He asked if the KDE had a response to the OEA special education report. Ms. Seiler said KDE did not have any additional comments and agreed with most of OEA’s findings.
Ms. Seiler encouraged legislators to contact OEA with any questions about the report. She asked if the committee would accept the report, but there was not a quorum.
With no further business before the committee, the meeting adjourned at 3:15 p.m.