The Education Assessment and Accountability Review Subcommittee meeting was held on<Day> Monday, October 27, 2003, at 10:15 AM, in Room 131 of the Capitol Annex. Representative Harry Moberly, Co-Chair, called the meeting to order, and the secretary called the roll.
Present were:
Members: Representative Harry Moberly, Co-Chair; Senators Lindy Casebier, Daniel Kelly, and Ed Worley; Representatives Jon Draud, Mary Lou Marzian, and Frank Rasche.
Guests: Clyde Caudill, Jefferson County Public Schools; Wayne Young and Mike Carr, Kentucky Association of School Administrators; Roland Haun, Kentucky Association of School Superintendents; Cindy Heine, Prichard Committee; Gene Wilhoit, Bonnie Brinly, Kevin Noland, and Scott Trimble, Kentucky Department of Education; and Helen Mountjoy, Kentucky Board of Education.
LRC Staff: Sandy Deaton, Audrey Carr, and Kelley McQuerry.
Representative Moberly introduced Gene Wilhoit, Commissioner, Kentucky Department of Education (KDE) to discuss the Commonwealth Accountability Testing System (CATS) results and his ideas for future adjustments to the assessment and accountability system.
Commissioner Wilhoit referred the subcommittee to the handout that was titled “Data from the 2003 State Release.” He said that the document gives the changes over the last two years. He said there are now five years of data and the database is being built every year to give a clear picture of how the schools in Kentucky are doing.
Commissioner Wilhoit said that over the past two years there was growth in all three grade levels, but more dramatic growth at the elementary and middle school level than at the high school. He said the elementary and middle school levels show progress within each content area and the high school level shows progress in all areas except science and social studies. He said that each grade level has registered yearly gains over the five years of CATS. He said that there are differences in achievement between subpopulations and it is more apparent as the schools raise the index scores. He said that Senate Bill 168 (2002) and the No Child Left Behind Act have put more emphasis on these areas.
Commissioner Wilhoit said that as the process moves toward the goal of 100 in each of the three areas, KDE has asked the schools to lower the dropout rate. He said that there is a legislative target of no more than five percent dropout rates by 2006 and the Kentucky Board of Education (KBE) has tagged to that a continual dropout reduction until 2014. He said that some regulation amendments will be brought before the subcommittee in November before the KBE’s board meeting in December.
Senator Kelly asked the Commissioner to explain the graphs and charts. Commissioner Wilhoit said that the target is a biennial target so there is not a identifiable target for 2003. He said the goal for 2004 is to be at 75.2 for a two-year growth pattern. He said that at this time the state is generally on target for making the next biennial target in 2004. He said that each school will be at a different level depending on how successful they are in this school efforts.
Commissioner Wilhoit said that in order to meet the goal for the 2004 biennium, schools will have to continue to meet the dropout criteria, continue to meet novice reduction requirements, and low-performing schools must dramatically raise performance.
Commissioner Wilhoit said that the academic index shows that Kentucky’s elementary schools are on target for meeting the 2014 goals. He said that reading is dramatically higher than mathematics. He said that the reading programs that have been emphasized at the elementary level appear to be working and have had support behind them. He said the impression at this point is that schools do not have a solid design or curriculum practices in place for mathematics as the reading programs do. He said that in the future, there needs to be a broader base of support in mathematics at the elementary level.
Representative Draud asked why the arts and humanities scores were low and if it is because some schools do not have enough resources. Commissioner Wilhoit said there are several issues and though the trend line is up, the number of teachers for arts and humanities compared but did not compare to other subjects. He said that it was primarily a matter in 1990 that there was not a curriculum in place to address the arts and humanities and there was no expertise in the schools. He said this has developed over time and is starting to change.
Commissioner Wilhoit said there is now a five-year trend with continuous growth over the five years at the elementary level. He said that there are differences in index scores by content area, but at the elementary level there is improved performance at every content area.
Commissioner Wilhoit said that at the middle school level, the reading programs have improved, but mathematics and writing are a concern. He said that when looking at any content areas, there should be improvement over each year. He said that all though the absolute scores are not as high as they are at the elementary level, there is improvement in every content area.
Commissioner Wilhoit said that at the high school level, the performance results have leveled off and the absolute performance levels are not as high at the other two levels. He said that at the high school level there is less of a dramatic or predictable pattern. He said that over the past two years there is a leveling off or a small decline in science and social studies. He said that there is more erratic behavior in the content areas.
Commissioner Wilhoit said that there is a task force meeting that involves the high school educators to re-design high school experiences. He said this does not only exist in Kentucky, but is a problem at the national level. He said that some of the high school level patterns do not show a constant growth line as the elementary and middle school levels do. He said that he thinks there is a problem in the instructional design at the high school level.
In response to a question from Representative Moberly, Commissioner Wilhoit said that this is an issue of dealing with the high school culture and looking at leadership patterns, having access to curriculum at the high school level, and looking at what opportunities the students have at the high school level. He said that some of the other data that is being reviewed shows there is less learning impact than is desired at the secondary level. He said that is it going to require more attention in the future.
Representative Draud said that it is more difficult to bring instructional changes at the high school level. He said that high school faculties have always had more resistance to change than at the elementary level. He said that this is why it has been so difficult to bring about the structural changes and increase learning. Commissioner Wilhoit said he agreed and that the high school is a more complex environment to work with.
Commissioner Wilhoit said that KDE is looking to see that the proficient and distinguished percentages go up and the novice and apprentice percentages go down. He said that the trend lines at the elementary level show a big jump in the last couple of years. He said that it is positive to see that the highest levels are showing progress. He said that middle school trend lines also show progress, but the high school levels are not as apparent.
Commissioner Wilhoit said that the elimination of novice learners is the goal and when looking at the math trend line at the elementary level there is a concern about the percentage of novices when leaving primary. He said that the middle school trend line is still down in math and writing with to many novices. He said that at the high school level there is 30 percent novice in mathematics and that is a major problem. He said that it points to the need for some major initiatives in the future to deal with the mathematics curriculum and the learning process.
Commissioner Wilhoit said there are eleven years left to reach proficiency in 2014. Fourteen schools are above the 100 mark, 82 schools are between 88 and 99 or at the highest level on their recognition points, and 35 schools are below the 55 mark. He said that some of the districts show an overall pattern that is consistent across all schools and this is positive. Commissioner Wilhoit gave some examples of districts based on the accountability index that is above next years goal.
Commissioner Wilhoit said that the most encouraging graph is the free and reduced lunch count. He said that the trend line in Kentucky shows a remarkable pattern that indicates that high wealth districts and high poverty districts are achieving at the same rate regardless of the demographics. He said that poverty and the educational level of adults is not the primary determinate in whether a school will achieve or not. He said the issue is high quality leadership and a set of conditions that will get students to high levels. He said that this supports the principles that the Legislature put in place with education reform in Kentucky.
Commissioner Wilhoit said that other information in the handout is the statewide results. He said that the issue of subpopulation achievement is something that will be an issue in the upcoming years.
Representative Moberly said that based on these and the results in the past years, what is the primary determinate of student success or a set of determinates. Commissioner Wilhoit said that the primary one would be consistent exposure to a highly qualified teacher for every student. He said that it is not enough to have one or two, there has to be a consistent pattern. He said that schools should assure that a child has maximum exposure to a high quality teacher.
Representative Moberly said that the chart on the free and reduced lunch shows the poverty levels of students, but it does not show the resources available to a district. He asked if anyone had researched this to see how a district’s resources would impact the students. Commissioner Wilhoit said that KDE has looked into this and could provide the subcommittee with this information.
Commissioner Wilhoit said that the additional resources have pushed everyone up. He said that there are patterns in other places that have similar economic resources that are getting different results for children. He said it is important to know that resources are targeted to the right areas.
Representative Moberly asked if there would be any recommendations from KDE or from the high school task force before the up coming session. Commissioner Wilhoit said he would push this issue, but was unsure there would be recommendations before the 2004 session. He said that he could not have comprehensive recommendations, but some necessary changes are apparent and would help at the high school level.
Representative Draud asked if other schools were having the same type of success on low socioeconomic schools achieving at a high level. Commissioner Wilhoit said that the Education Trust is doing some work on this across the country and has found that the predominant pattern has been that success is isolated by school or district and not by a statewide effort, though Kentucky has been cited for its statewide system approach.
Senator Kelly said that he has some concerns with the reading program. He said that he feels there should be an evaluation in regard to the early literacy and the early intervention component and gave some recommendations for that. He said that it seems to be an issue that requires determining on a state-wide basis to ensure what has been learned is working and is being implemented.
Commissioner Wilhoit said that the strategies need to be different in the content areas of mathematics and reading. He said that the reading component has brought knowledge in terms of how to organize a school around intervention strategies and the state money has helped with this, although all the schools in the state have not yet received the program. He said that the effort has helped to refine what it is going to take to intervene in the learning process of the low reading students. He said that there are some programs that have a very powerful effect on these students. He said that the federal funds that will be received in 2004 will allow schoolwide strategies around this, but it will not target everyone.
Commissioner Wilhoit said that the question is what needs to be done to take the reading program go to scale. He said that reading intervention should be used as a model for the mathematics program.
Senator Kelly said that there are sets of indicators and if those are present then the results will be high level performance and those are the types of things that will be incorporated into the assessments. Commissioner Wilhoit that the indicators were used in the lowest performing schools to see if there would be improvement.
Representative Moberly said that at this time a quorum was present and asked the secretary to call the roll.
Representative Draud moved for approval of the minutes of the September 22, 2003, meeting and Representative Marzian seconded the motion. The motion was passed by voice vote.
Representative Moberly said the next discussion is on possible future adjustments to the assessment and accountability system and asked Ms. Helen Mountjoy, Chair, Kentucky Board of Education (KBE), to join Commissioner Wilhoit at the table.
Commissioner Wilhoit said he agreed it was important that Ms. Mountjoy be there because the KBE has had conversations concerning these ideas and they will be revisited in December.
Commissioner Wilhoit said that no system is above review and no system should remain static over time. He said that as assessment and accountability is implemented in Kentucky it is important to listen to educators and policy makers and to think about new ways to improve.
Commissioner Wilhoit said there are seven areas that need discussion. The first, providing classroom level assessment tools that are not for state use. He said that these assessments that are provided by the state should be “CATS like” in terms of structure, both multiple choice and open ended responses and should be aligned with the Kentucky Program of Studies. He said this is important because observation in the low performing schools is showing that the tests that are being administered by the teachers do not resemble the kinds of goals that are being set for the students. He said that it is no surprise that when a student is asked a question to apply knowledge on the CATS test, they are not able to do so.
Representative Moberly said that continuous assessment is a basic tentent of reform and will prepare the students for the curriculum. He wanted to know if the failure is in the professional development in the districts.
Commissioner Wilhoit said that there was tremendous amount of energy put into the development of standards in those initial years of reform and there was not an equal amount of energy put into helping teachers understand the level of knowledge that was expected or helping them to develop the types of assessments that were needed. He said that some of the professional development that has been done is not of the quality that would allow them to develop the kinds of assessments that are required.
Representative Moberly said that if the teachers are not doing appropriate continuous assessment then they are not going to be successful. Commissioner Wilhoit said that was correct and that the change needs to be implemented as soon as possible.
Representative Draud said that one of the reasons that his former school system became successful at testing is the emphasis that the administration put on helping teachers understand how to implement open ended response questions. He said that administrators evaluated the questions on a weekly basis. He said that some people in the school districts do not know what open ended questions are. He said that after twelve years, leadership is to be blamed.
Commissioner Wilhoit said that if intervention with state action is not taken in providing some leadership on this immediately, there will be loss of time to turn these schools around. He said that KDE is proposing the development of snapshot assessments in reading and mathematics that would be tied to those two content areas at every grade level. He said the idea would be for the snapshot assessment to be geared toward the predictable level that students should be at that time. He said that this should be available at the beginning of the school year to assess students and plan instruction around it.
Commissioner Wilhoit said that there will be two forms, multiple choice and open ended response questions that can be pulled down from an electronic data base by the teacher. He said that the state’s role in this is to organize this information.
Representative Draud said that there should be an academy to teach the teachers how to do open ended response questions. Commissioner Wilhoit said that there is a plan to develop a professional development program around these assessments so they can be used by each district.
Ms. Mountjoy added that the purpose will not be to teach people how to take a test, but to demonstrate the types of questions that will show students how to integrate knowledge across the curriculum, to problem solve, and show their ability to use the material they have learned in more sophisticated ways. She said that nothing about this is teaching “test taking skills”, it is more “test making skills” for the teachers, so they are able to work with students on ways to demonstrate the use of the knowledge they have attained.
Commissioner Wilhoit said that the purpose is to better diagnose each student, to prescribe an improvement program, and to give the teachers the tools that are aligned with the expectation levels.
Commissioner Wilhoit said that one of the problems in the low achieving schools is that the teachers have a different definition about what proficiency is than teachers in high performing schools. He said they think the students are doing well, when in fact the student’s work does not compare to the performance expectation.
Commissioner Wilhoit said that the second area is the issue of turn-around time for assessments. He said that there are two things around timing that have to be corrected in the current system. First, there is no way to meet the turn-around time frames and hold onto the high quality system we have without changing some testing procedures. Second, the current turn-around time is not good for teaching and learning. He said that the assessment is given in the spring and teachers do not get that results back until the fall. He said that the opportunity for reflection by faculty and placement for students in the spring as well as the professional development in the summer is lost before students come back to school.
Commissioner Wilhoit said there are two areas that KDE wants to explore. First, part of the problem is moving from a paper and pencil test to some type of technology based assessment. He said that with a technology based assessment the results would be immediate on multiple choice questions.
Commissioner Wilhoit said that Kansas and Virginia are the other two states that are moving ahead with this and KDE will be working with them.
Representative Moberly asked what was meant by technology in this sense. Commissioner Wilhoit said that it would be a statewide system to have secure items delivered at school level via computers. He said that students would compute all their answers on a computer screen. He said that when the test is complete the answer would be transferred directly to the teacher and then automatically feed to the state system. Commissioner Wilhoit said that a configuration is four students to one computer. He said that Kentucky has that amount on paper, but he is not sure whether this holds true in every location.
Representative Moberly asked how the writing would be done on the test if computers were used. Commissioner Moberly said that the computer test would be multiple choice only. He said that the second piece of this new system would be Kentucky teacher scoring so that the teachers could score the tests and feed them back into the system. He said there would be more immediate responses, but there would also be problems. He said one of the biggest issues is how to get the professional development in place to assure that these teachers are prepared to do this scoring.
Ms. Mountjoy said that the KBE has not discussed these ideas, however staff has been asked to pursue discussions with other states where these efforts are currently underway. She said that it is more comfortable to try something new if someone else has succeeded at this.
Senator Kelly said that both these ideas are very intriguing, but there is a need to be careful of putting more burden on the teachers that are already pulled to meet the existing testing requirements. He said that it should also raise the questions as to the number of questions and the level of testing. He said that it might be the case that there could be fewer open response questions and more carefully prepared questions.
Commissioner Wilhoit said that all those issues are being looked at and the question is always on how to give an assessment that has assurances to cover core content and is not overburdening the students at the same time. He said that this issue has been taken to the National Technical Advisory Panel on Assessment and Accountability (NTAPAA).
Commissioner Wilhoit said that the third area that is being looked at is the longitudinal assessment component. He said that the items are being developed this year and the pilot will be in place next year, a year ahead of the federal schedule. He said that because there will be a full assessment in grades three through eight in mathematics and in reading, this will give KDE the capacity to look at annual yearly progress.
Commissioner Wilhoit said the fourth area is the issue of exit exams. He said that his personal recommendation is not to move in that direction for a number of reasons. He said that the exit exams have lowered the standards and run into a set of legal issues in the states that have implemented them. He said that the question is how to make sure there is a system in place to assure students move ahead and have the prerequisite skills in order to be successful. He said that instead of using CATS as a graduation requirement or putting a test on top of that, there is thought of using it as a trigger mechanism for other actions that could occur at the school level. He said that there should be a review of each student’s progress by the teachers and administrators at that school. He said that it would be required that additional assessments that would either affirm or refute the assessment of CATS. He said that if it refutes CATS then the student could move forward in the educational process, but if it affirms that the student needs more help it would be required that the school put an intervention program in place for that student.
Senator Kelly said that a test for literacy to determine whether or not intervention strategies need to occur at the transition from primary to the higher grades would be an affirmation of failure and he hoped that long before that is being done there should be a statewide implementation of diagnosis and intervention in the first grade. Commissioner Wilhoit said that this would be a program that would target the students that are moving through the system without the skills they need to be successful.
Senator Kelly said that research shows that if a student is falling behind in the first grade without intervention then by the fourth grade that student is lost. Commissioner Wilhoit said that without those programs this will never be successful.
Representative Draud said that he agrees with the idea to force people to make sure their children get remediation because most of the time there is not cooperation from the parents. He asked what the suggestion would be if legislative action would be taken. Commissioner Wilhoit said that it should be defined as a part of compulsory education and any violation of these laws would be treated as they are now.
Representative Moberly said that if they did not meet the compulsory education law then the student would not move forward. Commissioner Wilhoit said that was correct and for this program to be successful support is needed from the parents and participation from the students.
Commissioner Wilhoit said that the fifth area is that it is time to reform the norm reference test and there are two actions that will be taken immediately. First, to go more toward a schedule of renorming and second, to go toward a cycle of multiple forms on that test so there is less predictability on the questions that are presented year after year.
Commissioner Wilhoit said that the sixth area is the issue of writing. He said the Program Review and Investigations Committee’s report on CATS confirmed that the implementation of writing is the major problem. He said the report called for KDE to develop better guidance to the teachers, more professional development, and to provide stronger feedback to schools on how to improve writing. He said that KDE will move forward on all those recommendations unless there is a mandated change. He said that the upside is that universities are noticing Kentucky students as much better writers, but the downside is that there is disproportion to attention in writing tasks in the portfolio development portion in some schools. He said that this is an issue that needs to be addressed.
Senator Kelly said that the one thing that is heard the most is that the writing portfolio is more burdensome than helpful. He said that when the test results are showing no improvement in the writing area then there needs to be attention directed at what can be done to correct this. Commissioner Wilhoit agreed.
Commissioner Wilhoit said that the seventh area is the issue of high school accountability, and the conversation is on whether there should be end of course exams at the high school level. He said the advantages are that it is incremental rather than terminal and if a student does not show improvement in a certain area, they can go back and retake the course. He said that it would provide a way to develop some consistency in areas of concern if there was an assessment in place that could be used by schools and not by the state. He said it would be a good program for the schools to use to determine whether those students have acquired what is determined as the basic concepts in algebra and language development at the high school level. He said that is the way it is used in some states. He said the use of this program is a policy issue for the board and the legislature.
Senator Casebier said that in respect to the writing component, colleges are saying that freshmen are better writers. He said that he has heard twice this fall from both Dr. Lee Todd, President of the University of Kentucky and Dr. Jim Ramsey, President of the University of Louisville, comments about this being the best class of freshmen they have had. This is also the first group of college freshmen to have had all twelve years of the reform. He said the ones that have concerns about the writing component should have more concern about the impact this is going to have on the students, rather than the complaints of teachers.
Representative Moberly said he liked the idea of having a postsecondary application. He said that should be a good conversation for the P-16 council to have.
Representative Draud said that he found the comment about end of the course exams intriguing because it is going to be very hard to make instructional changes in the high school as far as teachers are concerned. He said that this would create a concept of accountability at a much faster pace or more often. Commissioner Wilhoit said that if this is done right, this could help with student accountability as well.
Representative Moberly asked if there are any other ideas for student accountability such as tying it to the Kentucky Education Excellence Scholarship (KEES) program. He said that some feel students do not have enough incentive to perform. Commissioner Wilhoit said that this is a high school issue and the idea of tying it to KEES is a popular idea with teachers. He said that would motivate high achieving students, but there would still be the issue of the dealing with the students who have no motivation. He said he would support some idea of tying this in with the KEES program.
Commissioner Wilhoit said that in two recent reports students were asked about their motivation on CATS and in both reports students said they had done their very best. He said that it is going to be a complex issue and so far they have came up with nothing other than tying achievement to the KEES program as a motivator.
Senator Casebier said that there seems to be a disconnect between the elementary and middle schools and even more so when the student reaches the high school level. He said that this happens sometimes at the freshman year and some schools have implemented a freshmen academy. He asked if there was information on how many are in place. Commissioner Wilhoit said the department does know and will get that information.
Commissioner Wilhoit said that the schools are reporting some extremely positive results on the freshman academies. Senator Casebier said that goes back to the leadership of the principal. Commissioner Wilhoit said it also brings faculty to the table that are more geared toward the freshman. He said that in most cases they ask for volunteers to go in and carry the middle school concept forward, but put more emphasis on academic achievement at that level and more counseling and support services. He said that parents seem to be positive as well.
Representative Moberly said that the next meeting would be Monday, November 24, 2003.
With no further business the meeting adjourned at 11:50 p.m.