The Education Assessment and Accountability Review Subcommittee meeting was held on<Day> Wednesday, December 4, 2002, at 2:15 PM, in Room 125 of the Capitol Annex. Representative Harry Moberly, Co-Chair, called the meeting to order, and the secretary called the roll.
Present were:
Members: Representative Harry Moberly, Co-Chair; Senators Lindy Casebier, Daniel Kelly, David L. Williams, and Ed Worley; Representatives Jon Draud, and Mary Lou Marzian.
Guests: Clyde Caudill, Jefferson County Public Schools; Bonnie Brinly, Gene Wilhoit, and Kevin Noland, Kentucky Department of Education; Jim Carlos, DRC; and Dr. James Catterall, National Technical Panel on Assessment and Accountability (via phone).
LRC Staff: Sandy Deaton, Ethel Alston, Audrey Carr,
Jonathon Lowe, and Kelley McQuerry.
Representative Moberly said that 2002 legislation requires that the subcommittee have co-chairs who will have joint responsibilities for committee meetings, agendas, and presiding at the meetings. He said that the legislation requires the subcommittee members of each body of the General Assembly to elect a co-chair. Representative Moberly recognized Senator Casebier who nominated Senator Williams and Senator Worley seconded the motion. The motion was approved by voice vote. Representative Moberly recognized Representative Draud who nominated Representative Moberly and Representative Marzian seconded the motion. The motion was approved by voice vote.
Senator Williams said that he appreciated the statements that Representative Moberly made about KRS 158.647. He said that in his eighteen years in the Legislature he has not seen any committee chair relinquish a committee or offer to be a co-chair. He said it was a magnanimous action by Representative Moberly, and it reflects the thinking that needs to be fostered by the Republicans and Democrats and the Senate and the House. Senator Williams said that on behalf of the Senate he wanted to thank Representative Moberly for his courage and he hoped more people will follow him.
Representative Marzian moved for approval of the minutes from the September 20, 2002 meeting and Senator Kelly seconded the motion. The motion passed by voice vote.
Representative Moberly introduced Commissioner Wilhoit, Kentucky Department of Education, to give a status report on the “No Child Left Behind Act.” Representative Moberly said that Dr. James Catterall, Chair, National Technical Panel on Assessment and Accountability was available by phone to answer questions.
Commissioner Wilhoit said, in describing the “No Child Left Behind Act,” that this is the first time the federal government has been this aggressive in terms of looking out for the educational needs of low achieving students and is holding states accountable for developing programs that assure that full attention is given to all students in the Commonwealth. He said that the U.S. Department of Education recently released the regulations for the implementation of the Act. He said that the regulations will give more guidance since the Kentucky Department of Education (KDE) has been operating from opinion documents and from interactions with the department as they develop the regulations. He said that now the department will write a proposal that complies with the Act that is due January 31, 2003. The State Board of Education, in preparation for this anticipated deadline, has been meeting around a number of issues.
Commissioner Wilhoit said that KDE will comply with the annual testing requirement in grades three through eight in reading and in mathematics. The board will include two additional tests in mathematics and two in reading. The federal law requires that those assessments be aligned with Kentucky’s core content. He said that the department thinks that an augmented assessment can be developed that is aligned with the other programs that are currently in place. The new tests do not have to be in place until 2005-2006, and he believes they will be in place a year before the deadline.
Senator Kelly asked what is meant by requiring the additional test to be aligned with the core content and why this is important rather than the purchase of an off-the-shelf test. Kevin Noland, Deputy Commissioner, Kentucky Department of Education, said that the federal law requires that the assessment have multiple measures and that it be aligned with the state’s core content or learning standards. Commissioner Wilhoit said that the attempt is to make some statement of what is important in reading and in mathematics and make sure the assessments are measuring these standards.
Senator Kelly asked if the alignment issue creates any problem with comparing Kentucky’s results with other states’ results. Commissioner Wilhoit said that it is possible for one state to set a very high standard for learning in reading and mathematics and another state to set a low standard and develop assessments that measure those, with the results being very different. He said that trying to make comparisons across states is very difficult with this system. He said that there was an attempt to come up with some agreed upon assessment that could be applied across the country, but it was not something that Congress would pass. The compromise was for a state to develop its own assessments and hope there is good faith in doing so, then measure all the states against the National Assessment of Educational Progress (NAEP).
Senator Kelly asked if this would require an additional round of testing. Commissioner Wilhoit said that Kentucky already participates in NAEP in grades four and eight around the country. Commissioner Wilhoit said that the U.S. Department of Education hopes that the NAEP results will put pressure on the states to develop rigorous assessments.
Representative Draud said that the tenth amendment of the U.S. Constitution has reserved the right of education to the states and there is supposed to be fifty different state educational systems. He said that one instrument to measure all states would destroy the states’ authority for education. He said that the NAEP instrument is an attempt to look at comparisons, but it is not a national test.
Commissioner Wilhoit said additional testing will provide a solution to the issue of longitudinal testing that has not been available in the past. He said that most of the schools are already doing assessments in the years that the department does not assess. This is important for the schools’ instructional purposes to determine where the students are and to develop interventions. He said that he thinks there will not be much additional testing and that he has asked for a report from the contractor to see which schools are doing off-year testing. He said that there will not be any additional burden on the schools that already have off-year tests. He said that the additional testing gives an annual progression of tests in reading and mathematics that will allow the longitudinal study to be put in place. He said that at some point the department will have to figure out on how to fold the new information into the accountability system. The teachers have said that something important is being left out of the current system and that the teachers should receive credit for bringing a child from one level of learning to another over a period of years. The new information will help provide that type of information and will allow decisions for the accountability system. He said that the developing of test items is starting immediately with a pilot program ready by 2003-2004 and fully implemented a year before the deadline in 2005-2006.
Commissioner Wilhoit said that the second issue is that the federal law has consequences for schools that do not progress for two years. He said that the state must have some intervention strategies for schools that receive Title I funds or money that is received for high poverty students. Senator Kelly asked whether the sanctions are for nonperformance in a targeted population, the Title I population, or whether they are across the board. Commissioner Wilhoit said that in Kentucky that is a minute point because most of our schools are classified as poverty schools and every district except one qualifies. He said that within a district you can have Title I and non-Title I schools. The federal sanctions apply to Title I schools, but Kentucky must have a plan to deal with all schools and a plan that makes sense across the board for the Commonwealth.
Commissioner Wilhoit said that old information has been applied for purposes of the new law in determining which schools are unsuccessful. He said that some schools have already been identified as schools that need improvement and are Level III schools. The federal government requires the state to intervene in those and intervention strategies are already in place that meet the requirements of the “No Child Left Behind Act.” He said that there are no major issues other than a few transitional issues in getting the system operating. Kentucky has school audits, the Highly Skilled Educators program, and the Commonwealth School Improvement Funds going to those schools that need improvement. Those interventions meet the criteria with one exception. The federal law says that after three years of not meeting goals there must be some type of defined intervention strategy on the part of the district. He said that in the last legislative session there was a bill that proposed action on the part of boards with schools that were not making progress. The bill would have removed decision-making authority from a school that has not turned around in three years after all the intervention strategies have been put in place. He said these are the issues associated with the provision that allows parents to transfer students and requires supplemental services be provided for students in those failing schools.
Commissioner Wilhoit said that all schools in the state must meet adequate yearly progress and a system has to be put in place to determine how to do so. Kentucky now measures every two years.
Senator Kelly asked about the requirements for measuring progress and if there is a limit to measuring progress on a class cohort, or a school building cohort, or a district. He asked if one could actually identify subcategories within a school building such as non-performing readers or students with math deficiencies and then show progress in that cohort.
Commissioner Wilhoit said that the federal government wants to see overall improvement of all students in that school in two content areas, reading and mathematics.
Senator Kelly said that if a student is performing at a high level, then improvement is not necessarily going to be shown.
Commissioner Wilhoit said that is one of the issues that is being raised since Kentucky already has a system in place that has high performing schools and the department does not want to go back to the old system.
Senator Kelly said that there could be a school that is performing high and in that school have some kids that are failing.
Commissioner Wilhoit said each state has to look at student performance in reading and mathematics and by various subpopulations performance in these areas.
Senator Kelly said that when the review was done in the schools that were doing well, they were using testing to identify students that were having trouble in certain areas and then intervention strategies were put in place to help the students. He said that if you are identifying students having trouble and then helping them out then progress is being measured.
Commissioner Wilhoit said that the federal government is going to force schools and states to look at those subpopulations and to do something about it. He said that this law will force more precision in those subpopulations in reading and mathematics.
Senator Kelly asked if there is enough flexibility in the federal act to allow that type of scheme to show progress.
Commissioner Wilhoit said that the federal government is interested in direct information by each subpopulation in those two content areas.
Senator Kelly asked if a non-performing student is an adequate subpopulation.
Commissioner Wilhoit said no, that it has to be an overall performance of the school.
Dr. James Catterall said that one way to respond to the question is to be precise on what the federal law requires. He said that if the percentage of all students in a school designated as proficient goes up enough every year, by the year 2014 all students will be proficient. He said that if there is a fifty percent increase over ten years that would be five percent per year and it would be one standard. Each subpopulation is held to essentially the same standard. He said that if a subpopulation of high poverty students in a school were twenty percent proficient, they would be required to gain more every year. He said that if any subpopulation does not meet an annual gain target or if the school does not meet the annual gain target, then the school is flagged and is under-performing. He said that every group and the whole school has to make the associated gain. He said that a way to approach getting all students to proficiency would be to be diagnostic and to look at groups that require more needs, but the federal government does not require this and does not identify them as a subgroup.
Representative Moberly asked Dr. Catterall if the subgroups have to come up to the percentage of the school as a whole. Dr. Catterall said that the subgroup has its own starting point and a straight line heading out to the year 2014. He said that line will be steeper than the line for the entire school and it has to make progress on its own line. He said that it is possible that the two groups will meet in 2014 and all students in each subgroup along with the whole school will be proficient.
Commissioner Wilhoit said that as the lines of subpopulations are being drawn in the state, the group of students that are in the greatest need for improvement are students with disabilities with the gaps being in different content areas and at different levels. He said that the first issue is maintaining the two year averaging, secondly there needs to be a way to roll in accountability for mathematics and language with subpopulations and then merge this information with Senate Bill 168 (2002). If this is done there is a good possibility for this all to come together. He said that the other issue is whether Kentucky can maintain the individual growth line for the schools or whether there will have to be a single line adopted for improvement off the average of all the schools, which is what the federal government is proposing. He said that this is a major issue and contemplates an averaging process in setting a straight line for all schools in the state of Kentucky. He said that this may create some problems. First, it removes incentives for progress for some high performing schools. Second, it may create a disincentive for the lowest performing schools because the goal is so high in the short term that there will be disillusionment and these schools may drop out of the system.
Representative Moberly asked Commissioner Wilhoit to elaborate more on how this system would work. Commissioner Wilhoit said that the federal government is proposing a system of calculation that by content area would take an average of all schools. He said that all schools and subpopulations would have to meet at that point and would be drawn to proficient by 2014. He said that it would remove the individual charts in place for all schools at this moment and would set the new line in place. He said that this is a major departure from the system that is in place now.
Representative Moberly asked if the state board made any statements about this issue. Commissioner Wilhoit said that the state board wants to continue with the current system and will negotiate with the federal government to apply some sort of a system across the board. He said that could mean that if the federal government sticks to the requirement, then it could be possible to maintain the current accountability system and come up with some sort of minimal performance level that has to be met in order to qualify for federal funds. He said that the level might be higher for some schools than the current chart.
Representative Moberly said that is a major requirement of the current accountability system and it is over reaching. Commissioner Wilhoit said that from the federal government’s point of view they were frustrated at other states that did not comply.
Commissioner Wilhoit said that KDE recently met with the senior leadership in the U.S. Federal Department of Education in Washington, D.C. Mr. Noland said that Kentucky is one of the states that is closer to compliance with the “No Child Left Behind Act” than many other states and is much further along with a well developed accountability system. He said that the one year anniversary date of the passage of the bill is coming up in January and they are looking for some success stories such as Kentucky’s.
Representative Moberly introduced Dr. Ken Henry, Director, Office of Education Accountability to present a progress report on a proposal for an external validation of CATS scores. Dr. Henry said he had talked with Dr. James Catterall and Dr. John Poggio, National Technical Advisory Panel on Assessment and Accountability (NTAPAA) about their views on the generalization of the CATS scores and ways to get at external validation. He said that the plan in the handout proposes that multiple measures be used over a course of three to five years to collect the data, and then put together an analysis to look at gains on several external measures to be compared back to gains on the CATS and particularly on the core content test. He said that they emphasized the importance of looking at overlap in terms of the content measured by the various tests that are involved in this validation process. He asked Dr. Catterall to speak about the technical parts of this process. Dr. Catterall said that one of the issues that NTAPAA has stressed is that no one study has a firm grasp on an issue. He further stressed the necessity to understand an area by using multiple methods, including research and reports.
Senator Kelly asked if there is some external validation work going on with NTAPAA as part of the work with KDE. Dr. Catterall said that the department’s validation agenda has not been directed to the type of external validation that is described in the proposal.
Dr. Henry said that he had reviewed a number of studies that did the external types of validation from a number of different researchers and those were the basis for the discussion that was held with Dr. Catterall and Dr. Poggio. Dr. Catterall added that one of the general issues that has not been addressed is the degree to which the content of a given external measure overlaps with content that is begin measured on a component of the CATS test.
Senator Kelly asked if OEA was going to do an external validation study or report and if there is a proposal on the table. Representative Moberly said that the subcommittee decided that it would determine whether or not that should be done. He said that the subcommittee instructed Dr. Henry to seek the advice of NTAPAA on how it would suggest it should be done.
Senator Kelly asked if there is any concrete advice for a proposal. Representative Moberly said that the information that Dr. Henry is providing to the subcommittee is to give an idea on whether to direct him further to give a proposal on an external validation to the subcommittee. He said that the first step was to get his information and findings from NTAPAA.
Senator Kelly said that he thought that Dr. Henry should give the subcommittee his proposal on how to proceed based on his discussions with NTAPAA. Representative Moberly asked Dr. Henry to develop a proposal on external validation. Senator Kelly said he would also like to see a proposal on an external validation, on how it could be met at a minimal level or what we might need at a more sophisticated level if it is warranted.
Senator Williams said he thought that the Department of Education (DOE) should be asked for its position and then have Dr. Henry give his position separately, that way there will be two different views. Senator Kelly said that after one year of reports the statutory requirement has not been met to have some type of external validation.
Senator Williams said that he was concerned about how independent the subcommittee might get and has always been concerned about the existence of the OEA office. He said it appears that at some juncture it is operating in an executive branch function.
Representative Moberly said that the subcommittee directed Dr. Henry to seek the advice of NTAPAA who is advising on the current assessment and accountability system. He said that there could also be advice sought from the DOE as to what would be the best way to proceed with an external evaluation. Dr Henry said that recommendation will be taken and OEA will value the advice as it goes through the process. He said one of the important things is that DOE is required by statute to engage in a validation process on validation studies related to the assessment program.
Senator Williams introduced Ken Myers who is the White House liaison for the U.S. Department of Education. He said Mr. Myers was in Frankfort to inform the new legislators on the “No Child Left Behind Act.” Mr. Myers said that he would be the contact person for Kentucky.
Representative Moberly asked Dr. Henry to present the annual finance report, which is a statutory requirement. Dr. Henry said that KRS 7.410 requires an annual finance report. He said that the SEEK base has increased in 11 years by 33.02 percent. He said that OEA looked at the issue of funding equity which was the basis of the original Council for Better Education suit. He said that it was looked at in several ways, first being vertical equity, which is defined as unequal treatment of unequals and horizontal equity, which is defined as equal treatment of equals. He said that this is looked at through coefficients of variations and by quintile. He said that there is a very small variation within each quintile. He said that the coefficients of variation started out at 0.17 in 1990 and in 2001 was at 0.11. He said that in terms of state and local dollars the funding gap has closed. It was $1,171 in 2001 and was at $1,558 in 1990. He said that in terms of federal dollars the gap is not so wide, it started in 1990 with $1,380 and today the gap is at $833.
Dr. Henry said that when the 41 districts that were grandfathered in because their local tax rates were higher were taken out of the calculus, then all five quintiles come closer together. Representative Moberly asked Dr. Henry to explain the grandfather clause. Dr. Henry said that when the SEEK formula was enacted there were certain districts that were levying beyond their maximum Tier I rate. He said rather than force the district to levy lower taxes and have less local revenues available to them they were grandfathered in and allowed to levy at the rate they currently had.
Senator Williams asked if there was reason to reevaluate those grandfathered districts. He asked if the other districts had moved closer to the level of local effort to those that were grandfathered.
Dr. Henry said that some districts were leaving money on the table by not levying the maximum Tier I rate. He said that a number of districts are levying the maximum Tier I now and getting the full state equalization that was available in 1994. He said that another thing that is looked at is the actual levied tax rates. He said that there is a maximum rate that can be levied and the maximum Tier I rate that is not subject to recall. That amount stayed the same from 1997-2001. The levied amount not subject to recall was higher going from 71 in 1997 to 80 in 2001.
Dr. Henry said that one of the things that has been reviewed is the notice of the assessment increase and the maximum four percent growth. He said that the SEEK money that comes in is reduced by the maximum Tier I rate for growth and assessment. Dr. Henry said that permissive taxes are not part of the SEEK calculus, but does effect the amount of local funds that go into calculating the funding available for the local district. Dr. Henry said that the percentage of median household income on real estate taxes paid is found to be a balanced approach.
Dr. Henry said that one of the problems in dealing with adequacy issues is the lack of a clear definition of adequacy. He said that the Council for Better Education has commissioned an adequacy study and OEA will be following that study. He said that other states that have had finance issues challenged in court are New Hampshire, New Jersey, Ohio, Wyoming, New York, and Arkansas. He said that in 2001 Florida passed legislation that required each district to undergo a Best Financial Management Practice Review every five years. The review identifies opportunities to save funds, improve management, and increase efficiency. He said that it is an estimated cost of $586 million in cost savings for districts if they adopted suggestions from reviews completed since January 1, 2002.
Dr. Henry said one of the unmet financial needs voiced by districts is for English as a second language (ESL) students. There is a migrant population in the state that has grown 25 percent since 1998. He said that there is no additional funding available for ESL and 47 percent of districts have ESL students.
Dr. Henry said that the at-risk component has to do with the free lunch count. He said the Department of Agriculture found National error rates ranging from 29 percent to 70 percent. He said that in 1999 Census figures estimated that 29 percent of students were ineligible for free lunch. He said that at a 29 percent error rate, 73,011 Kentucky students are counted at-risk that may not meet the standard.
Dr. Henry said that the exceptional children program served 82,009 students in 2000-2001. He said that represents $277,108,131 in addition to the basic SEEK funding and approximately 28 percent of total SEEK funding. He said that 15 percent of ADA statewide receives some sort of exceptional children funding. He said that there are 13,918 personnel providing services in the state. One of the issues is that federal funding has never reached the 40 percent of costs that was originally promised.
Representative Moberly said that the subcommittee would accept the report from OEA.
With no further business the meeting adjourned at 3:55 p.m.