The Education Assessment and Accountability Review Subcommittee (EAARS) met on Friday, February 10, 2006, at 10:30 AM, in Room 129 of the Capitol Annex. Representative Harry Moberly Jr, Co-Chair, called the meeting to order, and the secretary called the roll.
Present were:
Members: Senator Jack Westwood, Co-Chair; Representative Harry Moberly Jr., Co-Chair; Senators Dan Kelly and Ken Winters; Representatives Jon Draud and Frank Rasche.
Legislative Guest: Senator Katie Stine
Guests: Dr. Robert Tarvin and Mr. Bernie Sandfoss, School Facilities Construction Commission; Mr. Jim Frank and Mr. Marshall Lowe, Green County Board of Education; and Mr. Ben Hicks, CTB McGraw-Hill.
LRC Staff: Sandy Deaton, Audrey Carr, Jonathan Lowe, Janet Stevens, Anissa Johnson, and Lisa Moore.
Representative Rasche made a motion to approve the minutes from the December 16, 2005 meeting, and Representative Draud seconded the motion. The motion was approved by voice vote.
Representative Moberly introduced Ms. Linda Shields, Finance Officer, Boone County Schools, and Dr. Fred Bassett, Superintendent, Beechwood Independent School District, who gave a response to the Office of Education Accountability (OEA) study of the School Facilities Construction Commission (SFCC). Dr. Bassett urged the committee members to pass over voting on the SFCC report in order for more school districts to have a chance to review and make responses. He also asked the committee members to take a good look at the overall Support Educational Excellence in Kentucky (SEEK) formula and compare school districts by the total local and state revenue they receive versus only using property tax assessments. Dr. Bassett said another study should look at districts that are growing rapidly versus the districts that are not growing.
Senator Westwood discussed the SEEK formula and the inequities of the distribution of wealth and the total dollars that school districts have to spend. He asked Ms. Seiler, Deputy Director, OEA, to respond to Dr. Bassett's recommendation for the committee to pass over taking any action on the SFCC report.
Ms. Seiler said that Dr. Bassett's issues are relevant and should be addressed, however, the Kentucky Department of Education (KDE) is currently conducting a study of the SEEK formula, and the OEA did not have directives to study the SEEK formula in the approved workplan for the SFCC study plan.
Ms. JoAnn Ewalt, Analyst, OEA, noted that the purpose of the SFCC study, that was approved by the subcommittee, was to make recommendations to strengthen SFCC and not to deal with the SEEK formula. She did not see any reason to delay action on the report.
Representative Draud asked who was doing the SEEK study for the KDE. Ms. Kyna Koch, Associate Commissioner, KDE, said a Request For Proposal (RFP) was distributed, and there was one respondent, and a contract has been made with Dr. John Augenblick. Representative Draud affirmed that Dr. Augenblick was one of the original people to study the finance system in Kentucky.
Senator Westwood asked if the SFCC considered handicapped-accessibility issues and if so, how is it determined. Ms. Koch said the cost of handicapped-accessibility is included in the total costs of projects put forward by local districts in their SFCC plan, but it is not given any specific weight.
Senator Westwood asked how to deal with a situation for a school in a high growth district, but not a property wealthy district, that needs to expand, but cannot because of the funding for the formula. He said certain schools need to comply with the American with Disabilities Act (ADA) laws, but do not have the money to accomplish the goal. Ms. Koch said Mr. Gene Wilhoit, Commissioner, KDE, would address the issue, but the entire SFCC program is underfunded in Kentucky, and the underfunding certainly affects those districts with high growth. Senator Westwood asked if there would be recommendations made to address the issue. Ms. Koch said the KDE agrees with the OEA study that the funding mechanisms in place are sufficient, but are not adequately funded. Ms. Seiler also said that school districts need to adequately express their ADA needs on their facility plan to ensure that this goes to their unmet need. Ms. Koch said the KDE is not making any recommendations for changes in the overall structure of the SFCC at this point.
Representative Draud asked when Boone County last raised property taxes to raise money for their school district. Ms. Shields said that Boone County took advantage of the opportunities to increase their taxes through the growth nickel two years ago. Ms. Shields added that Boone County is so close to the tier two limit, that levying above the limit would not raise additional taxes. She said Boone County's property tax has increased aggressively over the last ten years, and that is the factor that is keeping the county's tax rate down. Boone County is collecting additional taxes because the tax rate is low, but property taxes are increasing.
Representative Draud said that tax revenues do not meet growth needs. Ms. Shields said the restriction of the equivalent nickel to only be spent for building funds is taking money away from the county's operating funds, and the flaw within the SEEK formula is taking away Boone County's state funds. Boone County is the second lowest in per pupil spending in the state, and approximately half of the schools there are over capacity. She said the operational funds are not there to build the new schools that are needed.
Representative Draud said the issue of the handicapped facilities should be dealt with in the formula. He said some schools obtain waivers in order to avoid making their schools equipped for the handicapped, and districts should be able to get money for this issue from the formula.
Senator Winters asked how many category four and five schools are identified in the state. Ms. Ewalt said the numbers reflected in the study were based on the KDE's assessment as of April of 2005, and the Division of Facilities Management has since updated the information. Although the number of these schools is a moving target, there are currently 183 category four schools, and 17 category five schools.
Senator Winters asked if the criteria used for rating schools was all given the same weight. Ms. Ewalt said the system of category rankings for schools is a very good indication overall of the condition of Kentucky schools, but for any specific school, it may or may not be what another evaluator would rank it because the criteria are fairly broad.
Senator Winters was pleased to hear that Ms. Ewalt recognized that there were evaluator inconsistencies, and that the ranking was a guideline, but not the sole factor in determining the funding of a school project. Ms. Seiler said the OEA recommends that the ranking system should be reevaluated if it is going to continue to be used for funding purposes. Senator Winters asked if it was being used currently to make decisions about funding school projects. Ms. Ewalt said in 2003 and 2005 there were specific programs that targeted category five schools for funding if they met certain criteria. She said the report provides findings and recommendations that show superintendents are trying to plan and fund maintenance, but the majority of plans cannot be fully funded, which tends to make category three and four schools reach the category five status because they were not maintained.
Senator Westwood asked if anyone had considered the design build concept to reduce the overall expense of the SFCC fund. Ms. Ewalt said that was not considered because it was outside the study proposal that the subcommittee approved for the OEA. She said some superintendents have mentioned the concept as a way they save money, but it was not an area that OEA was asked to review. Senator Westwood said the design build concept is something he would like to see analyzed as a way to save money.
Commissioner Wilhoit gave a response to the OEA study report. He said the KDE is conducting a study to look at the three-prong approach. He also said there are two extreme circumstances in the system that need to be dealt with. They are: 1) extremely fast growing districts; and 2) local revenue abilities. He said they are both unique situations that need to be addressed. He said the KDE feels the three-prong approach is generally solid, and should be defended in its overall approach.
Senator Kelly asked what the three-prong approach consisted of. Commissioner Wilhoit said it is general distribution based on size, general distribution based on need, and general distribution based on wealth. Senator Kelly said it is time to get back to funding the overall plan. Commissioner Wilhoit said these external kinds of unique situations are not going to be resolved through the regular formula.
Representative Moberly asked about merging the smaller districts that are not going to grow with surrounding districts. Commissioner Wilhoit said the efficiencies have to be brought to the table and looked at, but on the other hand, local education delivery needs to be kept fairly close to the students. It is a very difficult subject to discuss, and is almost a county-wide dilemma.
Representative Moberly said that if it is known that category five schools get funded, people will let their schools fall into a category five to get into an urgent needs trust fund, and not get the funding as they should through the regular formula. He asked how to address this issue. Commissioner Wilhoit said that the General Assembly has pretty much eliminated category five school buildings in the state, and the few left are transitional buildings. He said he is not an advocate of continuing to fund the urgent needs fund as the problem appears to be remedied.
Senator Kelly asked if new strategies needed to be looked at for the unique growth situations, while continuing to fund the three-prong approach. Commissioner Wilhoit said the biggest fear for school districts is that resources used for funding the overall program would be taken away to fund these unique needs in a few districts. He said it is a classical policy dilemma for everyone.
Representative Moberly said he would like to hear Commissioner Wilhoit's recommendations on how to handle these unique situations as both chambers are in the process of working on the budget.
Representative Draud said the two extreme scenarios that Commissioner Wilhoit discussed could be addressed in a formula rather than having the General Assembly to try to look at individual districts and causing an uproar in the political world. He said most superintendents would be more comfortable with a formula to address these two issues and to treat all districts in a fair manner. Commissioner Wilhoit agreed that the two extreme situations could be addressed in a formula.
Senator Westwood said he does not believe the three-prong is approach is working, particularly the wealth component. He said Senator Stine has a bill that addresses the wealth aspect, but the report of Commissioner Wilhoit contradicts her legislation. Commissioner Wilhoit said his point is that a formula that includes wealth as a factor is a solid program, but whether assessed property or a broader tax base is used as the base for determining the wealth factor can be determined. Senator Westwood said he believes in the three-prong as well, but he believes in looking at the total wealth of the district, both state and local. He said this is not happening equitably across the state. Commissioner Wilhoit said the issue of using just assessed property value, or using permissive taxes as part of the equation of figuring the wealth of a district, needs to be addressed.
Representative Moberly asked Senator Stine if she had any questions for the panel members. Senator Stine said Kentucky's analysis of wealth in the three-prong approach is not equitable because it only considers property taxes and not revenue in lieu of taxation, federal grants, and permissive tax assessments. She said she supports an analysis and revamping of how the wealth is assessed.
Senator Stine said she does not agree with the OEA report that weights would significantly reduce the ability of SFCC offers to address unmet facility needs because the weights would lower many districts offers and the formula would no longer be equitable. She said it is unconscionable that there are handicapped children that cannot get around the schools and have to be carried up and down steps. It is demeaning to those students, and it is unsafe. This should be a priority, particularly in high growth areas such as Boone County. Senator Stine said she has filed Senate Bill 165 that puts a weighting on schools in the most urgent need so that Kentucky can address its schools with the most critical needs. She also said it may be that more money needs to be placed into the SFCC fund.
Commissioner Wilhoit discussed uniformity and the process of developing the facility plans. He said there are three areas that need improvement. They are: 1) strengthening the ranking criteria both in terms of age definitions and the language criteria around them to add more uniformity; 2) the process of determining the rating of the facilities needs to have more standardization and centralized definitions; and 3) greater uniformity is needed in evaluating unmet need in school districts.
Commissioner Wilhoit said there was a recommendation to move from a four to two-year plan for updating facility plans. These are extremely aggressive programs and require tremendous facility conversations in communities, and he is not sure that it is wise to move from four to two years. He believes Kentucky should eliminate waivers and have all districts complete the process on a timely basis, improve efficiency by having more regular filing of updates against that plan both in terms of financing and needs, and look at technology-based reporting which might distribute information back to KDE in a more timely way.
Commissioner Wilhoit said the area of maintenance is very important. He said the KDE provides guidance, but it might need stronger criteria and requirements may need more processes from the state level. There is a national standard that can be applied to all schools, and KDE is looking at that data as a possibility. He is not sure if this will resolve the problem totally as it depends on local implementation of those policies.
Commissioner Wilhoit discussed the misalignment of the actual cost of the SFCC and the allotment that the KDE allows. He said the problem between the actual cost of facilities and the amount of money that KDE suggests in the formula may be a by-product of the KDE's numbers being too low. Another assumption is that there may be expenditures that are above and beyond what one would reasonably expect to be put into a facility. He is not ready at this point in time to say that the solution for this problem is for the KDE to upgrade and increase their allowances. He said there is some indication and there are some concerns at the cost of facilities and maybe the KDE needs to look at ways to deal with the situation. However, when this occurs, the KDE is taking on the whole balance and the historic nature of the facilities process. He said the KDE needs to have figures that are up-to-date in terms of guidelines, but the other factor is determining a reasonable cost for an elementary, middle, or high school facility in the Commonwealth.
Representative Draud said if there was additional revenue to put into the SFCC and the SEEK formula, then the issues of inequities could be addressed without causing harm to other districts. He said if the KDE redistributed current money and property poor districts are impacted in a negative manner, and property wealthy districts are impacted in a positive manner, this does work politically because there are so many more districts that would lose from the property wealth issue of redistribution. He said to make the situation work, there has to be additional funds to hold some districts harmless.
Commissioner Wilhoit said adequate resources need to be in place to support the system, but it has to be balanced with an assurance from the KDE that those resources are expended in efficient and effective ways.
Representative Draud asked if the inequities could be addressed without causing a negative impact for some of the districts with the current amount of money available. Commissioner Wilhoit said he does not agree with taking from one revenue source to support another, and additional revenue would have to be put into the system for it not to hurt other districts.
Senator Kelly said there is consensus on a few issues from the thorough study conducted by the OEA staff, responses from the KDE, and the SFCC, and the deadline is fast approaching for filing bills. He would like to hear recommendations on legislation that needs to be filed before the deadline to address some of these issues. Representative Moberly concurred. Ms. Seiler said the OEA staff would identify the areas of consensus among the three groups and communicate that. Senator Kelly said the KDE will also have to amend some of their regulations, and some issues will be addressed in the budget.
Dr. Bob Tarvin, Executive Director, SFCC, and Mr. Bernie Sandfoss, Chair, SFCC, gave their response to the OEA study. Dr. Tarvin said the SFCC is appreciative of the study. He said the study was well done and very helpful in pursuing policy evaluations.
Dr. Tarvin said the SFCC supports the recommendation to extend the escrow time to eight years. They also support the idea of allowing all the districts to have the option of assessing the nickel without recall. Feedback from across the state supports that having the ability to levy that nickel would be a tremendous help in their districts, just as it has been in the growth districts that have already assessed the nickel without recall.
Dr. Tarvin said the SFCC would like for the legislature to target funding for special needs among the districts, such as for category four and five schools, as a separate budget line item. It could be added to the SFCC need and not taken from the regular offers that the SFCC has available for all districts. He said the SFCC makes up the component of need in the three-prong funding system. He said that the new debt service in millions in SFCC construction offers needs to be more consistent. He also briefly discussed the new bonding potential. There are two graphs for the SFCC construction offers and the facility funding from the 2005 General Assembly that resulted in the total new bonding potential in the SFCC handout in the meeting folder located in the Legislative Research Commission library.
Dr. Tarvin said the SFCC maintains strong support for the maintenance of school buildings. He said the SFCC noticed that the maintenance of schools is not uniform from region to region, while at the same time, realizes that this is an enormous expense for the KDE.
Dr. Tarvin said the SFCC strongly supports the recommendation that the KDE's maximum project budget will be brought in-line with actual construction costs. He said the rating system is turning into a high stakes game. He receives calls weekly from school administrators asking how go get their category three schools to rate at category four, or get category four schools to a category five. He said more dollars need to be put into this to make it into a fair and consistent system, and the unmet needs calculations need to follow more stringent timelines.
Dr. Tarvin said one recommendation that the SFCC disagrees with strongly is the refinance and savings issue. The recommendation states that if legislative intent is that the savings generated through refinancing be used on behalf of districts in ways that adhere to SFCC requirements, the General Assembly should direct that the regulation be made consistent with statute. The SFCC does not disagree with the recommendation as written, if one accepts the OEA's legal rendering of the statute. However, because the SFCC was eminently involved in the creation of the statute in question and created the regulation discussed, it feels the regulation as implemented accomplishes what was intended with the passage of the statute in 1998 and is not inconsistent.
Dr. Tarvin and Mr. Sandfoss discussed the design build concept. He said in 1994 there was a bill sponsored in the legislature for design build, and it was defeated fairly soundly.
Dr. Tarvin also said he would like the members to support technology in the upcoming budget. He said school districts need technology dollars severely at this time.
Senator Kelly said he appreciated the comments and graphs that Dr. Tarvin provided the subcommittee. He said the graphs indicated that immediately before the Kentucky Education Reform Act, there was twice the debt service offering being made available, and then it went to nothing. He said it presents another huge challenge for the General Assembly to find money for funding.
Senator Westwood said he was not aware that the design build concept had been before the legislature in the 1990's, but that maybe the political climate had changed enough to try it again. He cannot believe that a building has to be replaced after 30 years as some schools claim, and Kentucky needs to find a building design that is going to last beyond a 30, 40, or 50 year timeframe.
Dr. Tarvin said maintenance is often put on the back burner by administrators on the local level due to more pressing issues such as teacher salaries, and other programs that need resources. Representative Draud concurred with this assessment.
Representative Moberly said he was one of the creators of the SFCC and he is proud of what it has accomplished over the years. He thanked Ms. Seiler and Ms. Ewalt for their outstanding work on the study, and said issues need to be looked at to see what can be handled in the budget, and which will require legislation. He said he wanted to vote to accept the study, not necessarily to approve it, and forward it on to the LRC, but he is willing to hear the pleasure of the committee.
Senator Kelly asked Ms. Seiler if she would make any changes to the report based on the recommendations heard in the meeting. Ms. Seiler said that OEA has spoken with the KDE and the SFCC prior to the meeting so they already knew their positions on the recommendations. She said there is nothing they have recommended that they would adjust at this point. Ms. Ewalt said it is LRC policy to incorporate the responders' recommendations within the report and so the KDE's and SFCC's comments will become part of the report.
Senator Westwood said he wanted to make it very clear that the committee is not approving or agreeing with the report because he has a very significant issue in the report with the wealth determination issue. He does not have a problem saying that the committee accepts the report and the comments, and are forwarding them on to the LRC.
Senator Kelly asked if any member of the committee can file a response or a minority report to include with the submission of the report. Representative Moberly said any committee member who wants to file a response with the report may do so and it will be forwarded on to the LRC.
Senator Kelly made the motion to accept the report and forward to the LRC with the comments of the KDE and the SFCC, and any committee member included, and the motion was seconded by Representative Draud. The motion passed with a voice vote.
Commissioner Wilhoit gave a status report on the Commonwealth Accountability Testing System (CATS) contract. The members were provided with a copy of the CATS RFP process timeline for 2005 through 2006. He said the new contract will combine both state resources and the new resources from the No Child Left Behind program, and will require no requests from the KDE for additional state money for the next two years, and possibly six years.
Commissioner Wilhoit said the KDE has contracted in four different ways with six different providers to put the contract package together. Those entities are divided into the various functions that the KDE wanted to get out of the contract. The item development and test development component, as well as processing duties, were given to the contractor Measured Progress, and two subcontractors, that are named WestEd and The Collaborative for Teaching and Learning.
Commissioner Wilhoit said a contract was offered to the ACT to handle the issue of predictive assessments, which is defined as those preliminary assessments towards college readiness. Those two tests would allow the KDE to test students at the end of the middle school experience and at the mid-point of high school. They are equated to the college entrance exam that is most used by higher education system in Kentucky, which is the ACT.
Commissioner Wilhoit said the Norm-Referenced Test (NRT) component is fairly flexible at this moment depending on where Kentucky goes with legislation, but there is a continued contract offer in place with CTB McGraw-Hill.
Commissioner Wilhoit said there is a contract with HumRRO, a research center, to conduct validity and reliability studies. This ensures all the steps taken by the KDE can be defended both in terms of the purpose, the delivery, and the results.
Commissioner Wilhoit discussed the document "Seven Steps Forward" and said the KDE wanted a test that was different in structure than what had been in the place in the last two cycles. He said the reasons for wanting a test that was different in structure were multiple. The current test structure was a matrix designed exam that used six different testing forms to administer the test to the students. Those six forms are then put together to make a total picture of a school and a district. He said this gives a good picture of a statewide result, but does not give the individualized direction that teachers and students need for instructional purposes. A better design in the new contract is a standard form with matrix items around it. He said this gives a standard result for a classroom that a teacher can use to make judgements about all students in the classroom in a relative sense, and the items can be released for classroom use by using common multiple-choice items released on a regular basis, and to contract for a process for the common open-response questions to be held at those schools for scoring.
Commissioner Wilhoit said there was a need for longitudinal data to track the progress of students from year to year. He said annual assessments were added in this new contract design for reading and math through grades three through eight, and the two core content areas will be monitored on a regular basis for all students.
Senator Kelly asked to what extent the ACT test could become the NRT test for Kentucky if Senate Bill 130 passes. Commissioner Wilhoit said the ACT could be the NRT, and would not cause a conflict. Senator Kelly asked if it would cause contract problems in the future to accommodate the change. Commissioner Wilhoit said they ensured flexibility in the contract to change to the ACT if Senate Bill 130 becomes legislation.
Representative Moberly asked if the ACT aligns with the Kentucky Core Content as well as the current NRT given to students. Commissioner Wilhoit said a group of teachers looked at that issue and saw comparable levels of alignment, but none align directly.
Senator Kelly said there is language in Senate Bill 130 that requires a study be completed to show there is sufficient alignment before one test replaces another. Commissioner Wilhoit said the ACT has provided the KDE with alignment studies, but there is content alignment and depth of knowledge alignment. It is his preference that the alignment is made as soon as possible, and if it is there, it is inappropriate to double test students, and the recommendation would be made to use the ACT as the NRT test.
Representative Moberly asked if the predictors and the ACT would work together and be consistent in looking at a student's need for remediation. Commissioner Wilhoit said yes, one predictor test is given at the end of the eighth grade, and the other is given in the tenth grade and is a solid predictor of how a student will perform on the ACT test, which is the primary judgement factor at the college level.
Representative Moberly said he is concerned over the number of students who require remediation classes after entering college, with 35 to 40 percent of students needing remediation in math. He said the predictors should alert schools that students need remediation before entering in college, and the KDE needs to link some consequences and responsibilities to schools or universities if they still have a high remediation rate when those students get to college.
Commissioner Wilhoit said the ACT score at the college level is aligned with a set of documents that the Council on Postsecondary Education has created that describes the skills and knowledge a student should have for each score on the ACT. He also said if nothing is done with the information, and students pass through the system without the acceleration, the remediation rates will continue to be high. The predictor tests and the ACT should ensure that students are learning at appropriate levels, and if not, this gives enough time to intervene and make a difference in the lives of children. He said this information will be made public and will be an important document for the schools and universities. However, on the other end of the spectrum, it is incumbent that the colleges begin to monitor how well the students do after they get there.
Senator Kelly discussed classroom observation and teacher judgement as a part of measuring student progress besides just using a single index that has to go up every year. He said if students are doing a good job at high levels then their scores may stay flat.
Senator Stine asked if the cost of the test for this year was $12 million. Commissioner Wilhoit said that was right, and Senator Stine commented that this was a higher amount than in previous years. Commissioner Wilhoit said Kentucky is spending $13.14 million, with $8.3 million being state money, and the remainder is money from No Child Left Behind federal program. He said Kentucky came up with a test design that did not ask for additional state money and complied with federal guidelines.
Senator Stine asked what role the Center for Assessment played in the process of the test creation. Commissioner Wilhoit referred the question to Ms. Robin Kinney, Associate Commisioner of Internal Administration and Support, KDE, who said Mr. Brian Gong, Principal, assisted in a consulting role on the test design for the RFP. He is a national test design consultant and a psychometrician.
Senator Stine asked if it would cause concern if it were shown that principals at the Center for Assessment had a prior relationship with personnel from Measured Progress. Ms. Kinney said Mr. Gong was not an employee of Measured Progress, but Mr. Richard Hill was a former president of Measured Progress many years ago. Ms. Kinney said the Center for Assessment is a national, technical advisory panel that performs work for many companies throughout the United States.
Representative Moberly asked if Mr. Richard Hill was an employee of the Center for Assessment, and what role did this center have in the design of the test. Ms. Kinney said they were a company that helped the KDE on a consulting level with the test design for the RFP. Representative Moberly asked if they had anything to do with the selection of the test vendors. Ms. Kinney said no, they were not members of the technical panel, the cost committee, or the executive committee, and they only acted as consultants on test design.
Representative Moberly asked what role Mr. Richard Hill plays in the Center for Assessment. Ms. Kinney said he is the chairman of the Center for Assessment. Representative Moberly asked if the KDE staff knew of his previous role with Advanced Systems. He asked if the KDE knew that Mr. Hill did not live up to his contract that Advanced Systems previously had in Kentucky. Ms. Kinney said that certain people in the KDE were aware that there was a problem with Advanced Systems with the previous contract and that the KDE and Advanced Systems decided not to renew the contract.
Representative Moberly asked how many people are on the payroll of Measured Progress that were also on the payroll of Advanced Systems. Commissioner Wilhoit said there were 108 employees at Advanced Systems, and it was reorganized in 1998, and is currently Measured Progress with 334 employees. There are 12 people in managerial roles that were there previously, and 14 people in managerial roles who were not there at all. The current president of Measured Progress also used to be a vice-president at Advanced Systems.
Representative Moberly expressed amazement and disappointment that Advanced Systems and Mr. Richard Hill are involved in the new contract in any way. He remembers Advanced System's negligence and their inability to perform the contract, and almost cost Kentucky to lose the main concepts of education reform. He said that he completely holds Commissioner Wilhoit and the KDE directly responsible for any problems that Kentucky's testing system suffers from using this company.
Senator Westwood shares the same concerns about utilizing the Advanced System's company. He also expressed concern with the timeline of the CATS RFP process. It was his understanding that legislation said the process was supposed to start in early January of 2005, and it did not start until February 24, 2005, and he asked Commissioner Wilhoit what the delay was in getting the process started. He also expressed concern that the EAARS committee did not have a chance to provide input due to the two month delay.
Commissioner Wilhoit said every step of the process takes a little more time, and the starting point was moved back. Senator Westwood discussed canceling the National Technical Advisory Panel for Assessment and Accountability (NTAPAA) in lieu of the KDE staff contacting NTAPAA members by phone to answer questions and receive consultation. He is worried that the phone calls are not open to the public, and asked if there were minutes of the phone call conversations, which Commissioner Wilhoit said there were. Senator Westwood asked if those minutes were available to the public and Commissioner Wilhoit said yes. Senator Westwood requested a copy of the minutes.
Senator Westwood asked Commissioner Wilhoit why the state board cancelled their March meeting. Commissioner Wilhoit said the state board meeting was cancelled at its request. He thinks they are looking at potential transition and their schedules conflicted with the special meeting date.
Senator Westwood said subcommittee members did not have ample time to communicate their concerns, and Advanced Systems does not have a good track record and some unfortunate relationships seem to be developing. Ms. Kinney said the KDE tried to get the information out to the public as quickly as possible, and wanted to get it in front of the Government Contract Review Committee as soon as possible. She said the negotiations with four different vendors and two subcontractors has taken more time than projected. She said the KDE was very diligent to get things completed expeditiously.
Senator Westwood asked who served on the committees that provided input for the CATS RFP. Did NTAPAA have input? Ms. Kinney said the Kentucky Board of Education held public meetings on test design, and NTAPAA weighed in very heavily as well. She said members from NTAPAA actually served on the technical committee and served as scoring and evaluating members, and OEA was involved by attending all of sessions. Senator Westwood asked if the CATS contract was a personal services contract. Ms. Kinney said yes, at the direction of the Finance and Administration Cabinet.
Commissioner Wilhoit said there are two other areas of concern. One is the turn around time of scoring assessments, and there is a phased plan for moving towards on-line administration of assessments. The technology infrastructure in the schools needs to be addressed as Kentucky moves ahead. He said the last item of issue is in-state scoring, and he is pleased to announce that a scoring center is moving to Kentucky permanently that would allow in-state scoring to involve teachers directly in the process, and to use the center to score assessments for other states as well.
Commissioner Wilhoit said he looked at using Advanced Systems from multiple angles. They were 1) by reviewing their relationship in Kentucky and other states; 2) by looking at the capacity of their organization today as opposed to the organization that it was; and 3) by looking at their record in states that they have served other than Kentucky. He understands that his personal integrity is a part of all this, and he assured members that it was a very diligent searching process to come to this point.
Representative Moberly commended Commissioner Wilhoit for all of his work on changing the structure of the test to meet some new goals. He asked if Kentucky was at the point of saying that the longitudinal data is reliable on the individual student level. Commissioner Wilhoit said the fact that students are not taking multiple forms of a test brings much more reliability to the process in terms of being able to make comparisons across students, but would like to weigh in with NTAPAA to make sure they can make that kind of judgement.
Representative Moberly said he believes the test will be much improved in its structure and in meeting the goals it ought to meet. He said he will personally reserve judgement to see how it works in the implementation, and remains concerned of Mr. Richard Hill's involvement. Ms. Kinney said Mr. Hill did not provide direct assistance to the KDE, only Mr. Gong and Ms. Karen Hess. Representative Moberly said Mr. Hill is the chair of the Center for Assessment and that it is enough. He reaffirmed that Commissioner Wilhoit's personal integrity is on the line, and if they are negligent then it becomes the KDE's negligence.
With no further business before the committee, the meeting adjourned at 12:55 p.m.