The fifth meeting of the Education Assessment and Accountability Review Subcommittee was held on Tuesday, October 9, 2007, at 10:00 AM, in Room 131 of the Capitol Annex. Senator Jack Westwood, Presiding Co-Chair, called the meeting to order, and the secretary called the roll.
Present were:
Members: Senator Jack Westwood, Co-Chair; Representative Harry Moberly Jr, Co-Chair; Senators Dan Kelly and Ken Winters; Representatives Mary Lou Marzian and Frank Rasche.
Guests: Clyde Caudill, Legislative Agent, Kentucky Association of School Administrators and Jefferson County Public Schools; Brenda McGown, Kentucky Education Association; Fred Carter, Deputy Secretary, Kentucky Education Cabinet; John Wilkerson, Kentucky Education Association; Carolyn Witt Jones, Director, Partnership for Kentucky Schools; Polly Marquette; Becky Burton; Cindy Heine, Prichard Committee; Wayne Young, Executive Director, Kentucky Association of School Administrators; and Dianne Bazell, Assistant Vice President, Academic Affairs, Council on Postsecondary Education.
LRC Staff: Sandy Deaton, Audrey Carr, Janet Stevens, and Janet Oliver.
Senator Westwood announced that Representative Draud has been reappointed to the subcommittee replacing Representative Milward Dedman.
Senator Westwood asked for approval of the minutes of the September 24, 2007, meeting. Upon motion by Representative Moberly, with a second by Senator Winters, the motion was approved by voice vote.
Senator Westwood asked the representatives from the Kentucky Department of Education (KDE) to give their presentation on the CATS results and the use of concordance tables. Committee members were provided a hard copy of KDE's PowerPoint presentation.
Mr. Kevin Noland, Interim Commissioner and General Counsel of KDE, said the purpose of the presentation was to provide an in-depth description of the concordance model used to link the old Commonwealth Accountability Testing System (CATS) to the new CATS test. He said the concordance model was recommended by the psychometricians on the National Technical Advisory Panel for Assessment and Accountability (NTAPAA).
Mr. Noland said that as a result of the No Child Left Behind (NCLB) Act, math and reading must be assessed in grades 3-8 and one grade in high school. He said prior to NCLB, Kentucky tested on seven content areas in grades 3-8 although each content area was not tested at every grade; and Kentucky previously tested math in grades 5 and 8 and reading in grades 4 and 7. He explained that the passage of Senate Bill 1 by the 2006 Legislature also required the EXPLORE, PLAN, and ACT assessments be administered. Mr. Noland said NTAPAA recommended that the performance standards for novice, apprentice, proficient, and distinguished be reviewed at least every five years and, when the standards were reviewed this summer by approximately 400 teachers, some revisions were made to the cut scores for the various classifications. He said the Kentucky Core Content Test (KCCT) was also changed resulting in an increase in the number of multiple choice questions and a decrease in the number of open response questions and writing portfolio items. Mr. Noland said the weights in the accountability index were changed to increase the weighting in math and reading and some changes in weighting were made to the nonacademic measures. He said that over 200 teachers made a concerted effort to make changes to the core content to reduce the breadth and increase the depth of the curriculum. Mr. Noland said, as a result of the combined changes, a new CATS system was created much like what occurred when KIRIS changed to CATS in 1999-2000.
Mr. Noland explained that education is currently in the middle of a biennial accountability period so the performance judgments will not be available until Fall 2008. He said NTAPAA advised that an accountability bridge would be necessary from the 2006 CATS to the 2007 CATS; but, at the same time, teachers would need the instructional benefit of seeing unadjusted scores for the content areas so they could focus instruction in those areas needed to help students achieve proficiency.
Mr. Ken Draut, Associate Commissioner of the Office of Assessment and Accountability, KDE, explained that today's presentation was also used to explain the concordance tables and related data to superintendents, school board members, and the general public. He said the presentation is a simplified version of the complex statistical model and a more detailed version of the tables is available on the KDE Website. He said because of all the changes in the 2007 tests, a direct comparison to 2006 CATS scores was not possible but it was important that schools and districts still be held accountable for progress or lack of progress. He said therefore NTAPAA recommended that concordance be used to allow comparison of the two tests across two different years and to link the 2007 scores back to the original school goals.
Mr. Draut utilized a baseball analogy to explain the concordance table. He said the concordance table allows KDE to take the rank order of schools, link them to the previous year, and then link the score to see how the schools would have done if the tests had not been changed. He said concordance tables are used by ACT and SAT, explaining that ACT could test a certain group of students and then using a concordance table make an accurate prediction of how the same group of students would score on the SAT without them actually taking the SAT.
Mr. Draut explained that the accountability index scores must be rank ordered, which reflects percentiles operating around a bell-shaped curve, from low to high for all Kentucky schools in both 2006 and 2007. He said the concordance table then links the 2007 scores as a non-adjusted accountability index from the actual scores from Spring 2007 tests administration to the actual 2006 scores to arrive at the adjusted accountability index. He explained there is an assumption that each school has growth, so instead of a straight connection to 2006, the concordance does an estimated growth to connect the actual 2007 scores back to what an estimated score would have been if CATS had not been changed. He said the estimation is created based on the growth pattern in the CATS data from 2000 to 2006, which in essence links the two scores together.
Mr. Draut said the concordance is only applied to the accountability index and that index is the only number in the Kentucky Performance Report that is adjusted with the remainder of the report showing the actual Spring 2007 scores. He said the concordance table results in a statistical adjustment and scores may remain the same or slightly increase or decrease. He related that the individual student score of novice, apprentice, proficient, or distinguished is not adjusted and is the actual spring 2007 score results of the new CATS.
Ms. Rhonda Sims, Director, Division of Assessment Support, Office of Assessment and Accountability, KDE, explained the Kentucky Performance Report growth chart stating it shows the state's performance in two-year cycles and sets in motion school recognition and school assistance. She said in 2006 Kentucky was in a progressing zone, having an adjusted accountability index in 2005 of 77.4 and an adjusted index in 2006 of 79.2, resulting in a combined index for the two-year period of 78.3. She said the 2007 adjusted index was 81.8 which is the only adjustment on the performance report and is linked to the past. She continued that Kentucky's goal for the 2008 biennium is 85.0 in order to achieve proficiency level of 100 by 2014, which would require a score of 88.2 in 2008 to be averaged with the 2007 score of 81.8, thereby requiring a growth of 7.0 points. She said to achieve such improvement will be a daunting task.
Ms. Sims explained that the breakout growth chart for elementary, middle, and high school was linked back to the past trends and the total index on the chart was an average of the elementary, middle, and high school indices. She said the chart shows consistent growth in each school classification with elementary schools out performing middle and high schools. She said that Slide 23 of the PowerPoint provided information on the number of schools in each adjusted accountability index with the lowest school performance in Kentucky being in the range of 50-59.9, which is approximately 55 points from the 100 goal to achieve proficiency. She said schools in that range would have to grow 15 points each two-year cycle and it appears that struggling schools still need assistance. She said schools on the higher end of the accountability index, such as those in the 70-79.9 range, would need an 8.0 growth every two years to obtain the proficiency goal by 2014.
Ms. Sims then discussed the non-adjusted data on Slide 24 of the presentation saying it is the first look forward into the new CATS system with the new cut scores and weighting applied. She said the total academic index is an average of all content area scores, with elementary schools having the highest total index. She said the index also clearly reflects why there is such great concern about high school mathematics, which had a non-adjusted index of 68.2. Ms. Sims said it is hoped that the end-of-course assessments in Algebra I, Algebra II and geometry and the increased curriculum for those courses will enable the math score to improve in the future. She related that the index for PLAN, which was first administered in September 2006 at Grade 10, was generated by giving credit for different ranges of performance, so a score of 18-24 would be given 100 points for proficiency, and then higher scores would be plotted to 140 representing distinguished. She said the index for nonacademic measures include attendance and retention at elementary, with dropout added to middle, and attendance, retention, graduation rate, dropout rate, and transition to adult life for high school.
Ms. Sims also discussed the disaggregated total academic index trends by populations of students including gender and ethnicity breakouts noting that Asian students tend to score higher overall and that African-American students tend to score lower. Index trends were provided for other population groups, which revealed that students participating in free or reduced-price lunches tend to score lower and students with limited English proficiency or with disabilities scored significantly lower than students not hindered by barriers.
Mr. Draut said that at the October meeting of the Kentucky Board of Education, the board gave KDE direction to use the concordance table for two years only, from 2006-2007 to 2007-2008, and when that cycle is completed, schools will receive a new baseline score for the 2008-2009 and 2009-2010 biennia, with a new set of goal lines to 100 by 2014. He said at that point there would be no further adjustments in scores but rather the scores will be based on actual CATS data. Mr. Draut said that regulatory language will be presented for review and action.
Senator Kelly said it is his opinion that the adjustment to the scores for the accountability index is based on a presumed growth which creates an artificial figure that reveals nothing and that single data point is being overemphasized. He said the EXPLORE and PLAN test results provide a much more accurate student score and reveal what is occurring at the school level right down to each student and the information can be used to determine what skills each student needs to meet the proficiency benchmarks. He asked if KDE is reviewing the EXPLORE and PLAN data for individual students and individual school districts, to which Mr. Noland replied that the department is reviewing the data. Senator Kelly said those results have provided meaningful information to him, citing statistics about some districts that had a high percentage of students participating in free or reduced-priced lunches were far from meeting the proficiency benchmarks. He said particular focus should be on those schools and that he is interested in seeing how the PLAN and EXPLORE results compare to CATS results. Senator Kelly asked if a correlation is being done between ACT scores and CATS. Mr. Draut said that KDE will begin an in-depth analysis, including correlations between the various test data. Mr. Noland said that Kentucky uses a variety of tests to assess student achievement, including CATS, ACT, PLAN, EXPLORE, and others. He said that local school districts also use other tests for continuous diagnostic assessment so they may intervene as soon as possible with low performing students. Mr. Noland said it may be helpful if NTAPAA members could appear at a future meeting of EAARS to further explain and answer questions regarding the concordance table. He said the EXPLORE and PLAN results do provide helpful information, just like other tests do, and KDE will assist school districts in analyzing the tests results. He reminded members that Kentucky is currently testing 100% of its student population with PLAN, while the national average to which Kentucky scores are compared is not based on testing of total student populations. Senator Kelly said he is not concerned about comparisons but about results and it appears that large percentages of Kentucky students are not meeting the benchmarks. He said averages are just what they are and that the focus needs to be on low performing schools and districts rather than using resources and valuable time on creating and justifying accountability index scores.
Senator Winters asked what changes were made to the CATS test. Mr. Draut replied that on the old CATS system there were 24 multiple choice questions in each subject matter but in the new CATS system the number of multiple choice items was increased to 33-38 depending on the subject; and, there were six open response questions on the old CATS and now there are four open response on the new CATS. Also, the weighting, which was 66% for open response in the old CATS, was changed to 50 % for multiple choice and 50% for open response in the index score. Senator Winters asked if the new cut scores for novice, apprentice, proficient, and distinguished have been adjusted and why were they adjusted. Mr. Draut said the new cut scores were reviewed and changed where necessary in July of this year because of all the changes that had occurred with CATS test. He explained that NTAPAA recommended that the department use the Bookmark method to set the cut scores, which is a nationally recognized sound procedure where a panel of teachers look at items and student work and make a judgment call on where the cut should be for novice, apprentice, proficient, and distinguished. He said the panel uses scoring rubrics and guidelines established by facilitators on how to set the scores and the facilitators ensure the process is being followed. Mr. Draut said the newly revised cut scores were applied to the Spring 2007 scores. Senator Winters asked if the new cut scores are higher or lower than last year. Mr. Draut said they are neither higher nor lower but are merely different based on the new tests. Senator Winters said philosophically he is concerned about the group of students in the 50-59 accountability index range and the magnitude of growth that group needs to accomplish to reach proficiency by 2014. He said that reading and math proficiency would be instrumental in assuring success in college and much emphasis needs to be placed on those subjects in the years ahead. Senator Winters said analyzation is not as important as the development of a plan to provide targeted and aggressive assistance.
Representative Moberly said he believes the accountability index is very important. He agreed with Senator Winters that the CATS, PLAN, EXPLORE and other tests appear to be revealing the same information that has been available for several years and he also agreed with Senators Winter and Kelly that the important factor is developing the strategies to address the issues. He said that currently the end-of-course examinations, a math initiative, and a more in-depth math curriculum is planned but many ideas have just been repeatedly discussed and he would like to see a greater sense of urgency rather than using piece meal initiatives to produce the needed results. Representative Moberly recommended that the next commissioner of KDE immediately meet with the president of the Council on Postsecondary Education (CPE) to develop an overall strategy to be approached with a sense of urgent accomplishment.
Mr. Noland said at the last meeting of the Subcommittee on Elementary and Secondary Education, Representative Rasche and Senator Westwood presented several recommendations that the Subcommittee on Assistance to Schools had received from various education professionals and recommended that at the next meeting of this committee that KDE be allowed to share information about what is currently in process and what needs to be done to address this situation. He said a work group has already been created with KDE and CPE representatives to develop strategies to improve college readiness and that KDE has been working with the state board over the last four months to develop strategic priorities on how to fill achievement gaps and needs, such as math.
Senator Westwood said that although there are numerous statistics now, he is concerned about the accuracy of the information. He said, for example, that if the scoring rubric has been changed to be less demanding, then that would make the test easier in his opinion. He said that some schools showed phenomenal growth on their unadjusted scores, but when the scores were readjusted, the statistics were different. He asked if the unadjusted scores would be the accountability index in two years and Mr. Noland replied that they would be. Senator Westwood questioned whether reaching the goal of 100 should be satisfactory since that would only be 71% of an ultimate score of 140. He said the Assistance to Schools Subcommittee received many great ideas and recommendations and that he is also working with the Education Cabinet to explore other areas and methods of improvement. He said the statistics relating to the Asian community reveal that they are succeeding and the reasons for that success also need to be examined.
Senator Westwood asked Mr. Noland if the statistics from the National Assessment of Educational Progress (NAEP) would be considered an "apples to apples" comparison to CATS. He said it was pointed out that Kentucky is one of only a few states recognized over the last five years by NAEP as making significant progress, but The Nation's Report Card released in 2007 indicated that no statistical progress was being made in Kentucky. Mr. Noland said he would be glad to provide the information from NAEP which showed that over the last ten years Kentucky has received many accolades for its accomplishments and consistent progress. He said that NAEP is the best comparison to use for CATS and when they did an analysis of each state's definition of proficiency based on cut scores and standards, NAEP ranked Kentucky in the top third nationally as being close to its own rigorous standard of proficiency.
Senator Winters said he recently attended the National Governors Conference, during which representatives from 17 states discussed high school curriculum and, compared to those states, he said Kentucky is making progress and applauds the efforts. He said legislators also are struggling with providing direction for education. Senator Winters said he would like to make sure that reaching 100 on the accountability index is equivalent to college preparation without remediation and work readiness; and, since the legislature has only three budget cycles to make it happen, everyone needs to work aggressively and cooperatively to achieve proficiency by 2014.
Senator Westwood asked if there is any information that shows reaching 100 would represent college or workforce readiness. Mr. Draut said a deeper analysis is necessary and KDE will take the new CATS scores and begin those correlations and connections and that the ACT results will also be helpful in that regard. Mr. Noland said Kentucky's proficiency standard is rigorous as compared to other states and KDE wants every student to show continuous improvement, even surpassing the 100 goal to strive to reach 140 which represents distinguished achievement and that some Kentucky students are already excelling to that point.
Representative Rasche asked how the shift from last year to this year was calculated in the concordance. Mr. Draut said that the statistical adjustment was developed from a review of the old CATS scores from 2000-2006, which showed a consistent pattern of a three point gain each year; so the assumption was that if CATS had not been revised, there would have been a three point growth between 2006 and 2007. Representative Rasche said when he calculated the growth it was only 2.1 and he will get with Mr. Draut to further discuss how the gain was calculated.
Representative Rasche then asked how the index would compare to a letter grade. He said there appears to be confusion between percent on a test for a letter grade and an index that also happens to be 0 to 100 and asked if a concordance has been done between the index and a letter grade. Mr. Draut said he is not aware of such a comparison and related that in the index if every student in the school was proficient the index score would be 100 but if every student in the school was distinguished the score would be 140 and therefore a comparison between a letter grade and percentile ranks would create confusion. Representative Rasche said a letter grade comparison to an SAT scoring model of 200-800 would have no significance, but when a percentile rank for apprentice is not considered failing but the corresponding percentile in a letter grade would be failing, confusion can occur. He said that is the reason he wanted to know if a cut analysis or Bookmark had been done to actually compare where a letter grade of A, B, C, D falls as opposed to where NAPD categories fall.
Senator Westwood said that, as Senator Winters pointed out, Kentucky is not unique in the nation as it appears each state is struggling in math, reading, and science, and it is important that Kentucky use every available resource which is the reason accurate and complete information is needed.
Senator Westwood asked KDE to give their presentations on the implementation of House Bill 197 relating to end-of-course assessments. Members were provided a two page handout on the status of the program.
Mr. Noland said that since the EAARS members currently present at the meeting had already heard his report on end-of-course assessments at the October 8, 2007, meeting of the Interim Joint Committee on Education, he would just answer any additional questions they may have. Senator Westwood asked if there is sufficient funding to pay for the end-of-course examinations. Mr. Noland said KDE has sufficient funding for the pilot but they do not have funding for statewide administration of the assessments for the subsequent two years. He related that when House Bill 197 was passed in the 2006 session, it required that pilot assessments for Algebra I and II and geometry be administered in Spring 2008 but no funding was provided. He said KDE identified $450,000 in unobligated Title II federal funds to try to accomplish the mandate. Mr. Noland said the Algebra II assessment was developed as part of a consortia with Ohio and eight other states and that 3200 students would participate in the pilot, 2400 using paper and pencil and 800 using online, and that Kentucky's share of the cost will be approximately $150,000. He said that KDE issued a request for proposals for the end-of-course assessments for Algebra I and geometry, but all of the bids were too high and they withdrew the request. Subsequently, they met with Dr. William Bush, Professor of Mathematics Education at the University of Louisville, and it appears he and others at the university may be able to develop an assessment for Algebra I and geometry which can be piloted in Spring of 2008 in a price range that will not exceed available funds in the $300,000 range. Mr. Noland said that KDE's budget request for the upcoming biennium will include a funding request for statewide administration of the examinations.
Mr. Noland said that Senator Winters had mentioned at another meeting that ACT had already developed end-of-course examinations. He said ACT participated in the bidding process and their proposal was very expensive. He said the statute specified that content teachers in the core courses, postsecondary faculty including subject matter specialists as well as teacher educators, curriculum specialists, and other appropriate practitioners were to be involved in the development and review of items for the assessments and there was to be alignment to end-of-course examinations with Kentucky standards, content goals, and academic expectations. Mr. Noland said if KDE is to continue with the end-of-course assessments, the initiative must be adequately funded. Senator Winters said he was aware that ACT had already developed nine end-of-course assessments and related curricula and were developing a comparable number in other subject areas. He thanked Mr. Noland for the information provided about the bidding process and the information on Kentucky's specific needs.
Representative Moberly asked what changes have been made in the curriculum in preparation for the end-of-course examinations. Mr. Draut replied that Dr. Bush has worked with Kentucky teachers and university staff in developing the diagnostic assessments for Algebra I and geometry to ensure that the framework matches Kentucky's program of studies and core content. Representative Moberly asked if an outside consultant will be needed when the assessments are administered statewide. Mr. Draut said if the pilot process is effective and the tests prove solid, then the assessments created by U of L could continue to be used. Representative Moberly asked if a cost estimate has been done for a statewide administration. Mr. Draut responded that the estimate is approximately $3 million, which would include scoring because the tests do have some open response items. He said the estimate is based on a cost of $24-$25 per student with approximately 45,000 students taking Algebra I and a similar number taking geometry. Representative Moberly asked if the statute required that the end-of-course assessments be included in the accountability system. Mr. Noland said the statute allows local districts to determine the extent to which the assessments are to be used in individual student accountability scores, for successful completion of a course, or as a high school graduation requirement. He said the statute does not address how they are to be used in the accountability index. Representative Moberly asked if including the assessments in the accountability index has been discussed. Mr. Draut said that former KDE Commissioner Gene Wilhoit had discussed the use of end-of-course assessments as a part of CATS or a replacement for high school CATS in a report called "Seven Steps Forward". Representative Moberly said he would like to see the statewide administration costs estimates as soon as they are available. Ms. Sims related, that although Algebra II is being done through a multi-state consortia, Kentucky teachers have been involved in the process.
Representative Rasche asked if the Algebra I assessment would also be given to eighth grade Algebra I students since the report did not mention it to which Mr. Noland responded that all Algebra I students would be assessed regardless of their grade level.
Senator Winters asked, since a consortium is involved, if comparison data for the Algebra II results would be available and Ms. Sims replied that the data will be available. Senator Winters said apparently no comparison would be available on Algebra I and geometry since the assessments would only be administered in Kentucky and if a more universally accepted end-of-course examination were used, comparison data to other students across the nation would have been available.
Senator Westwood asked if there were areas of concern other than cost on the bids received for Algebra I and geometry. Mr. Noland said the entire bid process was rejected because of insufficient funds and therefore an in-depth analysis of the bids was not completed. Senator Westwood asked if funding will be requested during the next budget cycle to which Mr. Noland said that it will be included and that the pilots for all three subject areas will be conducted in Spring 2008 and the funding request will be for the statewide administrations in 2009-10.
Senator Westwood asked Ms. Marcia Seiler, Director, and Ms. Brenda Landy, Research Analyst, Office of Education Accountability (OEA), Legislative Research Commission, to give their presentation on the 2007 NAEP results. Members were provided a copy of the PowerPoint presentation.
Ms. Seiler said the information provided is from The Nation's Report Card for 2007 from the National Assessment of Educational Progress (NAEP). She explained that NAEP is an assessment that is carried out by the U.S. Department of Education under the National Center for Education Statistics. She said there is a National Assessment Governing Board appointed by the Secretary of Education who sets the policy for NAEP and is responsible for developing the framework and setting curriculum and test facts. NAEP provides a national and a state report by subject matter but does not provide statistics on individual student scores.
Ms. Seiler recognized Mr. Mark DeCandia, NAEP Coordinator for the Kentucky Department of Education, who was in the audience, and explained that he works with NAEP data and would be available to provide information to the members.
Ms. Seiler explained that the population of students tested is selected from public and non-public schools and various under-represented populations. She said the review encompasses math and reading achievement at grades 4, 8, and 12 from four NAEP regions of the country and is also broken down by gender, race, and other categories, such as parent education, degree of urbanization of schools, and free and reduced-price lunch populations. She said there are fourth and eighth grade reports and a separate sample of twelfth grade at the national level only. She said NAEP also has assessments in arts, civics, geography, US history, and world history and other tests are being developed but only math and reading would be discussed at this meeting. Ms. Seiler said Kentucky schools have had students voluntarily participating in some of the other assessments.
Ms. Seiler explained that the assessment for the state report is identical to the national but different pools of students are selected for each report and the National Governing Board establishes how the representative sample is to be drawn. She said the NAEP assessment with state components are math, reading, science and writing for grades 4 and 8 and only students from public schools are tested for the state and national comparison, but the national report includes both public and private students. She said NAEP has an online site with various tools to assist in reviewing data and a schedule for national and state assessments and that reading, math, writing and science are administered in odd number years and the other assessments are given in even number years, although the twelfth grade national NAEP is only given every four years.
Ms. Seiler said the populations to be tested are selected by the National Governing Board and that typically 30 students per subject per grade are selected in each school, and information was provided on the number of Kentucky schools and students selected for each subject area and grade. She said the results of the NAEP reading and math assessments are released six months following administration, usually in the spring of the following year, and the assessments generally take students about 90 minutes and principals, teachers, and guidance counselors also complete questionnaires that take about 20 minutes.
Ms. Seiler said that the 2006 Kentucky public school characteristics revealed that 60.6% of the approximate 680,000 students are enrolled in Title I schools with 16% having individual education plans (IEPs). She said other statistics and information is included in the PowerPoint relating to student and school characteristics which should be kept in mind when reviewing scores.
Ms. Landy presented the average scores and percentage of students considered proficient by achievement levels. She said the achievement levels of basic, proficient, and advanced are set by the National Governing Board, and there is also a level called below basic for which a definition is not provided. She said the scores typically have a range of 0-500 for reading, math, history, and geography, but she would only be discussing reading and math, and that the scores for other subject areas is 0-300. She said the performance levels used by NAEP are not directly comparable to Kentucky's performance levels.
Ms. Landy provided the following information on NAEP results. For fourth grade math, Kentucky's score was 235 on a proficient scale of 249 compared to a national average of 239, which was statistically significant meaning there was a 95% probability that the differences are real and not just a result of random sampling error. Three out of ten Kentucky fourth graders met or exceeded the proficiency level as compared to almost four out of ten for the nation. When compared to other states, eight states were significantly below Kentucky in fourth grade math, 13 states were similar to Kentucky, and 29 were higher than Kentucky. She said only Massachusetts had an average that met or exceeded the level considered proficient. She said the trend chart revealed that Kentucky's fourth grade math scores has increased steadily and is on a similar pace to the national average increases. She said on eighth grade math, the proficient score is 299, Kentucky was at 279 and the national average was 280, with a little over one-fourth or 27% scoring at or above the proficient level compared to a national average of 31%. Compared to other states, 14 states scored significantly lower than Kentucky, 27 were significantly higher, and even the highest scoring state, which was Massachusetts, had an average below the level considered proficient. Eighth grade math trends showed that between 2000 and 2007, Kentucky's scores increased by seven points which was slightly higher than the US increase of six points. In fourth grade reading, proficient is 238 and Kentucky had a score of 222, which was higher than the national average of 220, with one-third or 33% of Kentucky students scoring at or above proficient compared to 31% of the nation. Ms. Landy said a two-point difference is considered significant. Nineteen states were below Kentucky on fourth grade reading and twelve above and there was no state that met the level of proficient. She said, since 1998, Kentucky made slow but steady gains and stayed just above the national average in fourth grade reading. On eighth grade reading, proficient was 281, Kentucky's score was 262, and the national average was 261, with 15 states having scores lower than Kentucky and 23 states with higher scores and no state at proficient level. The trend line showed that although there was a slight gain from 1998 to 2003, scores began to gradually decline beginning in 2003 to the 1998 level, which also appeared to be the trend nationwide.
Ms. Landy also provided information on the achievement gaps in math and reading for schools with predominately free or reduced-price lunch students. In fourth grade math, the gap was 22 points nationally and 19 points in Kentucky, and in eighth grade there was a similar pattern with the US gap being 26 and Kentucky at 21. With regard to fourth grade reading, Kentucky's gap was 22 and US gap was 27, and in eighth grade reading, the gap was 19 for Kentucky and 24 for the nation. She said overall Kentucky has a higher score for economically disadvantaged than the nationwide average. Ms. Landy also provided information on the achievement gap related to race, gender and disability, with Kentucky generally having less of gap than the nation as a whole.
Senator Westwood said he was pleased with Kentucky results on fourth grade reading but was concerned about the eighth grade reading results. He said the results appear to belie the notion that reading continues to improve as a student advances. Ms. Landy said she has not completed a deep analysis of the score but one factor may be that not as much emphasis is placed on reading in and of itself at middle school level and at that point students may be expected to read to learn, rather than improving their reading skills and comprehension. She related that during site visits she has observed that Kentucky schools are placing new emphasis on reading exercises in the higher grades in various classes. Senator Westwood said the NAEP results should be a signal to educators and policymakers to continue placing greater emphasis on reading using different teaching methods.
Senator Winters asked how the schools were selected for participation and by whom. Ms. Seiler said the National Governing Board has a process for selection. Mr. DeCandia said the National Center for Education Statistics selects the schools by demographic location and the demographic population in each state and they may sometimes repeatedly select the same region of a state because it meets their criteria. Senator Winters asked if performance factors are also considered, to which Mr. DeCandia said they are not. He related that NAEP will be doing long term national trend this year on reading and math and 12 schools in various districts in Kentucky have been selected to participate. He said when state assessments are administered next year, a much larger population will be tested with selection occurring in 200-300 schools. He said NAEP does not reveal the selection criteria other than saying it is based on demographics and they do not seek state input into the selection process. Senator Winters said the results of the tests could be skewed if selection is not representative of the states. Mr. DeCandia said they do test a cross section of the population of a state and that he will try to obtain more information on the selection process. Senator Winters said that the NAEP comparison will always be used to see how Kentucky is performing in these areas and it is important that the selection process is not something that would affect the scores.
Senator Westwood asked Ms. Seiler to discuss the proposed study agenda for OEA. Members were provided a copy of the proposed research topics for 2008. Ms. Seiler said that since the statute was changed in 2006 to require approval of OEA's study topics by EAARS, it has helped their office tremendously to focus their resources on interesting and informative research. She said if the proposed study items are approved, she will bring back actual study plans at a future EAARS meeting providing the detail on how the studies will be conducted.
Ms. Seiler said OEA is proposing that a study be done on the Kentucky district rankings using data such as instructional and non-instructional staff, teacher-student ratios, teacher pay, and other related information. She said data points may be revised depending on accuracy of information once the review is initiated and they will incorporate the findings into the national compendium as a new chapter within that book. Ms. Seiler said another topic needing study, which was revealed through various OEA investigations, is student fees with the emphasis being on unreimbursed curricular and noncurricular fees borne by students and their families, such as classroom fees, supplies, and sports. She said district policies would be reviewed to determine what monies flow outside of SEEK and whether are not the fees are readily visible in state and local funding. She said the fees to be reviewed are those collected and expended at the district level and she informed the members that OEA has received several complaints about fund raiser monies not being used for the purpose for which they were earmarked. Ms. Seiler said that in December 2006, OEA updated EAARS on information technology initiatives and an update can be provided on how the money is being spent and if the programs are being utilized. She said another study proposal is a review to determine the extent to which school districts have implemented Kentucky's Primary Program. Ms. Seiler said another proposal is that OEA do a broad overview of professional development plans by reviewing what KDE is doing with professional development and conducting a random sample of districts. She said Senator Kelly was interested in this particular study but expressed his opinion that it should be a more focused sample and they will modify those plans if the committee approves the agenda. Ms. Seiler said they will also do their annual finance report, provide an update on the compendium of national rankings, and the OEA end-of-year annual report.
Senator Westwood said each of the study items would be useful and the proposal was an ambitious agenda for the year. Ms. Seiler said they would be open to other ideas and could revise the agenda if needed to accomplish other areas of study.
Representative Moberly recommended that a study be conducted on students with disabilities because of the lack of progress on test scores for that population. Ms. Seiler related that study was scheduled to be completed last year with the assistance of NTAPAA but because their contracts have not been renewed, it was not completed. She said she was unsure what type of study could be done without the technical advice of NTAPAA, but perhaps they could do some analyses of scores from the various assessments. Representative Moberly expressed his concern about the non-availability of NTAPAA and said this committee should discuss this matter with the Legislative Research Commission (LRC) . He said it is his understanding that Mr. Noland has also expressed to LRC that the department desperately needs NTAPAA consultation and advice.
Senator Westwood asked the members to review the OEA study items and to contact Ms. Seiler with recommended changes or additions. Ms. Seiler said she will have her staff look at the special education issue to see what they can do without the expertise of NTAPAA and maybe other experts in special needs assessment could assist them. Representative Moberly said they do not need to do any study that needs NTAPAA expertise without NTAPAA and LRC should be urged to proceed with the contracts because non-resolution is hurting education efforts. Senator Westwood said that perhaps OEA could review the whole concept of special education, such as programs and policies related to the individual education plans and how those relate to the individual learning plans, instead of focusing on testing and accommodations.
Because a quorum was not present, Senator Westwood asked Ms. Seiler to proceed with the development of the detailed plans on the items discussed.
Senator Westwood informed the members they will be contacted about the next meeting date. There being no further business to discuss, the meeting adjourned at 11:50 A.M.