Call to Order and Roll Call
The first meeting of the Education Assessment and Accountability Review Subcommittee was held on Monday, June 13, 2011, at 10:30 AM, in Room 131 of the Capitol Annex. Senator Jack Westwood, Co-Chair, called the meeting to order, and the secretary called the roll.
Present were:
Members: Senator Jack Westwood, Chair; Senators Vernie McGaha, Gerald A. Neal, Jack Westwood, and Ken Winters; Representatives Linda Belcher, Ted Edmonds, Bill Farmer, and Mary Lou Marzian.
Guests: Clyde Caudill, Legislative Agent, Jefferson County Public Schools and Kentucky Association of School Administrators; Jim Thompson, Legislative Liaison, Education and Workforce Development Cabinet; Wayne Young, Legislative Agent, Kentucky Association of School Administrators, Sue Cain, Council on Postsecondary Education, and Marty White, Kentucky Medical Association.
LRC Staff: Sandy Deaton, Ken Warlick, Janet Stevens, C.J. Ryan, and Lisa Moore.
Introduction of New Members
Senator Westwood welcomed Representatives Linda Belcher, Ted Edmonds and Senator Gerald Neal to the committee.
Election of Co-Chairs for 2011-2013
Representative Marzian nominated Representative Edmonds to be elected as House co-chair. The motion was seconded by Representative Farmer. Representative Farmer made the motion that nominations cease and Representative Marzian seconded the motion. Representative Edmonds was elected House co-chair by voice vote.
Senator Winters nominated Senator Westwood to be elected as Senate co-chair. The motion was seconded by Senator McGaha. Senator McGaha made the motion that nominations cease and Senator Winters seconded the motion. Senator Westwood was elected Senate co-chair by voice vote.
Approval of Minutes
Senator Winters made a motion that the minutes of the December 7, 2010, meeting be approved. Representative Farmer seconded the motion and the minutes were approved by voice vote.
Status Report on the Development of End of Course Assessments
Ms. Rhonda Sims, Director, and Ms. Jennifer Stafford, Policy Analyst, Office of Assessment and Accountability, Kentucky Department of Education (KDE), presented on end-of-course assessments. Ms. Sims said Senate Bill (SB 1) enacted in the 2009 Kentucky General Assembly, requires a new public school assessment program beginning in the 2011-2012 school year. The legislation allowed, with approval by the Kentucky Board of Education (KBE), an end-of-course (EOC) program at the high school level.
Ms. Sims said the EOC assessments establish a common rigor for core courses. EOC assessments when purchased from national providers can link performance of Kentucky students to national results. EOC assessments allow students to be assessed immediately after completing course work and to receive feedback quickly. EOC test results may be used for a percentage of a student’s final grade in the course, as outlined in local policy. If that percentage is less than 20 percent, school districts will submit reports to KDE providing justification.
Ms. Sims said ACT, Inc. has been awarded the contract to provide EOC assessments for Kentucky. The ACT QualityCore® program is: syllabus-driven with curriculum and instruction support materials; based on research in high-performing classrooms that focus on the essential standards for college and career readiness; and connected to PLAN and ACT. English II, Algebra II, Biology, and U.S. History are the courses required for graduation.
Ms. Stafford said multiple testing windows will be available during the school year. The EOC assessments can be administered throughout the year as students earn credit in each course. The multiple-choice items may be completed online or on paper. The constructed response is paper only.
Ms. Stafford noted that the online administration of the multiple-choice items provides feedback within a day. The paper administration requires 10 days from the time ACT receives answer documents to process and post online reports. Electronic student and teacher reports will be available in the secure online system.
In response to questions from Representative Farmer, Ms. Sims said multiple test forms will be used annually and tests will not be repeated as in the past. Ms. Stafford said the EOC assessments will be based on standards and teachers are expected to teach the standards to the students.
Ms. Sims said the constructed response answers will show a student’s work and will be evaluated on the process to get to the answer. She said multiple-choice questions just evaluate the correct answer regardless of method used. The EOC assessments will be a combination of constructed response and multiple-choice questions.
Senator Winters noted that SB 1 gives districts the option of utilizing EOC assessments, but does not require it. Ms. Sims noted the ACT is willing to work with Kentucky on aligning the United States History course with Kentucky standards. Biology is already a good match to Kentucky standards. She said there is work being done on establishing a multi-state set of standards for social studies and biology, as well as establishing national norms to compare Kentucky against.
In response to questions from Senator McGaha, Ms. Stafford said students can choose to complete the multiple-choice questions on-line or using paper and pencil. On-line results can be seen by students within 24 hours and paper administration requires 10 days from the time ACT receives documents to process and post online reports. Ms. Sims noted that there has to be hand scoring involved with the constructed response portion of the test. Senator McGaha noted that 10 days is not a long turnaround time for the scores except if it occurs at the end of the school year.
Ms. Sims said the KBE made the decision for EOC assessments to account for 20 percent of the student’s grade. However, she said this is a suggestion and school districts still have the flexibility to decide how they will include the EOC exam score into a student’s grade. Many schools will want to link the score to a student’s grade in order to provide student accountability at the high school level.
Dr. Holliday said the KDE could not require school districts to use 20 percent as a mandatory percentage, but strongly encouraged them to use the 20 percent as a minimum threshold. Ms. Sims said it is important for the school districts to report what percentage they are using for the exam in the student’s grade. Senator McGaha said the timeframe of reporting back scores and allowing school districts to determine what percentage the EOC exam will weigh in the student’s grade is very troublesome. He would like to delay the implementation until these issues are resolved.
In response to a question from Senator Westwood, Dr. Holliday said it is extremely rare for schools to chose the paper and pencil option. KDE is helping school districts to implement the on-line assessments, and 90 percent of Kentucky schools have this capability. He noted that the constructed response scores will be used for school accountability, rather than student accountability. All districts and schools will be accountable for the same measures, including multiple-choice and constructed response.
In response to a question from Representative Belcher, Ms. Stafford said it is up to the teachers to print out individual score reports and distribute to students to share with their parents. Ms. Sims said she wants to work with ACT to allow parents to see student scores online.
In response to a concern from Senator Westwood, Dr. Holliday said the ACT covers national norms and ensures there is adequate student sampling. He said the National Technical Advisory Panel on Assessment and Accountability (NTAPPA) is an independent group appointed by the Education Assessment and Accountability Review Subcommittee (EAARS) that has also endorsed this approach. The state board has not finalized the school district accountability model, and anticipates it being finalized in the August 2011 meeting. He said one of the administrative regulations that will be sent to EAARS for regulatory review deals with EOC assessments.
Office of Education Accountability’s 2010 Annual Report
Ms. Karen Timmel-Hatzell, Division Manager, Investigations, Office of Education Accountability (OEA), said KRS 7.410(2)(c)4 allows OEA to investigate allegations of wrongdoing; make referrals to other agencies with jurisdiction; recommend legislative action to EAARS; and submit reports to EAARS at each regular meeting. She said KRS 160.345(9)(b) allows OEA to investigate violations of the school-based decision making council (SBDM) statute, and resolve conflict or forward to KBE for action.
In response to questions from Senator Westwood, Ms. Timmel-Hatzell said the majority of cases that the OEA receives to investigate are not related to SBDM’s. However, the complaints with SBDM’s are generally associated with the principal trying to rush the election of the membership. She said the statute calls for the teachers to conduct their own election and the parent organization in the PTO will conduct their own elections. The principal should not have anything to do with who gets nominated unless there is a need to find a minority candidate.
Ms. Seiler encouraged the members to send requests for items to be researched to the OEA. She said it is important for OEA to receive the requests early because of data issues.
Dr. Ken Chilton, Director of Research, OEA, discussed the 2011 research agenda approved in December 2010. He said the “Kentucky State Testing Data Validation: Processes, Missing, Pieces, and Results” will describe Kentucky’s current data validation processes at the development, implementation, and post-testing levels for each test. OEA will review literature for best practices for secure testing, methodologies for detecting potential problems, and actions to address outliers. A model will be developed for future data validation efforts and will discuss the fiscal implications. Finally, OEA will provide outlier results based on the most recent KCCT and EPAS analyses.
Dr. Chilton said on-behalf payments will be reviewed including analyzing KDE expenditures from 2006 through 2010. The review will focus on administrative fees, flex spending, health and life insurance, Kentucky teacher retirement, vocational education, School Facilities Construction Commission, technology, and school activity funds.
Dr. Chilton said funding equity would be researched and include an update of 2008 school finance report that used 2006 finance data. The report will measure funding equity by district wealth quintiles and focus on funding trends and equity implications in Kentucky during period of economic decline.
Dr. Chilton said the special education update will identify special education identification trends since publication of OEA’s 2008 report. It will describe challenges facing districts and schools in appropriately identifying and meeting service needs of special education students. The OEA will explore revenue and expenditure trends for special education in Kentucky, and identify any changes in the assessment of special education students.
Dr. Chilton said OEA will compile a Kentucky Educational Excellence Scholarship (KEES) study using data from the Kentucky Higher Education Assistance Authority (KHEAA) and KDE. The study will analyze KEES’ impact on: ensuring access to postsecondary education; encouraging and rewarding achievement; and retaining high achievers in Kentucky.
Dr. Chilton said OEA will use P-20 data to follow a cohort of students from 2009 high school seniors through their first year of college in 2010. College and career readiness will focus on persistence by race, income, and sex. The report will examine trends in two-year, four-year, and vocational education paths. It will explore relationships between ACT, GPA, and college performance, and demonstrate the strategic value of a P-20 system.
Dr. Chilton said OEA compiles and updates two annual reports: “Kentucky District Data Profiles” and the “Compendium of State Education Rankings”. He reviewed the past research reports and explained to members that they can locate these reports on-line.
In response to questions from Senator Neal, Dr. Chilton said the College and Career Readiness study will focus on following high school seniors through postsecondary, vocational, military, or their career paths. It is basically linking the student’s outcome from K-12 with their postsecondary paths. Ms. Seiler noted this is possible for the first time by accessing the P-20 data. She is not sure how comprehensive the data is at this point, but will let Senator Neal know. Dr. Chilton said the quality of the data will ultimately depend on data availability and how complete the data is.
Mr. Keith White, Research Analyst, OEA, said the Senate Bill 1 (2009) initiative includes OEA making updates on common core standards and Kentucky’s proposed accountability model. The revised accountability model will include a Next-Generation Learner’s Model. Assessment development will include valid instruments and valid inferences. The new common core standards for reading and mathematics will hit Kentucky classrooms in the fall of 2011 and initial review of the revised standards, by testing and education professionals indicates that the new standards are more rigorous than current standards. The impact of this increase in rigor will become apparent as the new standards become part of classroom instruction and formative assessments.
District Data Profiles
Dr. Chilton said the purpose of the “Kentucky District Data Profiles School Year 2009-2010” study is to provide easy access to commonly used education data. It allows longitudinal analysis of district data trends, and can be used for comparative purposes.
The organization of the report is in the format of a data dictionary. It lists state and district profiles and highlights overview and trends, staffing, finance, performance, and comparative tables. At the end of the district profiles, a Kentucky-wide profile is included for comparative purposes. This is followed by five tables that sort districts by adjusted average daily attendance, free and reduced-price lunch percentage, per-pupil state revenue, per-pupil property assessment, and junior composite ACT average for the 2010 school year. These tables are provided to allow profile users to identify similar districts for comparative purposes.
Dr. Chilton said overall, the “Kentucky District Data Profiles”, provides a basic snapshot of each school district. Given differences in district size, geography, and socioeconomic conditions, the profiles are not conducive to direct comparisons of district effectiveness. The profiles present a broad array of indicators across multiple areas of interest.
In response to a question from Senator Westwood, Dr. Chilton said they are using KDE’s numbers for linking the demographic profiles and dropout rates for each school district. He noted the formula is the same for each school district.
Senator McGaha made the motion to accept the “Kentucky District Data Profiles” report and Representative Farmer seconded the motion. The report was adopted by voice vote.
Adjournment
With no further business before the committee, the meeting adjourned at 12:00 p.m.