Call to Order and Roll Call
The Education Assessment and Accountability Review Subcommittee met on Tuesday, December 7, 2010, at 9:30 AM, in Room 169 of the Capitol Annex. Representative Kent Stevens, Chair, called the meeting to order, and the secretary called the roll.
Present were:
Members: Senator Jack Westwood, Co-Chair; Representative Kent Stevens, Co-Chair; Senators Vernie McGaha, and Ken Winters; Representatives Bill Farmer, Mary Lou Marzian, and Harry Moberly Jr.
Guests: Jim Thompson, Legislative Liaison, Education and Workforce Development Cabinet; Keith White, Office of Education Accountability; Clyde Caudill, Legislative Agent, Jefferson County Public Schools and Kentucky Association of School Administrators; and LuAnn Asbury, Kentucky Education Association.
LRC Staff: Sandy Deaton, Audrey Carr, Ken Warlick, Janet Stevens, and Janet Oliver.
Approval of Minutes
Representative Marzian made a motion that the minutes of the October 12, 2010, meeting be approved. Representative Farmer seconded the motion and the motion was approved by voice vote.
Review of 703 KAR 5:171 (Repeal of 703 KAR 5:170) and 703 KAR 5:190
Kevin Brown, Legal Counsel, and David Cook, Director, Division of Innovation & Partner Engagement, Kentucky Department of Education (KDE), explained the regulatory changes. Mr. Brown said that 703 KAR 5:171 was promulgated to repeal 703 KAR 5:170 relating to the Highly Skilled Educator Program. Regulation 703 KAR 5:190 establishes a new program utilizing education specialists selected pursuant to KRS 158.782 to provide assistance to persistently low-achieving schools. The regulation creates Centers for Learning Excellence in KDE’s District 180 Assistance Program. The centers are located in state universities and include an education recovery director; education recovery leaders; education recovery specialists; math and reading intervention specialists; and an advisory committee. Mr. Brown said the centers are already in place and staffed to provide assistance at the recently identified ten persistently low-achieving schools.
In response to questions from Senator Winters, Mr. Brown said that the three participating universities were selected based on their proximity to the first cohort of identified persistently low-achieving schools. The three universities are the University of Louisville, Morehead State University, and Western Kentucky University and more sites will be added as low-achieving schools needing assistance are identified. Mr. Brown said the commissioner of education has broad authority over programmatic areas and K-12 education to hire specialists and other personnel to assist low-achieving schools in addition to the specific authority set forth in KRS 158.782.
Upon motion by Representative Farmer, seconded by Representative Marzian, the regulatory changes were approved by voice vote.
Office of Education Accountability’s Proposed 2011 Study Agenda
Marcia Seiler, Director, Office of Education Accountability (OEA), said the proposed OEA study agenda for 2011 includes six topics and the annual reports on district data profiles and compendium of state education rankings. OEA will review expenditures of on-behalf payments appropriated during the past five years. On-behalf payments are disseminated by KDE and other state agencies to school districts for administrative fees, health and life insurance, teachers’ retirement, vocational education, technology, and other payments not included in SEEK funding. A funding equity analysis will be completed to update the OEA school finance report provided to EAARS in 2008 and will include all available data for years since FY 2006. A study will be conducted on the validity of Kentucky test data including a review of processes, missing pieces, and results. The study will examine test security, testing environments, and validation efforts and the National Technical Advisory Panel on Assessment and Accountability (NTAPAA) will be asked to provide guidance and input. The 2009 OEA special education study will be updated to include information on how special needs students are identified and assessed and the progress being made to implement the recommendations contained in the 2009 study. OEA will conduct a study on career and college readiness rates using Kentucky P20 Data Collaborative information to provide benchmarking and validation data on students who exited high school in the spring of 2009. OEA will also conduct an analysis of the Kentucky Education Excellence Scholarship (KEES) program including how funding is awarded and if there are any trends based on various student demographics and geographic locations. Ms. Seiler said she hopes that some reports will be ready for presentation to EAARS as early as May 2011.
In response to a question from Senator Westwood, Ms. Seiler said it is anticipated that the funding equity study and the review of on-behalf payments will be conducted concurrently since they are intertwined.
Senator Westwood moved that the 2011 OEA study plan be approved. Senator Winters seconded the motion and the motion was approved by voice vote.
Review of Teacher Evaluation and Compensation
Ms. Brenda Landy, OEA Research Analyst, presented the report on teacher evaluation and compensation. She said Kentucky’s system provides for more evaluations of first year teachers and it takes longer to achieve tenure than in most states. As in many states, evaluations are not used to their fullest potential to improve performance and states tying student performance to evaluations and compensation are receiving mixed results. Kentucky is currently conducting a three-year initiative to improve the evaluation process with involvement from major stakeholders. OEA research included review of district evaluation forms and research and policy literature and information obtained in an online survey in which responses were received from 14,000 teachers and 700 principals with representation from all the districts.
Ms. Landy said teacher evaluation is an ongoing process of measuring performance based on identified criteria and providing formative feedback to teachers followed by a year-end summative evaluation. Frequency of evaluations is usually based on teacher experience with first year teachers in the Kentucky Teacher Internship Program (KTIP) being evaluated multiple times and tenured teachers receiving a summative evaluation about every three years. The primary evaluator is the immediate supervisor, who is generally the school principal. The summative evaluation sums up and draws conclusions from all performance measures and should be conducted in person between the evaluator and teacher. The evaluation should include recommendations to be incorporated into the teacher’s professional growth plan and establish training opportunities and resources for the teacher’s self-improvement efforts. A consequence of a poor evaluation can be non-renewal of the contract which is relatively straight forward for new teachers during the first four years but much more difficult to accomplish once a teacher has achieved tenure. Detailed written documentation of performance issues or improper conduct following due process procedures set out in the statutes and collective bargaining agreements in various districts are necessary to terminate a tenured teacher contract.
All evaluators are required to have an initial two-day training and ongoing training. The evaluation process is governed by KRS 156.557 which established ten broad standards for evaluating performance with further guidance set forth in 704 KAR 3:345. Each district may develop their own evaluation forms and processes although most districts use model forms and guidance provided by KDE.
Survey results indicated that strengths in the current system include overall compliance with requirements, satisfaction with systems used by districts, and effective monitoring of non-tenured teachers, which exceed requirements in other states. Weaknesses included burdensome paperwork rather than useful tools for improving performance; lack of administrative support and resources for self-improvement efforts; and lack of action on behalf of administrators to improve the performance of ineffective teachers. Information on evaluations and dismissals is not collected by KDE so complaints about disciplinary actions are difficult to confirm. KTIP data shows that about one percent of teachers do not complete the program. Evaluation criteria are often viewed to be subjective and do not include student achievement measures and rating scales rarely distinguish excellent and poor performance from just average. Teacher pay is not governed by evaluations but by years of experience and formal education, which impacts the credibility of the system.
A three-year initiative to provide recommendations on improving system is underway and a working conditions survey to be disseminated in the upcoming year may provide useful feedback. Initiatives being undertaken in other states include stricter tenure policies, annual evaluations for all teachers, and using student growth as an evaluation criterion. Concerns at the national and state level include whether federal funds will continue to be available to support incentives to improve teacher performance and pay. There is also concern about using student performance as a criterion for evaluation when numerous other factors beyond teacher control may affect student growth, such as poverty, lack of parental involvement, school culture, and not holding students accountable for poor performance. OEA found that the differentiated compensation pilot in 2002-2004 had mixed results. Teachers responding to the survey prefer financial incentives for teaching in low-performing schools and teaching difficult subjects and 44% favor incentives for above average student growth. Principals would like more flexibility in evaluating tenured teachers and a reduction in the paperwork and procedures required to dismiss tenured teachers, although information from other states indicates that procedural changes had minimal impact on dismissing ineffective teachers because of social, political, and teacher supply issues. Experts believe that systematic and holistic changes are needed to improve the evaluation system and include sustainability of funding for incentives, improving working conditions, providing advancement opportunities, and using multiple performance measures.
In response to a question from Representative Stevens about the lack of teacher response to the online survey, Ms. Landy said principals may have had more time to respond to the survey and greater access to technology than classroom teachers and teachers may be more satisfied with the current evaluation system.
In response to a question from Representative Farmer, Ms. Landy said that each person responding to the survey was provided a unique identifier and that duplication would have been difficult.
In response to a question from Senator Winters, Ms. Landy said that the national data showing 96% of teachers are paid based on experience and education is several years old. She said that a working condition survey will be disseminated next year and it is anticipated that working conditions, including safety, cleanliness, school culture, and other factors, may be as important as monetary incentives.
In response to questions from Senator Westwood, Ms. Landy said evaluators attend a two-day training provided by the Kentucky Association of School Administrators and are required to complete 12 hours of ongoing training during each subsequent two-year period. Ms. Landy said many of the survey respondents believe that poor performance does not produce a bad evaluation with consequences.
In response to a question from Senator McGaha, Ms. Landy said the information that ineffective teachers do not receive poor evaluations was gleaned from open-ended comments from teachers. Senator McGaha said that the data may be skewed since evaluators generally conduct evaluations in private. He said as a former principal he could attest to the difficulty in trying to remove an ineffective tenured teacher.
Representative Stevens said he also could attest to the difficulty of removing ineffective teachers. He said he believes the Kentucky Teachers Internship Program is proving to be an effective tool in training new teachers.
Senator Westwood announced that this is the final EAARS meeting for Representatives Kent Stevens and Harry Moberly and Senator Ed Worley and thanked them for their service and dedication to improving the education of Kentucky’s students.
Compendium of State Education Rankings 2010
Ms. Landy said the rankings in the compendium are based on information from the fifty states and the District of Columbia but the data contained in the tables focuses on the states in the Southern Region Educational Board and states that border Kentucky. The chapter on demographics provides various comparisons of factors that may affect academic performance, such as poverty and non-English speaking students. Ms. Landy said the services chapter reiterates OEA concerns about how students with disabilities are identified. The chapter relating to fiscal matters shows that even after adjusting for geographic differences and costs of services and goods, Kentucky’s per pupil revenues and expenditures are still low compared to national averages, although teacher salaries are close to the national average. The 2009 National Assessment of Education Progress (NAEP) results showed that Kentucky’s fourth grade reading and math scores increased faster than the national average and eighth grade math scores were close to the national average. NAEP results do not include assessment data for students with disabilities and language barriers or students tested with accommodations. The percentage of students passing advanced placement (AP) examinations continues to improve. Although Kentucky’s graduation rank had improved to 23rd between 2002 and 2006, it has declined to 32nd in 2008. All Kentucky juniors have been required to take the ACT since 2008. In 2010 five other states also required 100% participation for ACT testing and the data reflects that higher participation rates results in lower average scores. In 2008, when the participation rate was 72%, Kentucky ranked 24th overall nationally on the composite score but it now ranks 50th. The average composite score in Kentucky is 19.4, which is a half-point below the national average of 20.
In response to a question from Representative Moberly, Ms. Landy said that the National Center for Education Statistics comparable wage index is used to rank teacher salaries but OEA also factors in the geographic cost of living index in the ranking. Representative Moberly said he noted in the report that the student/teacher ratio is below the national average which is a positive step.
In response to a question from Senator Westwood, Ms. Landy said on-behalf payments and benefits are not included in teacher salary rankings. Senator Westwood said he would be interested in a comparison of benefits with other states. Ms. Seiler said the information could be included in the 2011 study of on-behalf payments.
In response to a question from Representative Stevens, Ms. Landy said accurate comparisons of ACT and SAT results is difficult because many states do not require all students to take the tests and, even when required to take the examinations, students may not put forth their best effort if they are not planning to attend college. Senator Winters said it was recognized when the ACT was mandated for all high school juniors that the overall average would decrease but the benefit of having many more students pursue postsecondary education based on ACT test results outweighs having a lower state average.
In response to a question from Senator McGaha, Ms. Landy said that the definition of school administrator does not include counselors. She said Kentucky ranks sixth in the number of school administrators because it is a rural state with many small schools.
Analysis of Collective Bargaining Agreements in Kentucky Districts
Dr. Ken Chilton, Director of Research, OEA, said collective bargaining agreements in Kentucky districts, with the exception of Jefferson County, has minimal impact on school staffing, school-based decision making management at the school level, and teacher evaluations, except in Jefferson County. There are nine districts in Kentucky with collective bargaining agreements representing approximately 23% of Kentucky’s teachers and 22% of Kentucky’s students. Membership in teacher associations in those districts range from a high of approximately 95% in Jefferson County to a low of about 35% in Newport Independent although the average is 60-80%. With the exception of Jefferson County, political activism by the teacher association is limited. The main focus in most contracts is teacher assignment issues regarding transfers, evaluations, non-renewal of contracts, termination, and related personnel actions.
OEA’s research included a review of all of the contracts and memoranda of agreement and responses to a questionnaire. Also site visits were conducted in various schools in the nine districts with collective bargaining agreements and also in schools in four non-contract districts to provide an adequate comparison of contract effects.
Dr. Chilton said that statutes do recognize the right of teacher unions to organize in the state. The study focused on the provisions of KRS 160.345(2)(h)1., which is the school-based decision making (SBDM) council law that recognizes the right of a collective bargaining agreement to utilize transfer provisions that may deviate from SBDM staffing protocol and allows collective bargaining agreements to supersede the statutory process for staffing followed in non-contract districts. The study also focused on KRS 160.380(1) (c) relating to the filling of vacancies in districts with collective bargaining agreements. Contracts can be modified between inception and renewal by memoranda of agreement, some have deviation clauses that permit amendment, and many of the contracts could be reopened at various times for renegotiation of terms. OEA believes that some of the modifications to the contracts have not been transparent.
Dr. Chilton said that the principal in contract districts that have strong seniority preferences are somewhat limited when filling vacancies, although most administrators in those districts had little complaint about the effect on school staffing. Contracts can be used to determine the order of interviews and guarantee courtesy interviews although principals said candidates are chosen based on experience and related factors. OEA did find that seniority provisions in Jefferson County’s contract has more significant impact on SBDM council and principal choice in hiring and transferring staff, especially on the distribution of teachers across the district. An analysis of the transfer list for 2009 revealed some delineation in transfer requests between high demand and low demand schools. Schools with the lowest reading and math proficiency levels had much higher numbers of teachers with three years or less of experience and in some cases higher rates of teacher attrition. Teachers in Jefferson County can apply for transfer after completing the KTIP program. The schools with the highest rate of attrition tend to be in neighborhoods that have negative perceptions. One of the schools was Frost Middle School which has been identified as a persistently low-achieving school. Tests data in Jefferson County showed a correlation between low test results and transfer and teacher experience issues. Dr. Chilton said that principal and school council authority is limited in Jefferson County although other contract districts did not have issues with council authority. An example was that a math curriculum change in Jefferson County required a two-thirds vote through a deviation clause in the contract, which may conflict with the statutory provisions relating to the autonomy of SBDM councils.
Dr. Chilton said contracts have very little impact on teacher evaluations although four of the collective bargaining agreements expressly forbid the use of student test data to evaluate teachers. Due process procedures outlined in the statutes are often included in the contracts and principals in all schools said the process of removing ineffective tenured teachers is too burdensome and takes too much time away from other important tasks, such as providing instructional leadership in the building. Newer teachers in most districts, including contract districts, are less likely to be involved in teacher associations and tend to work toward developing collaborative relationships with administrators.
Based on information collected, Dr. Chilton said collective bargaining agreements, memoranda of agreement, and contract modifications need to be more transparent. Also districts need to ensure that seniority based transfer provisions in collective bargaining agreements are aligned with 2009 Senate Bill 1 and 2010 House Bill 176 to promote student achievement.
In response to a question from Representative Moberly, Dr. Chilton said the statutes referenced in his presentation were passed in the 1990-1992 timeframe. He said the study should provide useful information to help identify some of the issues that may be impacting the low performing schools in Jefferson County.
In response to a question from Senator Westwood, Dr. Chilton said that language in some of the contracts may conflict with statutes relating to the authority of SBDM councils, such as requiring a two-thirds vote to implement a curriculum change. It was noted by Kevin Noland, former general counsel for KDE, who was in the audience, that the authority of councils relating to curriculum and assignment of students and teachers cannot be overridden by a collective bargaining agreement.
In response to a question from Representative Moberly, Dr. Chilton said that differentiated pay had been discussed in contract negotiations for hard to staff schools. He offered to provide more detailed information to Representative Moberly on specific comments or contract language.
Assistance to Low-achieving Schools and Districts: Strengths, Limitations, and Continuing Challenges
Ms. Deborah Nelson, Research Analyst, OEA, said that although providing assistance has resulted in substantial gains in some low-achieving schools, many schools have achieved limited or no gain after several years of intervention. Intervention measures often do not address attracting and retaining building leaders and teachers with the skills necessary to be successful in certain school environments, especially when local community support is not available. OEA analysis included review of assessment, demographic, and staffing data; review of research information; and interviews with over 55 educators at all levels of schools in Kentucky.
Ms. Nelson said that previously Level III schools, which were identified as the bottom third of schools in Kentucky, were eligible for assistance through state funding and Title I schools that had not met adequate yearly progress (AYP) goals under federal guidelines received federal funding assistance. Level III schools qualified for the services of highly skilled educator (HSEs), scholastic audits, and grants from the Commonwealth School Improvement Fund that could be used to pursue new strategies. Title I schools were eligible for No Child Left Behind (NCLB) grants. The smaller grant was the Section 1003(a) grant for schools that had not met AYP for two consecutive years and the larger grant was issued pursuant to Section 1003(g) for schools that had not met AYP for five consecutive years and was generally in the $100,000 range. During the period, Kentucky did not choose the severe strategies of restaffing or closing a school but chose major restructuring in some schools.
Historically, KDE had the primary statutory responsibility for identifying and providing intervention assistance to the lowest-achieving schools with district superintendents and administrators taking a lesser role and receiving no direct funding to implement strategies, even though research shows that districts should play a critical role in identifying schools for assistance. Even though the Kentucky Education Reform Act increased the authority and responsibility of local school councils, some districts do not accept responsibility for poor student outcomes while others, even in high poverty areas, are exceeding state averages for student achievement.
Currently the only schools that can receive assistance are Title I schools meeting the federal definition for improvement and persistently low-achieving schools identified by the criteria set forth in 2010 House Bill 176. Title I persistently low-achieving schools are also eligible for Section 1003(a) and (g) grants and the amount of funding from the 1003(g) grants can be as high as $1.5 million over a three-year period. State funding is available in the current year to provide an education recovery leader to assist the principal and one reading and one math education recovery specialists to provide content specific assistance. Each persistently low-achieving school is also eligible for a leadership assessment, which is similar to the scholastic audit except it directly assesses the capacity of the council and the principal to carry out the interventions. KDE has recently reorganized and coordinates all forms of assistance to schools under the Division of District 180. Centers for Learning Excellence have been established at three state universities to coordinate assistance to the schools needing assistance. Interventions may include using external management, restaffing, school closure, or transformation. Six of the ten recently identified persistently low-achieving schools, which are all located in Jefferson County, have chosen the restaffing model and the remaining schools have chosen the transformation model.
State funding for the HSE program declined from $5.5 million in 2008 to $5.2 million in 2011 and no funds have been appropriated for the HSE program for 2012. The HSE appropriation is currently being used to fund the education recovery personnel at the Centers for Learning Excellence. No funds have been identified at the state level to continue funding the centers in the upcoming year, although districts may use their substantial federal funds to continue the services. The Commonwealth School Improvement Fund has declined from about $1.5 million in 2008 to $1.4 million in 2012 and KDE was granted authority in 2009 to use this funding source to pay for scholastic audits and leadership assessments instead of giving the funds directly to the district. In 2009, federal funding exceeded state funding by $15 million. Currently approximately $33 million is flowing to the schools, mostly from the 1003(g) fund to provide grants to low-achieving schools.
Ms. Nelson said the only valid assessment data available during the last decade for all schools is the Kentucky Common Core Test (KCCT) data for reading and mathematics. No historical data is available on college and career preparedness. Although NAEP data does not substantiate KCCT data, KCCT results showed a substantial improvement in all Kentucky elementary schools in reaching proficiency rates in combined reading and math scores from 2000 to 2009. Similar data was available for middle and high schools. The lowest deciles of schools lagging behind state averages remained unchanged during the period.
Ms. Nelson said the system that had been used to identify low-achieving schools under the Commonwealth Accountability Testing System (CATS) was not totally accurate. Based on current federal guidelines, the overwhelming majority of Kentucky schools will soon be identified as needing assistance. Although KDE has revised its method of identifying low performance, using only quantitative data does not accurately reflect the specific challenges faced by the schools, such as social and political culture, experience of teachers, and other contributing factors. In the future audit data may compliment quantitative data in identifying schools needing intervention strategies.
Ms. Nelson said based on a review of 2009 KCCT data, less than 50% of high school students were proficient in math and reading, and about 83% had less than a 50% combined reading and math proficiency score. Some of the characteristics of the lowest-achieving schools have been higher than average poverty and minority student rates. In Kentucky, many of the lowest achieving schools are located in Jefferson County and other urban areas, which is consistent with nationwide data. Ms. Nelson said data has shown little correlation between assistance and achievement gains in low-achieving schools as a group, although there have been exceptions. Also, intervention efforts appear to be less effective in schools facing deep challenges evidenced by the fact that some of the lowest achieving schools have been receiving assistance as long as eight years without any significant improvement in assessment scores. Ms. Nelson said that it is difficult to conduct a rigorous analysis of the reasons for failure and to determine what issues need to be addressed because of the lack of true comparison groups. The lack of leadership skills appears to be a key factor in achieving goals and holding students and staff accountable for academic improvement. Low-achieving schools are characterized by inconsistent expectations for students and staff, disunity among faculty, low morale, and a perception among teachers that challenges, lack of student discipline and large numbers of students below grade level, remain the same even after interventions have occurred. The use of scholastic audits and an infusion of federal and state monies have proven ineffective except in those schools where changes have already begun to occur.
A significant challenge has been attracting and retaining effective building leaders and teachers. Although Jefferson County has chosen the restaffing option for all six of its low-achieving schools, which means that 50% of teachers were replaced with more effective teachers, a review of teacher credentials showed those teachers are being replaced with a high percentage of new teachers with very few teachers having greater than three years experience. The same situation is occurring in the rural areas, especially in finding teachers qualified to teach mathematics. Another significant challenge is lack of support from school and district administrators and local school board members to implement the changes necessary to increase student learning.
Ms. Nelson said that no single strategy will be effective in all schools. Scholastic audits and working condition surveys may help identify specific needs. Other strategies may be the development of pipelines of teachers and leaders with skills necessary to succeed in difficult environments; providing recruitment and retention incentives; working directly with district administrators and local school boards to increase their capacity to monitor schools and provide support; and rigorous evaluations of intervention measures. OEA recommends that KDE revise the audit process to include staffing, student mobility, and other data; explore alternative forms of providing assistance to schools that have not responded to intervention measures; and establish tiered interventions in districts that have schools that are not responding to interventions. OEA recommends that the General Assembly allow KDE to continue to use the Commonwealth School Improvement Funds for assistance efforts; update the statutory responsibilities of local school board members to include monitoring of student learning in schools; support district efforts to improve that learning; and link future funding to rigorous evaluation.
In response to a question from Senator McGaha, Ms. Nelson said that the NCLB 1003(g) funds are being used for Title I schools in their fifth year of consequences. Additional funds are being received through the American Recovery and Reinvestment Act (ARRA) allocations for both Title I and non-Title I persistently low-achieving schools based on criteria established by the federal government. The ARRA funds were approved for two years and may not be a renewed funding source.
In response to a question from Senator Westwood, Ms. Nelson said some low-achieving schools have made remarkable progress, others have improved, and others have made little or no progress as a result of state assistance and intervention. Each school is unique and no single intervention has proven effective in every school. Senator Westwood said that the legislature could not correct cultural issues and the challenge seems to be determining which measures will be effective in the various districts.
In response to a question from Senator McGaha, Ms. Nelson said an HSE could only serve three years in the HSE program. Schools that had been in assistance for more than three years would have had more than one HSE assigned. Ms. Nelson said there have been instances where a change in school leadership has resulted in dramatic increases in student achievement. Ms. Connie Lester, Branch Manager, Supporting Schools & District Branch, Division of District 180, KDE, said HSEs were given one year contracts and their performance was monitored through their reports and school data.
Approval of Reports
Senator Westwood made a motion that the four reports discussed at the meeting be accepted. Representative Stevens seconded the motion and the motion was approved by voice vote.
Adjournment
There being no further business to discuss, the meeting adjourned at 12:00 Noon.