Chapter 5: Evaluation and Accountability

Evaluation and Accountability

Accountability

Accountability work is everyone’s responsibility. This includes students, parents, and all personnel at the school and district levels— teachers, instructional assistants and administrators. Rather than considering evaluation as a specific event (or an annual report), we consider it to be part of our daily work. We are all accountable for ensuring that programs are optimally effective. All district personnel are required to follow the procedures specified in this Master Plan.

The Home-School Compact outlines how parents/guardians, the entire school staff, and students will share responsibility for improved student achievement.

Multilingual Services Council

The Superintendent or designee convenes and leads the Multilingual Services Council. The council will convene at a minimum of four times per year. This council is designed to ensure that district stakeholders have a mechanism to plan and coordinate the implementation of English learner instructional services. The council is composed of Superintendent, Assistant Superintendents, Department Directors, the Coordinators, as well as representatives from the following groups: Principals, Coordinator for EL Instruction, teachers, Curriculum Instruction and Assessment committee members, and Association representatives. Others participate as needed.

The Multilingual Services Council meets to discuss topics pertaining to the implementation of programs for English learners in the district. The council also:

1. Serves as a clearinghouse for strategies, ideas and suggestions for English learner programs, staff development, as well as a focus group for collaborative problem-solving
2. Provides a forum to evaluate and determine that practices, resources, and personnel are being used effectively to implement the district’s programs for English learners 
3. Makes recommendations for reporting on the performance of English learners 
4. Contributes to the review of the LCAP Federal Addendum, Title III  provisions not addressed in the District LCAP.
5. Ensures communication and integration as the council continues to bring clarity, consistency, compliance and continuing improvement to the District’s programs for English learners

The district will conduct an annual evaluation of programs and services for English learners. Programs and services for English learners are structured around seven goals. The evaluation activities will focus on the evaluation questions listed in Table 5.1: Evaluation Design – Goals, Evaluation Questions, and Measures/Source

Curriculum, Instruction, & Assessment Committee (CIA)

The committee is composed of both school site and district educators. The purpose of the committee is To provide feedback, perspective and leadership in leading curricular, instructional and assessment learning plans that support the implementation of the district LCAP goals of Conditions of Learning, Pupil Outcomes and Engagement.

Specific areas of focus include:

• Identify curriculum, instruction and professional learning needs based on data
• Assess the alignment between intended, taught and assessed learning in real classroom practice
• Assess the effectiveness & transfer of professional learning
• Make recommendations to Asst. Superintendent and school board
• To ensure committee recommendations reflect the rigor students will need to be college and career ready

Evaluation Design and Goals

The district will conduct an annual evaluation of programs and services for English learners. Programs and services for English learners are structured around seven goals. The evaluation activities will focus on the evaluation questions and the measures/sources listed in the table below.

Goals	Evaluation Questions	Measures/Sources
1. English learner programs are fully implemented as described in the Master Plan for English learners.	1.1 Are English learner programs (ALA, ELM, and DLI) fully and consistently implemented in ways that meet or exceed requirements of state and federal law? (See Chapter Two for program descriptions) 1.2 To what extent is the content of the Master Plan for English learners used by teachers, administrators, and parents as a tool to meet the needs of English learners and staff?	Principal assurances (self-review)Instructional Schedule Reviews English learner Document Review Site Visits (District and Site initiated)Data Support Team visits Classroom Walk-throughs using Observation Protocol such as OPAL ALA to ELM to Mainstream English Monitoring CLAD/BCLAD Staffing Professional Development
2. English learners will receive a well-rounded, holistic education with full access to the core academic curriculum.	2.1 How are English learners given full access to the core curriculum? 2.2 How are English learners receiving best instructional practices (e.g.,  Research-Based Lesson Design)? 2.3 How are English learners receiving daily Integrated and Designated ELD leveled instruction?	Instructional schedules Site Walk-throughs using Observation Protocol such as OPAL Daily Designated ELD Integrated ELD Content and Language Objectives Lesson Plans Data Support Team Visits Assessment Results Evidence of English learner Scaffolds Student Grouping/ Small Group Instruction Data review in SMART system Content  Integration Physical Fitness Data
3. All English learners will master the English language efficiently and effectively.	Are English learners making adequate progress on ELD assessments? 3.1 Do English learners meet the state’s performance indicators (either green or blue)  with regard to English learner gains on the ELPAC test? 3.2 How are English learners making progress on table 5.3 EL Expected Progression and Growth Targets? 3.3 Are all English learners reaching reasonable fluency (Oral score of Level 3 or higher) on the ELPAC in 2 years or less? If not, are students receiving an appropriate intervention? 3.4 Are all English learners reclassified in accordance with the EL Expected Progression and Growth Targets? If not, are students receiving an appropriate intervention?	English Learner Performance Indicator on California Schools Dashboard ELPAC  Data Analysis Designated ELD Assessment Analysis Progress toward reclassification based upon the Expected EL Progressions chart At-Risk for Long Term English learner (LTEL) Annual Data RtI2 (Progress Monitoring)
4. English learners will achieve academic success comparable to their EO peers.	4.1 Are English learners making adequate progress on local assessments  in content areas? 4.2 Do English learners (and R-FEPs) meet the state accountability measures  in English Language Arts and Mathematics? 4.3 How are English learners proportionally represented (refers to equitable student participation) in Special Education and GATE referrals?	Data Analysis (SBA/CAA) (ELA 3rd-6th grade), Math 3rd-6th grade, Science (5th grade), Physical Education California Schools Dashboard Local  Assessment results Special Education / GATE demographics
5. Parents / Guardians of English learners and R-FEPs participate meaningfully in their children’s education.	5.1 How are parents of English learners and R-FEPs participating in school activities (parent-teacher conferences, volunteer in class, parent trainings, leadership roles, etc.)? 5.2 What measures are used to ensure that parents are knowledgeable about EL program placement, language acquisition and their child’s academic progress? 5.3 Is the rate of parent engagement increasing?	Attendance and participation at parent-teacher conferences, School Site Council, ELAC, and DAC/DELAC meetings Parent Surveys Parent Trainings (ELPAC, Leadership Development, EL Master Plan) School Events At-home parent and family engagement Community involvement Collaboration with Anaheim Union High School District
6. Rates for English learners and R-FEPs in categories indicating risk for school failure are no greater than those for EOs.	6.1 Are English learners (and R-FEPs) over-represented in the following categories?  If so, is there an annual decrease?: Excessive absences and tardies Suspensions, expulsions, other discipline Retentions in grades TK–6 Enrolled in Alternative Programs due to deficiencies Percent of Standard Not Met on SBA	Attendance data Discipline data SBA levels LTEL data Table 5.3 EL Expected Progression and Growth Targets California Schools Dashboard
7. Students enrolled in the Dual Language Immersion Program will master language skills in English and a partner language (e.g., Spanish, Korean).	7.1 What percentage of all students (ELs, I-FEPs, R-FEPs, EOs) participating in DLI score at/above proficient on the partner language proficiency assessment, such as LAS Links? 7.2 What percentage of all students (ELs, I-FEPs, R-FEPs, EOs) participating in DLI score at/above proficient in English Language Arts and Mathematics on the SBA and Spanish Standards Tests? 7.3 What percentage of students in the DLI program obtains the Biliteracy Attainment Award at 6th grade?	LAS Links en Español (Language Assessment Scales) California Spanish Assessment (CSA) Other partner language assessments (Korean etc.) ELPAC SBA Local Assessments (STAR, DCAs) Smarter Balanced Interim Assessment Blocks (IABs) Biliteracy Attainment Award

Monitoring Program Goals

Goal 1: English Learner programs are fully implemented as described in the Master Plan for English Learners.

District and site staff will periodically monitor implementation of all English learner programs. The primary goal of monitoring is to ensure that every school in the district has effective and compliant programs for English learners. The monitoring process is designed to:

• Establish appropriate expectations based on students’ needs
• Promote full involvement of all stakeholders (administrators, teachers, parents, students) in all phases of planning, implementation and evaluation activities.
• Provide for high levels of coordination between district-level and site-level improvement efforts 
• Ensure that program evaluation is an integral part of school improvement initiatives and activities 
• Provide a basis for review and modification of the Master Plan for English learners every three to four years

This monitoring will consist of the following activities:

• Principal Assurances (self-review) School principals are responsible for the daily, site-level implementation of the Master Plan for English learners. Throughout the academic school year, principals complete sections of the Principal’s Assurances Checklist Principal’s Assurances Checklist A checklist of the deadlines and duties for the principals to facilitate the needs of English Learners. (Form EL-6.1) for English Learner Program Services and submit them to the Coordinator for English Learner Instruction according to the timeline indicated on the checklist. This facilitates ongoing communication with the English learner Specialist, and assists in the monitoring of consistent implementation of this master plan throughout the school district. The Director of DATA DATA Department of Assessment, Translation, and Accountability will review all Principals’ Assurances Checklists after each submission (Fall and Spring). 
• Instructional Schedule Reviews All schools will turn in ELD instructional schedules; create ELD Class Rosters in SMART, and send Master Schedules to the Coordinator for English learner Instruction within the first 3 weeks of the beginning of each school year, or anytime there is a student schedule modification. 
• English learner document reviews State-mandated English learner records and documents will be reviewed annually by DATA staff. These reviews will include evidence of the following: English learner identification, parent notification, placement, assessment, and reclassification. These reviews will be scheduled in all schools on an annual basis. Findings are reported back to EL Services Council who will help develop appropriate next steps as needed. 
• Site Visits and Instructional Walkthroughs (district and site facilitated self-reviews).     These will be conducted yearly. The reviews will include monitoring of ELD schedules and instructional minutes, adjustment of instruction and/or student groupings based on formative assessment,  and level of implementation of EL scaffolds via visits to selected classrooms. The district facilitated reviews will be conducted by teams consisting of the Director of Curriculum and Instruction, curriculum specialists, and the site principal. Site facilitated reviews will consist of school teams which will include the site principal and classroom teachers. An observational protocol such as OPAL will be used to document trends and patterns.  Findings will be compiled and used to determine next steps for school improvement work.
• Ongoing coaching and staff development support the Curriculum Specialist for Emergent Bilingual SupportCoordinator for EL Instruction will help organize site staff development and assist with classroom coaching (see Chapter Seven). The specialist for EB supportCoordinator will assist with the ELD Instructional Walkthroughs and will be responsible for organizing in-depth reviews and any needed follow-up.

Goal 2: English Learners will receive a well-rounded, holistic education with full access to core academic curriculum.

Each site principal is responsible for ensuring that English learners have access to core curriculum. Principals will work with site staff and Coordinator for EL Instruction to determine appropriate placement and instructional setting for English learners. The Director of Curriculum and Instruction will review ELD schedules, ELD groupings by language proficiency levels, and instructional minutes.

Goal 3: All English Learners will master the English language efficiently and effectively. 

The Director of DATA, principals, Coordinator for EL Instruction, and classroom teachers will analyze annual ELPAC results, local English proficiency assessments, and progress toward meeting criteria for reclassification (See Chapter Four). Data will be disaggregated by number of years in US schools (US Entry Date), program placement, initial language proficiency level, and other relevant variables. 

The DATA Department will complete an analysis of ELPAC growth (English Learner Performance Indicator (ELPI)) by district, school, language group and program. The Director of DATA will review and report findings and trends to principals and Coordinator for EL Instruction, who will work with site staff to continue the development of school improvement work.

Goal 4: English learners will achieve academic success comparable to their EO peers. 

Each elementary site principal will review and analyze SBA performance data in English Language-Arts and Mathematics. The analysis will include cross-sectional profiles of performance by ELPAC level as well as disaggregation of data by school, grade level and language group. 

• Data on other indicators will be distributed by the DATA Department using data visualization formats that are standardized across schools. 
• Principals will analyze data with site staff to determine trends, and areas of need, and develop action plans. 
• In addition to reviewing SBA and ELPAC data, principals will analyze data from local assessments to determine progress toward action plans and will modify as needed. The teachers will review data through the PLC and collaborative problem solving processes.

Goal 5: Parents / Guardians of English Learners and RFEPs participate meaningfully in their children’s education.

The Director of DATA and the Curriculum Specialist of Parent Engagement will work with the DELAC and ELACs to develop specific indicators, benchmarks, instruments, and a calendar of procedures for evaluating parent engagement with schools and the district. Sites will report to the district on parent activities implemented during the year, as well as parent and family engagement in a set of activities to be defined in a standardized way across sites. A parent participation survey will be completed each spring and submitted to the district. Data from these forms will then be aggregated to develop a picture of parent and family engagement in the district as a whole and will guide parental involvement practices. 

Goal 6: Rates for English Learners and RFEPs in categories indicating risk for school failure are no greater than those for EOs.

The DATA Department will provide the following data regarding English learners: 

• Suspensions, expulsions, and other disciplinary actions 
• English learners identified for Individualized Educational Plans (IEPs) 
• Retention Rates 
• Attendance 

Goal 7: Students enrolled in the Dual Language Immersion Program will master language skills in the partner language and English. 

Site principals, Coordinator for EL Instruction and school staff will analyze English learner’s Language Arts and Mathematics scores on the state and local assessments in the partner language (Korean or Spanish) and English.  In addition to reclassifying English learners as R-FEP, students in our Dual Language Immersion programs have the opportunity to be reclassified as proficient in the partner language (Korean or Spanish).  Partner language reclassification rates will be monitored annually.  As sixth graders, students participating in DLI also have an opportunity to earn the Pathway to the Seal of Biliteracy Award for exhibiting proficiency in both English and the partner language.  The number of students earning these awards are also monitored annually.

Using Program Effectiveness Information to Improve Implementation and Modify the Program

The evaluation data gathered and the analysis performed provide a rich source of information on program implementation and outcomes. The data will be used at the district and site levels to promote continuous improvement.

Site Level Use of Information

The student data will assist sites to determine strengths and areas of need in their programs, and to make adjustments accordingly. Site level reports identifying progress and proficiency goals will be provided in order to assist school staff in identifying students in need of greater support. Individual teachers and grade level teams will use ELD and observational protocols to do self-assessments on program implementation. School visitations conducted by site staff, administrators and district office personnel will provide an additional perspective on classroom implementation of recommended instructional practices. After gathering and discussing the checklist information, each site will develop an action plan for professional development that establishes needs, long-term goals, and types of training experiences that are needed to improve instruction. The site plan for professional development should be clearly articulated with the district plan for professional development, and should include prioritization and support. Annual goals will be set and included in the school’s Single Plan for Student Achievement.

The site will regularly monitor student outcomes for improved services for English learners. The data reviewed and conclusions reached in the process of site level planning will be shared with the site SSC/ELAC members.

District Level Use of Information 

At the district level, the annual analyses of student outcomes will determine the level of effectiveness of English Learner programs. Putting outcome data together with the site self-reviews will enable district staff to identify areas of strength and target areas in need of improvement on a district-wide basis. This will provide a foundation for program improvement planning, including professional development priorities and plans, in future years. 

English Learner outcome data aligned to the seven goals in this chapter will be shared with DAC-DELAC as well as the LCAP committee for the purpose of eliciting stakeholder feedback and informing our Local Control Accountability Plan.

Evaluating Program Effectiveness 

The district has identified benchmarks for all program options for English learners. These benchmarks follow table 5.3 EL Expected Progression and Growth Targets. These tables list expectations for the level of achievement that students should attain as they progress through district schools. Note that the tables are built around two main parameters in addition to program type: 

1. The student’s initial ELPAC level and 
2. The number of years in US schools (US School entry date). 

The evaluation questions related to student outcomes are keyed to these tables and to expectations for achievement.

Roles and Responsibilities

Role	Responsibilities
Student	Attends school daily, arrives on time, and works for high achievement Participates in school activities Communicates regularly with parents, teachers, and support staff
Parent	Monitors and promotes English learner’s progress in academics, homework, attendance, behavior Supports English learner in activities to promote student achievement Communicates regularly regarding student progress with student, teachers and school Attends parent conferences and school functions (e.g., Open House, conferences) Participates in school committees—ELAC, Site Council, etc.
Classroom Teacher	Implements specific English learner’s programs as described in the Master Plan for English learners and provides instruction that aligns with state frameworks and district/state standards  Ensures delivery of appropriate Designated and Integrated ELD instruction Ensures access to core curriculum through specific, intentional, and explicit use of SDAIE strategies  Monitors English learners’ progress, reviews school/classroom data, uses data to modify instruction, reviews content and California Common Core ELD standards and assessment procedures Determines/implements differentiated strategies for English learners and R-FEP students  Advocates for support services for students not meeting standards and benchmarks who may be at risk of retention or who require interventions in order to reach goals  Attends team meetings and informs parents of progress and strategies to support students in meeting standards  Uses data to understand and respond to student needs
Principal	Monitors all aspects of staffing for and instruction in English learner programs, according to the Master Plan  Analyzes and uses data to facilitate student progress monitoring Is responsible for all procedures and legal requirements pertaining to English learners at the school Monitors placement of English learners and oversees reclassification process Reports periodically to district administrators on implementation of English learner programs and services  Provides leadership in all aspects of the educational program
District English Learner Advisory Committee (DELAC)	Examines program evaluation findings on an annual basis and frames recommendations for program improvement for the following year as part of its advisory role Reviews the Annual Language Census report Provides annual report to Board of Trustees  Advises on issues relevant to English learners in the district
English Learner Advisory Committee	Advises the principal and school staff on topics related to English learners (including advising in the development of the school plan) May review site data on program effectiveness and student achievement
Director of Curriculum & Instruction	Supports sites in implementing the Master Plan for English learners, monitors implementation of Master Plan, Evaluation Plan, and Monitoring Plan; reviews district and site English learner data Works with principals and Assistant Superintendent of Human Resources to ensure timely recruitment, hiring and training of teachers for EL assignments  Coordinates Instructional  Walkthroughs for program monitoring
Director of Special Programs	Monitors and supports implementation of theLCAP/LCAP Federal Addendum and Title III Plan  Monitors fiscal compliance to ensure alignment with the LCAP/LCAP Federal Addendum and Title III Plan and the Master Plan for English learners  Monitor school plans and budgets for services to English learners Monitor grants and evaluations for services to English learners  Develops work plan for, supervises, and works closely with Curriculum Specialist of Parent and Family Engagement Meets with principals to review site plans for services to English learners
Coordinator of Multilingual Language Instruction Programs	Provides leadership and strategic direction to the District’s multilingual language instruction programs. Provides supplemental support to our Language Acquisition programs which make instructional use of both English and a target language to ensure that all ELs attain English language proficiency (ELP), develop high levels of academic attainment in English, and meet the same challenging state academic standards as all other students
Curriculum Specialist-Emergent Bilingual Support	Supports Language Acquisition programs teachers to provide effective, evidence-based educational strategies that close the achievement gap and enable students to meet the state’s challenging academic standards; Supports Language Acquisition programs teachers to provide high quality instruction on the state-adopted academic content and ELD standards through Integrated and Designated ELD.
Curriculum Specialist, Parent and Family Engagement	Under the supervision of the Director of Special Programs, the Curriculum Specialist, Parent and Family Engagement: Supports sites in implementing parent and family engagement activities in accordance with the Master Plan for English learners Supports and provides training for parent leadership at site and district advisory meetings (DELAC, ELAC) Meets with principals to review site plans for parental involvement for parents of English learners Monitors Title III parental involvement policy compliance and  at the site and district levels
Director of DATA	Monitors implementation of Master Plan, Evaluation Plan, and Monitoring Plan; reviews and communicates district and site English learner data  Monitors compliance and English learner procedures at the site and district levels Works with other administrators to provide ongoing data support for school staff  Works closely with the Coordinator of EL Instruction. This includes the preparation of the English learner outcome reports.
Assistant Superintendent of Human Resources	Recruits and monitors placement of English learner staff in collaboration with principals and the Director of English learner Services  Monitors credentials of all personnel working with English learners Arranges/publicizes CLAD/BCLAD training and other needed staff development to ensure implementation of Master Plan for English learners in collaboration with the Coordinator of English Learner Services and Director of Curriculum and Instruction.
Assistant Superintendent of Educational Services	Supervises Director of Special Programs, Director of Curriculum and Instruction, and Director of DATA Analyzes district and school site data Provides overall support for English learner Programs  Meets with principals and administrators to review plans, program modifications, timelines for implementation, and support services for school sites Collaborates with district staff and parent groups on annual program evaluation  Shares results of program evaluation with stakeholders (Superintendent, Executive Board, and School Board)
Superintendent	Evaluates District’s goals relative to the LCAP and Title III Plans, including implementation of the Master Plan for English learners, student achievement, professional development, and evaluation and accountability  Convenes and chairs (or appointed designee) the English learners Services Council