home
how we work
recent
projects
reports
evaluation and
research reports
policy related
papers
people
contact us
Reports: Presenting What We Learn
Education Matters produces evaluation and research reports that are designed to reflect the complexity of the school reform work underway, highlight strengths and weaknesses of the work, and offer suggestions for next steps. The firm's policy reports aim to synthesize what we are learning across projects with the goal of informing others who may be in a position to develop new reform initiatives.
Evaluation/Research Reports
Whole-School Improvement/Standards-Based Reform in the Boston Public Schools. The following set of reports details the implementation and continuous development of the whole-school improvement agenda in the Boston Public Schools. They devote considerable attention to the development of strategies to make teaching a public practice in order to create collaborative, instructionally- focused school cultures, and to the role that principals and coaches play in this important work. And, they address the role of the Boston Plan for Excellence, the local education fund that has served as a research and development partner of the district.
The four papers that focus on Collaborative Coaching and Learning in Boston form a set. This approach to coaching began at the start of the 2001-2002 school year in a sample of the district’s schools. By the middle of the school year, the district had decided that this new coaching model should be tried in all of the district’s schools. In light of that decision, Education Matters wrote Using What We Know: Implications for Scaling-Up Implementation of the CCL Model (January 2002). At the end of the school year, we completed Off to a Good Start: Year I of Collaborative Coaching and Learning in the Effective Practice Schools (July 2002) which described the first year implementation of this coaching model. In July 2003, one year later, we reported on the second year of implementation in the same sample of schools in the report titled, Year II of Collaborative Coaching and Learning in the Effective Practice Schools: Expanding the Work. Finally, in light of the district’s decision to engage all schools in collaborative coaching at the start of the 2003-2004 school year, we wrote a brief report in the experiences of a different sample of schools, schools that had not yet achieved “Effective Practice” status, as they attempted to implement this complex coaching model. That report, our most recent, is Year I of Collaborative Coaching and Learning in the Boston Public Schools: Accounts from the Schools.
- Formative Assessment Pilot Implementation: Final Report (July 2004)
Download (PDF format, ~780kb)By Barbara Neufeld and Sara Schwartz. Report prepared for the Boston Plan for Excellence. This report evaluates the pilot phase of the implementation of a set of formative assessments that were designed to provide teachers with timely, targeted, usable data that could inform their reading instruction with respect to finding evidence and drawing inferences. The assessments were built in light of data gleaned from the state's MCAS test and were designed to be used by students in grades three, four, seven, and ten, the grades assessed by the state test: MCAS.
- Getting Our Feet Wet: Using Making Meaning™ for the First Time (August 2004)
Download (PDF format, ~230kb)By Barbara Neufeld and Annette Sassi. Report prepared for the Boston Public Schools and the Boston Plan for Excellence. This report evaluates the use of the Making Meaning™ curriculum materials produced by Developmental Studies Center in Oakland, CA. It is a small, qualitative study that concludes that these materials can be used to support the district's teachers in implementing Readers' Workshop as their approach to literacy instruction.
- Year 1 of Collaborative Coaching and Learning in the Boston Public Schools: Accounts from the Schools. (July 2003)
This report focuses on the ways in which a small sample of schools that varied in the extent to which they had implemented Boston's whole-school improvement agenda went about implementing the district's Collaborative Coaching and Learning model. It highlights the challenges schools face in implementing this model when other supporting components of reform are not yet in place.
Download (PDF format, ~170kb)- Year II of Collaborative Coaching and Learning in the Effective Practice Schools: Expanding the Work. (July 2003)
This report describes the progress of the second year of this group-oriented coaching model in the same sample of Boston Public Schools described in our Year I report, Off to a Good Start (see below). The report details the professional development that coaches and teachers do collaboratively to improve the implementation of Readers' and Writers' Workshop. It highlights the progress as well as the challenges that accompany the coaching model and the literacy strategies.
Download (PDF format, ~280kb)- Off to a Good Start: Year 1 of Collaborative Coaching and Learning in the Effective Practice Schools. (July 2002)
This report describes the successful first year of a new, group-oriented coaching model in a sample of Boston Public Schools that had achieved high levels of implementation with respect to the district's "Essentials" of Whole-School Improvement. The coaching model involves teachers working together in small groups with a coach over a six-week period.
Download (PDF format, ~4mb)- Using What We Know: Implications for Scaling Up Implementations of the CCL Model. (January 2002)
In light of the formation of the "Coach Working Group" (CWG), a collaboration of the Boston Public Schools (BPS) and the Boston Plan for Excellence (BPE) designed to consider the possible scale-up of the Collaborative Coaching and Learning (CCL) model to a range of the district's schools - not only to those that have achieved Effective Practice status - Education Matters wrote this report to provide the BPS and BPE with a) a summary of what we learned from content/literacy, whole-school change, and math coaches in the 2000-2001 school year, and b) our judgment of how those findings might inform the development of the CCL model in many more BPS schools.
Download (PDF format, ~150kb)- Taking Stock: The Status of Implementation and the Need for Further Support in the BPE-BAC Cohort I and II Schools. (final version, May 2001).
This report describes the theory and practices that undergird whole-school improvement in Boston Public Schools. It is designed to enable the Boston Plan for Excellence (BPE) - Boston Annenberg Challenge (BAC) to take stock of the status of implementation of the Essentials at the end of the four-year period of intensive funding for Cohort I schools. It is designed to assist the BPE-BAC in considering what kinds of supports to provide to a) Cohort I schools in light of more limited funding resources, and b) Cohort II schools as they enter their fourth and final year of intensive funding. And, finally, it is designed to help the BPS as it a) continues intensive implementation of the Plan for Whole-School improvement in Cohorts III and IV, and b) becomes more involved with Cohort I schools during the 2000-2001 schoolyear as a result of diminished BPE support for these schools.
Download (PDF format, ~400kb)- The Boston Annenberg Challenge: Baseline Evaluation Report.
This first evaluation report to the Boston Annenberg Challenge (BAC) provides baseline data on two components of the BAC's work with Cohort II schools (the second group of schools to begin whole-school improvement work). The components are (1) the early phase of implementing the literacy instructional focus, and (2) the work of the whole-school change coaches and the support that the BAC provides to them. In this report, we also provide a review of the Pilot Schools, a sub-group of ten schools within the Boston Public Schools (BPS) that were originally created in 1994 to be models of educational innovation and to serve as sites of promising research and development for effective urban public schools.
Download (PDF format, ~390kb)- The Boston Plan for Excellence's 21st Century Schools Program: Mid-Year Evaluation Report, 1998-1999.
In this second report, Education Matters follows-up on two components of the 21st Century Schools program that were considered integral to deep implementation of the reforms (see report dated July 1998). They are (1) the implementation of an instructional focus on literacy, and (2) the continuing work of the whole-school change and content coaches. These two components of the reform direct attention to the improvement of teaching and learning and to the development of schools as learning communities in which all adults work together to forward student achievement.
Download (PDF format, ~270kb)- Evaluation Report on Year Two: The Boston Plan For Excellence's 21st Century Schools Program.
This evaluation report describes the design of the first year of reform and analyzes three areas at the center of standards-based reform: 1) the implementation of standards; 2) looking at student work and other professional development opportunities for teachers and principals; and 3) the collaborative relationship between the BPE and the Boston Public Schools (BPS). While describing important progress in each area, the report cautions that particularly in the area of standards implementation - the heart of school reform in Boston - more work must be done to ensure that the reforms have their intended affect on teaching and learning and student achievement.
Download (PDF format, ~280kb) - Year II of Collaborative Coaching and Learning in the Effective Practice Schools: Expanding the Work. (July 2003)
Standards-Based Middle School Reform In San Diego, CA; Long Beach, CA; Louisville, KY; and Corpus Christi, TX.
Standards-Based Middle School Reform in San Diego. Education Matter's work in San Diego began in 1993, however, in light of the extensive changes in the district that began in the 1998-1999 school year with the new administration of Alan Bersin, Superintendent and Anthony Alvarado, Chancellor of Instruction, the reports referenced here begin with that school year. The reports highlight the district's approach to improving literacy instruction through multiple approaches to professional development including the extensive use of school-based coaches.
- Growing Instructional Capacity in Two San Diego Middle Schools: A Report prepared for The Edna McConnell Clark Foundation. (June, 2003)
This report describes the efforts of two middle schools to develop in-school coaching capacity in the context of the district's shortage of highly trained literacy coaches.
Download (PDF format, ~310kb) - San Diego City Schools: Indicators of Coherence and "Planfulness" in Implementing Middle School, Standards-Based Reform. (March 2001)
Written for the Edna McConnell Clark Foundation staff, this report summarizes the district's accomplishments through the middle of the 2000-2001 school year.
Download (PDF format, ~80kb) - Implementing Standards-Based Reform in San Diego City Schools: Update Report, August 22, 2000.
This report is a companion to the February 2000 report identified below. It describes middle school teachers' views of the Literacy Framework and the staff developer role near the end of the first year of implementation. While teachers are quite positive about the work in which they are engaged, they also raise concerns that the district needed to address.
Download (PDF format, ~190kb) - Update Report: Implementing Standards-Based Reform in San Diego City Schools, February 2000.
This report describes the first phase of the staff developer role in San Diego's middle schools. The report highlights the positive initial implementation of this role based on the perspectives of principals and staff developers who work at seventeen of the district's twenty-four middle schools.
Download (PDF format, ~190kb) - Update Report: San Diego City Schools, August 1999.
This report highlights three facets of San Diego's approach to implementing the first year of the Literacy Framework: a) the Instructional Leader role, b) three literacy strategies: Read-Alouds, Independent Reading, and Accountable Talk, and c) the district's literacy portfolios.
Download (PDF format, ~210kb) - Update Report: Middle School Standards-Based Reform in San Diego. (April 1999)
This report reviews the initial direction that San Diego's new administration took with respect to implementing its Literacy Framework and the new district organization designed to support its implementation.
Download (PDF format, ~80kb)
Standards-Based Reform in Louisville (Jefferson County Public Schools). Standards-based reform in Louisville takes place in an extremely complex environment of teaching, learning, and assessment reforms launched by KERA (the Kentucky Education Reform Act of 1989) and its associated set of assessment practices initially called KIRIS (the Kentucky Instructional Results Information System). KERA and KIRIS together are a form of standards reform; they require schools to teach so that all children make progress away from the standards called "novice" and "apprentice" and toward those called "proficient" or "distinguished" within a designated period of time. Education Matters' reports on middle school, standards-based reform in Louisville consider the interaction of the district's reform efforts in the context of the state's high stakes accountability system.
Standards-Based Reform in Corpus Christi. Middle school, standards-based reform in Corpus Christi was significantly influenced by the Texas accountability system which included the Texas Assessment of Academic Skills (TAAS) now known as the Texas Assessment of Knowledge and Skill (TAKS). The basic skills orientation of the assessment as well as the high stakes attached to it led the district to have difficulty focusing on the kind of work implied by standards-based reform. Nonetheless, the district was successful in increasing the number of students from all demographic groups who achieved the minimal standards required by the state.
Standards-Based Middle School Reform: Long Beach Unified (August 1999).
Policy Related Papers Coaching: A Strategy for Developing Instructional Capacity
Coaching is an increasingly popular strategy for districts seeking large-scale improvement in instruction. In this paper, Coaching: A Strategy for Developing Instructional Capacity, Barbara Neufeld and Dana Roper describe what coaching is, what coaches do, the kinds of supports that coaches need, and the potential benefits to both educators and students. The paper is co-published by the Annenberg Institute and the Aspen Institute Program on Education. A print copy of this report can be ordered via the web site of the Annenberg Institute.
Transforming Events: A Local Education Fund's Efforts to Promote Large-Scale Urban School Reform
This paper was co-authored by Barbara Neufeld and Ellen Guiney, Executive Director of the Boston Plan for Excellence, the local education fund that has taken a leading role in supporting whole-school improvement in the Boston Public Schools. The paper describes the ways in which this local education fund has been working with the Boston Public Schools. It was originally presented at the Annual Meeting of the American Educational Research Association in New Orleans in April 2002 and revised for publication in June 2002. [This report is available for PDF Download on the web site of the Annenberg Challenge. The article by Neufeld and Guiney begins on page 51.]
Transforming Abbott Schools in New Jersey, Guidance from the Field. (November 1997)
This paper was prepared for the Education Law Center, Newark, NJ, as part of a set of papers written to inform discussions about how to improve urban schools in New Jersey in light of the state Supreme Court's May 14, 1997 order that the state of New Jersey "immediately establish parity in regular education spending between each special needs district and average spending in wealthy suburban districts." The papers were compiled into a volume titled Transforming Teaching and Learning in Special Needs Districts which was background reading for a meeting designed to address "next steps" in this process which was held at the Education Testing Service's Chauncey Center in Princeton, NJ during the weekend of December 12-14, 1997.
In this report, we highlight teachers', principals, and school-based professional developers' understanding of performance standards to make the point that the district ought not move forward with performance tasks without 1) clarifying the links between the components of the reform and 2) agreeing on common standards of proficient work across grade levels and content areas. In this report, we also provide descriptions of a range of instructional strategies that teachers use in their classrooms. We provide these descriptions to identify the issues with which teachers are struggling and the ways in which they are trying to address them. This report was written to provide an overview of progress and food for thought about what needs to happen next to achieve the district and the schools' goals.
Download (PDF format, ~180kb)
In this report, we describe the district's progress with a) completion of its performance standards, b) plans for end-of-year professional development to engage principals and teachers in using those standards to improve instruction, c) continuing work to enhance principals' knowledge of instruction and their role as instructional leaders, and d) initial positive reactions to the District Dialogue Process.
Download (PDF format, ~130kb)
In this update report, we highlight components of the district's strategy for achieving its vision and explain why we think they are indicators of genuine progress with standards-based school reform. These components include, 1) focused principal professional development, 2) the design and implementation of the Cadre Teacher position, 3) implementation of district level dialogue teams, and, 4) development of performance standards and a comprehensive local assessment system.
Download (PDF format, ~100kb)
This memo emphasizes aspects of on-going and proposed work at the district level that can contribute to standards-based reform. In particular, we highlight aspects district level work that can contribute to standards-based reform.
Download (PDF format, ~100kb)
In this report, we look at the similarities between what the district and the state require with respect to standards-based reform and the ways in which the district is struggling to reconcile what it sees at competing demands.
Download (PDF format, ~170kb)
During the 1999-2000 school year, with the district's agreement, Education Matters focused on the ways in which the district approached the implementation of Looking at Student Work (LASW) as a strategy for improving the quality of teachers' assignments and, thereby, students' work. This report details the variations in implementation as well as several common and often interrelated factors that led to those variations: 1) individual principal's knowledge of LASW and leadership of its implementation; 2) teachers' knowledge of the purpose of LASW; 3) clarity about the use of protocols and, 4) structured time in which to do LASW. The greater the presence of these factors, the greater the likelihood that LASW would be more deeply understood and effectively implemented at a school. Conversely, the greater their absence, the greater the likelihood that LASW would be shallowly understood and ineffectively implemented.
Download (PDF format, ~150kb)
This report synthesizes what we learned from observing classrooms in our middle school sample during March 1999. Our focus during the observations was on the extent to which students had access to high level content and were being taught with strategies that enabled them to have a genuine opportunity to be actively engaged in their own learning. Across all four schools, our findings were similar: in our judgment, the range of pedagogical strategies in use was far too narrow to enable CCISD's middle school students to achieve at high standards, and the quality of work they were expected to produce was far too low.
Download (PDF format, ~140kb)
In this report, we emphasize assessment and quality of student work because a) it is these two issues that we understood to be the center of the district's work during the 1998-1999 school year, and b) we think that a genuine focus on these issues will lead to improved student learning. We discuss the implications of focusing on these issues for professional development and consider the district's capacity to move forward with this work.
Download (PDF format, ~150kb)
Building on previous reports, this update emphasizes the issues of quality and consistency in assessing student work in a standards-bases system. We present the concerns and issues that teachers raise as they try to assess the quality of student work. In addition, we consider the district's latest approaches to providing "safety nets" for students who are not making sufficient progress toward passing performance standards or who have failed them. And, we briefly review the focus of the Superintendent's Leadership Conference as an opportunity to enhance teachers' and principals' ideas about assessment.
Download (PDF format, ~190kb)
The purpose of this report is to describe changes in the individual middle schools that are in Education Matters' Corpus Christi sample. The report covers the "early implementation" of standards-based reform, from the middle of school year 1995-1996 to the end of school year 1996-1997.
Download (PDF format, ~210kb)
This update focuses on districtwide initiatives that seem most closely related to improving classroom teaching so that it leads to more rigorous instruction and improved student performance. To this end, the report describes how the district is a) developing and revising standards in academic and other curriculum areas; b) revising and implementing scoring guidelines with ongoing review and analysis; c) tackling the issue of grading guidelines; d) administering the first standards-based summer school; e) building on the successful piloting of Algebra for All; and, f) supporting professional development.
Download (PDF format, ~110kb)
This report describe's CCISD's strong commitment to implementing standards in middle schools across the district. It concludes that building-based administrators and central office staff view standards as the center of reform and see themselves as accountable for advancing the standards agenda. The report also describes the challenges the district faces in trying to implement standards while increasing students' test scores on the statewide test, Texas Assessment of Academic Skills (TAAS). The report describes the ways in which teachers engaged in a balancing act: promoting innovative performance tasks while preparing students, through drill and practice, for TAAS.
Download (PDF format, ~170kb)
Download (PDF format, ~110kb)
Download (PDF format, ~210kb)
Download (PDF format, ~150kb)
Download (PDF format, ~170kb)
This report focuses on a major collaborative effort between Long Beach Unified, California State University, and Long Beach City College called, "Seamless Education," designed to improve teacher preparation K-16.
Download (PDF format, ~110kb)
This annual evaluation report focuses on quality professional development as a key to the effective implementation of reform efforts. The success of interventions in the district corresponded directly to the strength of the professional development to support each initiative.
Download (PDF format, ~260kb)
Written for the Edna McConnell Clark Foundation, this report focuses on the district’s progress on four initiatives identified by the district as their priorities in raising student achievement.
Download (PDF format, ~130kb)
This six-year review chronicles the major developments of six years, highlighting the successes and the challenges encountered along the way.
Download (PDF format, ~170kb)
This report summarizes the progress of the Reading Development program in Long Beach from 1997 through 2001. The progress seen underscores the importance of having highly trained reading teachers throughout the district.
Download (PDF format, ~260kb)
Download (PDF format, ~730kb)
Download (PDF format, ~260kb)