home
 how we work
 recent
 projects
 reports
  evaluation and
  research reports
  policy related
  papers
 people
 contact us

Reports: Presenting What We Learn

Education Matters produces evaluation and research reports that are designed to reflect the complexity of the school reform work underway, highlight strengths and weaknesses of the work, and offer suggestions for next steps. The firm's policy reports aim to synthesize what we are learning across projects with the goal of informing others who may be in a position to develop new reform initiatives.

Evaluation/Research Reports

Whole-School Improvement/Standards-Based Reform in the Boston Public Schools. The following set of reports details the implementation and continuous development of the whole-school improvement agenda in the Boston Public Schools. The early reports and those directed to instructional coaching devote considerable attention to the development of strategies to make teaching a public practice in order to create collaborative, instructionally- focused school cultures.  These reports also address the role of the Boston Plan for Excellence, the local education fund that has served as a research and development partner of the district.  Later reports on the development of high school renewal focus on the district's organization for high school whole-school improvement and the role that partner organizations have played in this work.

In this section, we present first our reports on high school renewal in the Boston Public Schools. The first report, Baseline Report: High School Renewal in Boston (September 2004) describes the ways in which a sample of the district's comprehensive high schools are restructuring into smaller units — small thematic high schools or small thematic learning communities –  to provide students with greater opportunities to learn and succeed.  The second report, High School Renewal in the Boston Public Schools: Focus on Organization and Leadership (October 2005), focuses on the links between the district's organization and process for developing small high schools and small learning communities and on progress at the schools. The report includes findings from student focus group interviews concerning students' perceptions of their teachers, classes, and their small schools' and small learning communities' thematic foci.  Lastly, in the Final Evaluation Report (May 2007), we describe a small number of positive changes in light of the district’s goals; specifically, more attention to instruction at the schools and greater deputy knowledge of and involvement with the goals of high school renewal.  However, we also report that the challenges to the success of the initiative we identified in our first two reports remain, and that others have surfaced.  These challenges, we stress, must be addressed if high school renewal is to result in the desired and deserved outcomes for the district’s students. 

  • Baseline Report: High School Renewal in Boston (September 2004)
    Download (PDF format, ~230kb)
    By Barbara Neufeld and Anne Levy, with assistance from Sara Schwartz Chrismer. 

  • High School Renewal in the Boston Public Schools: Focus on Organization and Leadership (October 2005)
    Download (PDF format, ~350kb)
    By Barbara Neufeld, Anne Levy,     with assistance from Sara Schwartz Chrismer.

  • Final Evaluation Report: High School Renewal in the Boston Public Schools, 2003-2006 (May 2007)
    Download (PDF format, ~150kb)
    By Barbara Neufeld.

Second, we present our baseline report on the School Leadership Institute, The Boston School Leadership Institute: 2004-2005 Evaluation Report. This report describes and provides formative feedback on the three main programs of the Boston School Leadership Institute during the 2004-2005 school year: 1) Boston Principal Fellows Program, Boston's year-long, in-district principal preparation and certification program, 2) the New Principal Support System, which provides support to the district's first- and second-year principals, and 3) the Exploring the Principalship Program, a series of seminars designed to introduce school leadership in Boston to teachers, administrators, and others interested in learning about and potentially pursuing school leadership positions.

Third, we group together the four papers that focus on Collaborative Coaching and Learning in Boston. This approach to coaching began at the start of the 2001-2002 school year in a sample of the district's schools. By the middle of the school year, the district had decided that this new coaching model should be tried in all of the district's schools. In light of that decision, Education Matters wrote Using What We Know: Implications for Scaling-Up Implementation of the CCL Model (January 2002). At the end of the school year, we completed Off to a Good Start: Year I of Collaborative Coaching and Learning in the Effective Practice Schools (July 2002) which described the first year implementation of this coaching model. In July 2003, one year later, we reported on the second year of implementation in the same sample of schools in the report titled, Year II of Collaborative Coaching and Learning in the Effective Practice Schools: Expanding the Work. Finally, in light of the district's decision to engage all schools in collaborative coaching at the start of the 2003-2004 school year, we wrote a brief report in the experiences of a different sample of schools, schools that had not yet achieved "Effective Practice" status, as they attempted to implement this complex coaching model. That report is Year I of Collaborative Coaching and Learning in the Boston Public Schools: Accounts from the Schools.

Fourth, we include two papers that reflect the efforts of the district and the Boston Plan for Excellence to strengthen and deepen the literacy-focused work developing in the schools. The first report, Formative Assessment Pilot Implementation: Final Report (July 2004) considers the Boston Plan’s initial work in developing formative assessments that can provide teachers with usable data about students’ comprehension of reading materials. The second, Getting Our Feet Wet: Using Making Meaning for the First Time ((August 2004) explores the district’s pilot implementation of this comprehension focused reading program.

  • Formative Assessment Pilot Implementation: Final Report (July 2004)
    Download (PDF format, ~780kb)
    By Barbara Neufeld and Sara Schwartz. Report prepared for the Boston Plan for Excellence. This report evaluates the pilot phase of the implementation of a set of formative assessments that were designed to provide teachers with timely, targeted, usable data that could inform their reading instruction with respect to finding evidence and drawing inferences. The assessments were built in light of data gleaned from the state's MCAS test and were designed to be used by students in grades three, four, seven, and ten, the grades assessed by the state test: MCAS.
  • Getting Our Feet Wet: Using Making Meaning™ for the First Time (August 2004)
    Download (PDF format, ~230kb)
    By Barbara Neufeld and Annette Sassi. Report prepared for the Boston Public Schools and the Boston Plan for Excellence. This report evaluates the use of the Making Meaning™ curriculum materials produced by Developmental Studies Center in Oakland, CA. It is a small, qualitative study that concludes that these materials can be used to support the district's teachers in implementing Readers' Workshop as their approach to literacy instruction.

Remaining reports focus on the initial design and implementation of Boston's whole-school improvement effort.

  • Taking Stock: The Status of Implementation and the Need for Further Support in the BPE-BAC Cohort I and II Schools. (final version, May 2001).
    This report describes the theory and practices that undergird whole-school improvement in Boston Public Schools. It is designed to enable the Boston Plan for Excellence (BPE) - Boston Annenberg Challenge (BAC) to take stock of the status of implementation of the Essentials at the end of the four-year period of intensive funding for Cohort I schools. It is designed to assist the BPE-BAC in considering what kinds of supports to provide to a) Cohort I schools in light of more limited funding resources, and b) Cohort II schools as they enter their fourth and final year of intensive funding. And, finally, it is designed to help the BPS as it a) continues intensive implementation of the Plan for Whole-School improvement in Cohorts III and IV, and b) becomes more involved with Cohort I schools during the 2000-2001 schoolyear as a result of diminished BPE support for these schools.
    Download (PDF format, ~400kb)
  • The Boston Annenberg Challenge: Baseline Evaluation Report.
    This first evaluation report to the Boston Annenberg Challenge (BAC) provides baseline data on two components of the BAC's work with Cohort II schools (the second group of schools to begin whole-school improvement work). The components are (1) the early phase of implementing the literacy instructional focus, and (2) the work of the whole-school change coaches and the support that the BAC provides to them. In this report, we also provide a review of the Pilot Schools, a sub-group of ten schools within the Boston Public Schools (BPS) that were originally created in 1994 to be models of educational innovation and to serve as sites of promising research and development for effective urban public schools.
    Download (PDF format, ~390kb)
  • The Boston Plan for Excellence's 21st Century Schools Program: Mid-Year Evaluation Report, 1998-1999.
    In this second report, Education Matters follows-up on two components of the 21st Century Schools program that were considered integral to deep implementation of the reforms (see report dated July 1998). They are (1) the implementation of an instructional focus on literacy, and (2) the continuing work of the whole-school change and content coaches. These two components of the reform direct attention to the improvement of teaching and learning and to the development of schools as learning communities in which all adults work together to forward student achievement.
    Download (PDF format, ~270kb)
  • Evaluation Report on Year Two: The Boston Plan For Excellence's 21st Century Schools Program.
    This evaluation report describes the design of the first year of reform and analyzes three areas at the center of standards-based reform: 1) the implementation of standards; 2) looking at student work and other professional development opportunities for teachers and principals; and 3) the collaborative relationship between the BPE and the Boston Public Schools (BPS). While describing important progress in each area, the report cautions that particularly in the area of standards implementation - the heart of school reform in Boston - more work must be done to ensure that the reforms have their intended affect on teaching and learning and student achievement.
    Download (PDF format, ~280kb)

back to top

Standards-Based Middle School Reform In San Diego, CA; Long Beach, CA; Louisville, KY; and Corpus Christi, TX.

Standards-Based Middle School Reform in San Diego. Education Matter's work in San Diego began in 1993, however, in light of the extensive changes in the district that began in the 1998-1999 school year with the new administration of Alan Bersin, Superintendent and Anthony Alvarado, Chancellor of Instruction, the reports referenced here begin with that school year. The reports highlight the district's approach to improving literacy instruction through multiple approaches to professional development including the extensive use of school-based coaches.

  • Growing Instructional Capacity in Two San Diego Middle Schools: A Report prepared for The Edna McConnell Clark Foundation. (June, 2003)
    This report describes the efforts of two middle schools to develop in-school coaching capacity in the context of the district's shortage of highly trained literacy coaches.
    • Download (PDF format, ~310kb)

  • San Diego City Schools: Indicators of Coherence and "Planfulness" in Implementing Middle School, Standards-Based Reform. (March 2001)
    Written for the Edna McConnell Clark Foundation staff, this report summarizes the district's accomplishments through the middle of the 2000-2001 school year.
    Download (PDF format, ~80kb)

  • Implementing Standards-Based Reform in San Diego City Schools: Update Report, August 22, 2000.
    This report is a companion to the February 2000 report identified below. It describes middle school teachers' views of the Literacy Framework and the staff developer role near the end of the first year of implementation. While teachers are quite positive about the work in which they are engaged, they also raise concerns that the district needed to address.
    Download (PDF format, ~190kb)

  • Update Report: Implementing Standards-Based Reform in San Diego City Schools, February 2000.
    This report describes the first phase of the staff developer role in San Diego's middle schools. The report highlights the positive initial implementation of this role based on the perspectives of principals and staff developers who work at seventeen of the district's twenty-four middle schools.
    Download (PDF format, ~190kb)

  • Update Report: San Diego City Schools, August 1999.
    This report highlights three facets of San Diego's approach to implementing the first year of the Literacy Framework: a) the Instructional Leader role, b) three literacy strategies: Read-Alouds, Independent Reading, and Accountable Talk, and c) the district's literacy portfolios.
    Download (PDF format, ~210kb)

  • Update Report: Middle School Standards-Based Reform in San Diego. (April 1999) 
    This report reviews the initial direction that San Diego's new administration took with respect to implementing its Literacy Framework and the new district organization designed to support its implementation.
    Download     (PDF format, ~80kb)

back to top

Standards-Based Reform in Louisville (Jefferson County Public Schools). Standards-based reform in Louisville takes place in an extremely complex environment of teaching, learning, and assessment reforms launched by KERA (the Kentucky Education Reform Act of 1989) and its associated set of assessment practices initially called KIRIS (the Kentucky Instructional Results Information System). KERA and KIRIS together are a form of standards reform; they require schools to teach so that all children make progress away from the standards called "novice" and "apprentice" and toward those called "proficient" or "distinguished" within a designated period of time. Education Matters' reports on middle school, standards-based reform in Louisville consider the interaction of the district's reform efforts in the context of the state's high stakes accountability system.

back to top

Standards-Based Reform in Corpus Christi. Middle school, standards-based reform in Corpus Christi was significantly influenced by the Texas accountability system which included the Texas Assessment of Academic Skills (TAAS) now known as the Texas Assessment of Knowledge and Skill (TAKS). The basic skills orientation of the assessment as well as the high stakes attached to it led the district to have difficulty focusing on the kind of work implied by standards-based reform. Nonetheless, the district was successful in increasing the number of students from all demographic groups who achieved the minimal standards required by the state.

  • Standards-Based Reform in Corpus Christi: A Focus on the Early Implementation of Looking At Student Work, August 2000.
    During the 1999-2000 school year, with the district's agreement, Education Matters focused on the ways in which the district approached the implementation of Looking at Student Work (LASW) as a strategy for improving the quality of teachers' assignments and, thereby, students' work. This report details the variations in implementation as well as several common and often interrelated factors that led to those variations: 1) individual principal's knowledge of LASW and leadership of its implementation; 2) teachers' knowledge of the purpose of LASW; 3) clarity about the use of protocols and, 4) structured time in which to do LASW. The greater the presence of these factors, the greater the likelihood that LASW would be more deeply understood and effectively implemented at a school. Conversely, the greater their absence, the greater the likelihood that LASW would be shallowly understood and ineffectively implemented.
    Download (PDF format, ~150kb)

  • Update Report: Standards-Based Middle School Reform, August 1999.
    This report synthesizes what we learned from observing classrooms in our middle school sample during March 1999. Our focus during the observations was on the extent to which students had access to high level content and were being taught with strategies that enabled them to have a genuine opportunity to be actively engaged in their own learning. Across all four schools, our findings were similar: in our judgment, the range of pedagogical strategies in use was far too narrow to enable CCISD's middle school students to achieve at high standards, and the quality of work they were expected to produce was far too low.
    Download (PDF format, ~140kb)

  • Update Report: Standards-Based Middle School Reform, January 1999.
    In this report, we emphasize assessment and quality of student work because a) it is these two issues that we understood to be the center of the district's work during the 1998-1999 school year, and b) we think that a genuine focus on these issues will lead to improved student learning. We discuss the implications of focusing on these issues for professional development and consider the district's capacity to move forward with this work.
    Download (PDF format, ~150kb)

  • Update Report: Standards-Based Reform, August 1998.
    Building on previous reports, this update emphasizes the issues of quality and consistency in assessing student work in a standards-bases system. We present the concerns and issues that teachers raise as they try to assess the quality of student work. In addition, we consider the district's latest approaches to providing "safety nets" for students who are not making sufficient progress toward passing performance standards or who have failed them. And, we briefly review the focus of the Superintendent's Leadership Conference as an opportunity to enhance teachers' and principals' ideas about assessment.
    Download (PDF format, ~190kb)

  • Update Memo: Standards-Based Middle School Reform, August 1997.
    The purpose of this report is to describe changes in the individual middle schools that are in Education Matters' Corpus Christi sample. The report covers the "early implementation" of standards-based reform, from the middle of school year 1995-1996 to the end of school year 1996-1997.
    Download (PDF format, ~210kb)

  • Update Memo: Standards-Based Middle School Reform, February 1997.
    This update focuses on districtwide initiatives that seem most closely related to improving classroom teaching so that it leads to more rigorous instruction and improved student performance. To this end, the report describes how the district is a) developing and revising standards in academic and other curriculum areas; b) revising and implementing scoring guidelines with ongoing review and analysis; c) tackling the issue of grading guidelines; d) administering the first standards-based summer school; e) building on the successful piloting of Algebra for All; and, f) supporting professional development.
    Download (PDF format, ~110kb)

  • Standards-Based Reform: Baseline Report, Corpus Christi Independent School District.
    This report describe's CCISD's strong commitment to implementing standards in middle schools across the district. It concludes that building-based administrators and central office staff view standards as the center of reform and see themselves as accountable for advancing the standards agenda. The report also describes the challenges the district faces in trying to implement standards while increasing students' test scores on the statewide test, Texas Assessment of Academic Skills (TAAS). The report describes the ways in which teachers engaged in a balancing act: promoting innovative performance tasks while preparing students, through drill and practice, for TAAS.
    Download (PDF format, ~170kb)

Standards-Based Middle School Reform: Long Beach Unified (August 1999).

  • December, 1997: The December 1997 report describes the diffusion throughout the district of an increasing unity of vision and coherence of standards-based reform, particularly in the areas of professional development and student assessment.
    • Download (PDF format, ~110kb)

  • August, 1998: This report continues to focus on the developments discussed in the December 1997 update, the development of a comprehensive professional development system and classroom-based assessment, within in the context of increasing state accountability pressures. Long Beach responded by taking major steps toward ending social promotion..
    Download (PDF format, ~210kb)

  • February, 1999: Data collection for this report focused on schools and teachers' and administrators' perspective of district's rapid pace of reform. We begin by first enumerating a number of new district developments and then discuss the implementation of those initiatives as experienced at the school sites.
    Download (PDF format, ~150kb)

  • 1998-1999: Data collection for this report focused on schools and teachers' and administrators' perspective of district's rapid pace of reform. We begin by first enumerating a number of new district developments and then discuss the implementation of those initiatives as experienced at the school sites.
    Download (PDF format, ~170kb)

  • Update Memo: Standards-based Middle School Reform: Long Beach Unified (February 2000)
    This report focuses on a major collaborative effort between Long Beach Unified, California State University, and Long Beach City College called, "Seamless Education," designed to improve teacher preparation K-16.
    Download (PDF format, ~110kb)

  • Implementing Standards-Based Middle School Reform in Long Beach Unified School District. (August 2000)
    This annual evaluation report focuses on quality professional development as a key to the effective implementation of reform efforts. The success of interventions in the district corresponded directly to the strength of the professional development to support each initiative.
    Download (PDF format, ~260kb)

  • Update Memo: Standards-based Middle School Reform in Long Beach Unified (February 2001)
    Written for the Edna McConnell Clark Foundation, this report focuses on the district's progress on four initiatives identified by the district as their priorities in raising student achievement.
    Download (PDF format, ~130kb)

  • Six Years of Standards-based Middle School Reform in Long Beach Unified Schools (December 2001)
    This six-year review chronicles the major developments of six years, highlighting the successes and the challenges encountered along the way.
    Download (PDF format, ~170kb)

  • Standards-based Reform in Long Beach Unified: Progress in Reading Development: Final Evaluation Report (December 2002)
    This report summarizes the progress of the Reading Development program in Long Beach from 1997 through 2001. The progress seen underscores the importance of having highly trained reading teachers throughout the district.
    Download (PDF format, ~260kb)

back to top

Policy Related Papers

Coaching: A Strategy for Developing Instructional Capacity
Download (PDF format, ~730kb)

Coaching is an increasingly popular strategy for districts seeking large-scale improvement in instruction. In this paper, Coaching: A Strategy for Developing Instructional Capacity, Barbara Neufeld and Dana Roper describe what coaching is, what coaches do, the kinds of supports that coaches need, and the potential benefits to both educators and students. The paper is co-published by the Annenberg Institute and the Aspen Institute Program on Education. A print copy of this report can be ordered via the web site of the Aspen Institute.

Transforming Events: A Local Education Fund's Efforts to Promote Large-Scale Urban School Reform

This paper was co-authored by Barbara Neufeld and Ellen Guiney, Executive Director of the Boston Plan for Excellence, the local education fund that has taken a leading role in supporting whole-school improvement in the Boston Public Schools. The paper describes the ways in which this local education fund has been working with the Boston Public Schools. It was originally presented at the Annual Meeting of the American Educational Research Association in New Orleans in April 2002 and revised for publication in June 2002. [This report is available for PDF Download on the web site of the Annenberg Challenge. The article by Neufeld and Guiney begins on page 51.]

Transforming Abbott Schools in New Jersey, Guidance from the Field. (November 1997)
Download (PDF format, ~260kb)

This paper was prepared for the Education Law Center, Newark, NJ, as part of a set of papers written to inform discussions about how to improve urban schools in New Jersey in light of the state Supreme Court's May 14, 1997 order that the state of New Jersey "immediately establish parity in regular education spending between each special needs district and average spending in wealthy suburban districts." The papers were compiled into a volume titled Transforming Teaching and Learning in Special Needs Districts which was background reading for a meeting designed to address "next steps" in this process which was held at the Education Testing Service's Chauncey Center in Princeton, NJ during the weekend of December 12-14, 1997.

back to top