Annual Report Overviews

In August of 1997, the Austin Independent School District (AISD) launched a Local Systemic Change initiative to implement national standards for mathematics curriculum and instruction in all K-8 classrooms. This district initiative, the Austin Collaboration for Mathematics Education (ACME), is funded in part by the National Science Foundation (NSF), and is a partnership with the Charles A. Dana Center and the University of Texas at Austin. ACME is a unique approach in that it provides systematic and continuing professional development for every elementary and middle school mathematics teachers in a large urban district.

Through ACME, each K-8 mathematics teacher in the district participates in a minimum of 120 hours of formal professional development including summer institutes, follow-up training during the academic year, and campus level support such as peer coaching, collegial sharing, and modeling. The intent of professional development is (1) to build sound practice in mathematics instruction, (2) to ensure consistent implementation of a quality mathematics curriculum, and (3) to provide ongoing support for teachers and administrators to implement standards-based curriculum and instruction in every classroom in the district.

The initiative takes place in a growing urban district with over 75,000 students, approximately 55,000 of whom are of elementary and middle school age. The project currently works directly with 68 elementary schools and 15 middle schools. The size of schools is diverse, ranging from 500 to 1700 students for middle schools and 275 to 1300 students for elementary schools. The district serves a racially and ethnically diverse student population. Forty-three percent of the students are Hispanic, 37% are Anglo, 18% are African American, 2% are Asian or Pacific Islander and less than 1% are Native American. Among the students, 14% have limited English proficiency and 50% qualify for free or reduced lunches through federal funds.

The major goal of the project is to raise the mathematics achievement of all students in the district by implementing a rigorous, standards-based curriculum for every child. The Texas State Accountability System has revealed inequities in students’ mathematics achievement through their performance on the Texas Assessment of Academic Skills (TAAS). Although in recent years the percentage of students who pass the mathematics segment of the TAAS test has increased, students who are African American, Hispanic, or economically disadvantaged consistently perform below the average for all students (Williams, 1998). District administrators recognize the need to provide students with the mathematics understanding, reasoning, and communication skills that move beyond mathematics computation.

To address these needs, the initiative assists elementary and middle school staff in implementing the district curriculum that is based on standards designated by the National Council of Teachers of Mathematics (NCTM, 1989, 1991, 1995.) District staff are purchasing the curriculum resources of Investigations in Number, Data, and Space for elementary grades and Connected Mathematics for middle grades to support teachers’ implementation of standards-based instruction. These curriculum resources are particularly well suited for AISD because they support the following teaching practices (cf. Russell, 1998):

• Promoting children’s mathematical thinking, reasoning, and problem-solving skills;
• Developing children’s deep understanding of mathematical concepts through hands-on experiences, real-world problems, and communication; and
• Supporting a vertically and horizontally coordinated curriculum that considers the needs of all students, including those who are special education, gifted and talented, limited English proficient, and bilingual.

These investigative practices emphasize mathematical literacy through the understanding of mathematics concepts and approach instruction through problem-solving and communication of mathematical ideas. These practices contrast with traditional practices that emphasize mathematical algorithms, rote memorization, and mastery of computation (Cohen & Ball, 1990).

To support improvement in students’ mathematical competence in every classroom, district administrators also recognize the need to build a community of learners that fosters teachers’ professional growth as mathematics instructors. The ACME project provides long-term professional development designed to facilitate teachers’ exploration and acquisition of mathematics content, the philosophy behind standards-based pedagogy, and classroom management for investigative, inquiry-based mathematics instruction.

Through the ACME project, each of the district’s K-8 mathematics teacher, participates in a series of professional development activities over three years. The more than 2000 participants include general education, special education, bilingual, and English as a Second Language (ESL) teachers. The first two years include week-long summer institutes and five follow-up days during each academic year. The third year involves less formal professional development to continue through campus initiatives that the project will support. Teachers are paid a stipend to attend the summer institutes and follow-up sessions outside school hours, and substitutes are provided to release teachers during the academic year.

The implementation plan begins at the transition between elementary and middle school. Thus, in the summer of 1997, staff development began with fifth and sixth grade teachers, followed by fourth and seventh grade teachers in the summer of 1998, and will continue with kindergarten through third grade and eighth grade teachers in subsequent years. In addition to grade-by-grade implementation, eight elementary schools and three middle schools are serving as pilot sites with all grade levels participating in the initiative simultaneously beginning in year one of the project.

A second goal of the initiative is to establish systemic change by developing school cultures in which communities continually improve mathematics teaching and learning. In the summer of 1998, ACME provided an institute for principals and their assistants that addressed administrative issues such as strategies for supporting teachers in implementation of standards based curriculum and instruction. The project also makes use of organizational structures already present (e.g., curriculum specialists) to support teachers on campuses. Through these campus leaders, the project disseminates information to teachers on every campus. The ACME project staff is also developing customized professional development for lead teachers so that they may facilitate professional development sessions and support their peers on the campus level in a variety of ways, including peer coaching, demonstration teaching, information sources, and sharing classroom management techniques. To garner parent participation in the mathematics curriculum, the project staff provides schools with technical support (e.g., pamphlets and videos in English and Spanish) as well as assistance with organizing parent education and involvement (e.g., parent math nights).

Additionally, the project staff enlists support from the district’s administrative leaders. Central and campus administrators have undergone numerous changes over the past five years, including the resignation of the superintendent and appointment of an interim superintendent in April of 1998. Throughout these changes, support for the ACME project has continued.

A third and final goal of the project is to ensure that all of the district’s resources and efforts in mathematics education are aligned with local, state, and national standards. ACME project staff assert that the district’s mathematics curriculum document, the state standards in the Texas Essential Knowledge and Skills (TEKS), and the NCTM (1989, 1991, 1995) standards are aligned. To ensure that these standards emerge in the district’s mathematics classrooms, the ACME project channels local, state, and federal funds to teachers’ professional development and to support the district’s textbook adoption of the curriculum resources.

Purpose of Evaluation

NSF requires an evaluation of ACME in collaboration with Horizon Research, Inc. (HRI), an educational research group contracted to design the national evaluation of mathematics and science systemic change initiatives. This evaluation is formative, and provides information about project implementation so that decision makers can make improvements. The purpose of this report is to establish a baseline for what mathematics education and instruction looks like in the district and the current supports for implementation. The report presents attitudes of teachers and principals toward mathematics reforms, documents mathematics classroom instruction, examines the quality of teachers’ professional development, and assesses the extent of support for the project districtwide.

To establish a clear picture of mathematics education and instruction throughout the district in the ACME project’s baseline year, data were gathered from multiple sources and in various formats that are described next.

The local evaluator prepared a list of all elementary and middle school mathematics teachers who were employed by the district as of February of 1998. A previous survey revealed that some of the district’s elementary schools departmentalize grade levels such that some educators teach language arts and/or social studies but not mathematics (Batchelder, 1998). Therefore, the evaluator phoned campus administrators to screen the list for mathematics teachers only. From this list, HRI selected a random sample of 300 mathematics teachers, and the evaluator mailed questionnaires to these teachers. These questionnaires, designed by HRI, survey teachers’ attitudes toward national standards for mathematics instruction, preparation to implement the standards, classroom practice, mathematics content knowledge, perceptions of district support, demographic characteristics, and experiences in ACME professional development. (See Appendices for a sample teacher questionnaire.)

A few weeks after the first mailing of questionnaires in April of 1998, 50% of the teachers had returned the forms. A second mailing was sent out in May to teachers who had not yet returned a completed questionnaire. Additionally, teachers who had returned the questionnaire but skipped some items were sent photocopies of the pages with missing information and a request to return the completed copies. At the beginning of May, principals were given a list of teachers at their school who had not yet returned the forms and were told to ensure that the forms were returned. Out of the original 300 mathematics teachers in the sample, 11 were omitted for various reasons such as not teaching mathematics, parenting or maternity leave, resignation, and retirement. A total of 248 mathematics teachers out of the 289 valid respondents returned completed questionnaires yielding a response rate of 86%.

In addition to these mathematics teachers, a sample of 50 special education teachers was sent questionnaires. Four teachers were omitted because they did not teach mathematics, had resigned, or were not presently teaching special education. Thirty-nine special education teachers out of the 46 valid respondents returned questionnaires with a response rate of 85%.

Additional sources of information from teachers included informal interviews and observations during professional development. The ACME project staff also designed questionnaires for teachers to evaluate the professional development. Teachers’ responses to these evaluations provided information about enhancement of their knowledge of mathematics content, strengths of the sessions, concerns about implementing the standards-based curriculum and instruction, and suggestions for improving the project.

In April of 1998, a survey was sent to the principals of all of the 15 middle schools and the 66 elementary schools, as well as to the principal of the Alternative Learning Center. The principal questionnaires assessed the extent of school reforms, attitudes toward standards-based curricula in mathematics and science, district and state support for the ACME project, familiarity with the project, and school demographics. (See Appendices for a sample principal questionnaire.) Data collection procedures for the principals who did not complete their questionnaires paralleled those used for teachers. In mid-May, the evaluator called principals who had not yet returned their forms. All 82 principals returned the questionnaires for a 100% response rate.

From the sample of 300 elementary and middle school mathematics teachers, HRI randomly selected 10 teachers, none of whom had participated in ACME professional development, for classroom observation. One back-up teacher was observed because one teacher in the original sample had been pulled from the classroom to teach a literacy project full-time. All of the observed classrooms were kindergarten through fourth grade classes, except one, which was a seventh grade class. The lead evaluator also drew a random sample of four special education teachers to observe. One of these teachers was on leave and another declined to be observed because she was going on maternity leave. A back-up special education teacher was randomly selected so that a total of three special education classrooms could be observed. For all of the observations except two in which there were scheduling conflicts, an entire mathematics lesson was observed.

In March of 1998, the lead evaluator and ACME project manager were both trained and certified to rate classroom observations reliably using HRI’s classroom observation protocol (HRI, 1998; see Appendices for a sample protocol and pre- and post-classroom observation interviews). The lead evaluator rated all of the classroom observations except one, which the project manager rated. The protocol is used to rate a classroom observation on an ordinal scale, which consists of five global categories that are qualitatively different and describe the extent to which a teacher’s mathematics instruction is effective and meets national standards.

The purpose of the classroom observations in the baseline year is to capture a snapshot of mathematics instruction in the district before teachers participate in ACME professional development. It is important to note that the small sample of 10 observations is not representative of all mathematics instruction in the district; the frequencies and proportions of the categories do not generalize to the district as a whole. Rather, these observations generally depict some of the different types of instruction in the district as the project launches, not the relative frequency of the types.

The lead evaluator observed 16 professional development sessions to assess quality and content and formally rated 8 of the sessions with HRI’s professional development observation protocol. (See Appendices for a sample protocol.) This protocol is similar to the one for classroom observation protocol in that it results in global ratings on a 5-point ordinal scale ranging from ineffective to exemplary facilitation but for adult learners. The eight sessions observed represented three different formats of the project’s professional development. One session was for school administrators and provided logistical and conceptual information about the new mathematics curriculum and instruction, suggestions for answering parent concerns, support for teachers, and so on. Another session was a follow-up during the academic year for fifth and sixth grade teachers to compare their materials. The other six sessions observed were week-long summer institutes that covered pedagogical and mathematics content for fourth grade teachers in their first year of the ACME project and fifth grade teachers, most of whom were in their second year. The evaluator conducted these additional sessions to meet the project director’s request for a complete assessment of professional development provided by all of the facilitators. For the remaining sessions, the evaluator was a participant observer. Two of these observations included the summer institutes for sixth and seventh grade teachers for which the project hired facilitators from Michigan who have extensive experience as trainers and have test-piloted CMP in their classrooms. Observations that were rated lasted from one hour to one-half of a day, and informal observations lasted from one hour to one-half of a day or longer.

This evaluation also included information about project procedures and its history as well as district policy gleaned from semi-structured interviews with district administrators and project staff. These sources include the deputy superintendent of instructional services and school operations, the mathematics supervisor and ACME project director, the project manager, and a mathematics specialist who facilitates professional development for all district employees. A project director of the State Systemic Initiative (SSI) at the Dana Center reported on the project’s history and design. This person has been instrumental in planning the project and participated in an early mathematics reform effort in the district as an area superintendent. In addition, the lead evaluator participated in the ACME project meetings, in a retreat at which the facilitators reflected on the first year of implementation, and in informal conversations with project staff. These interviews and observations provided different perspectives of the project’s functioning in the baseline year.

Additional sources of data for this report include the district’s grant proposal to NSF and subsequent revisions, the district and state mathematics curriculum documents, professional development materials, and brochures for parents.