Demographics and Performance in New York City's School Networks School Hetnrm Annenheru Institute tor An Ini ial Inquiry SS A SS. . . ABOUT THE AUTHORS Norm Fruchter is a principal associate and Christina Mokhtar is a principal research associate at the Annenberg Institute for School Reform at Brown University. Toi Sin Arvidsson is a doctoral student at Teachers College, Columbia University. The late John Beam was principal analyst at Pumphouse Projects. Warren Simmons, executive director of the Annenberg Institute, wrote the Introduction “Reflections on National Significance.” ACKNOWLEDGMENTS We are grateful for the help, critiques, and support provided by Ray Domanico and Asa Wilkes at the New York City Independent Budget office, Jason Becker at Allovue, and Rosann Tung and Megan Hester at the Annenberg Institute for School Reform. Participants in Leslie Santee Siskin’s New York University seminar on high school research offered many useful suggestions. Our colleague John Beam’s analyses of New York City principals’ evaluations of their networks sparked our initial efforts. His death, on November 11, 2014, when this study was nearing completion, ended many decades of commitment to improving learning and living conditions across New York City’s struggling neighborhoods. ABOUT THE ANNENBERG INSTITUTE FOR SCHOOL REFORM AT BROWN UNIVERSITY The Annenberg Institute for School Reform (AISR) is a national policy-research and reform-support organization, affiliated with Brown University, that focuses on improving conditions and outcomes for all students in urban public schools, especially those attended by traditionally underserved children. AISR’s vision is the transformation of traditional school systems into “smart education systems” that develop and integrate high-quality learning opportunities in all areas of students’ lives – at school, at home, and in the community. AISR conducts research; works with a variety of partners committed to educational improvement to build capacity in school districts and communities; and shares its work through print and Web publications. © 2015 Brown University, Annenberg Institute for School Reform SUGGESTED CITATION Demographics and Performance in New York City's School Networks: An Initial Inquiry. 2015. Fruchter, N., T. S. Arvidsson, C. Mokhtar, J. Beam, and W. Simmons. Providence, RI: Brown University, Annenberg Institute for School Reform. Demographics and Performance in New York City's School Networks An Initial Inquiry Norm Fruchter Toi Sin Arvidsson Christina Mokhtar John Beam with an introduction by Warren Simmons Contents Executive Summary Reflections on National Significance About the New York City Networks Study History of School Networks in New York City Findings: Network Demographics Findings: Academic Outcomes Conclusions References Appendix Executive Summary Reflections on National Significance Large urban districts across the United States face an often-daunting challenge: What kind of system will best administer and support schools with widely differing needs and result in high achievement for all the city’s students? New York City is no exception. The city’s current school support system – the Children First Networks (CFNs) – organizes schools into non-geographic networks that aimed to preserve school autonomy by providing instructional support that was separate from supervision. Supporters of this system also anticipated that spreading out the networks geographically would break the relationship between demographics and student achievement. Our study found that while some principals and educators felt that the CFNs have successfully supported their needs, the networks have not been very successful in meeting the needs of students and communities. Student and school demographics are still the best predictor of student and school outcomes. The lessons from New York City’s experiment are not only highly relevant to the new city administration, but can also help inform similar discussions in other cities. Research by the University of Chicago Consortium on Chicago School Research and the experiences of New York City and London support the idea that effective professional learning communities must be firmly anchored in local neighborhoods and organized around the needs of students and their communities. This builds the capacity of educators to get to know their students well and reflect their aspirations and needs; allows for the creation of multiple pathways to learning through internships and other local partnerships; encourages parental involvement in support of learning; takes advantage of the cultural capital of communities; and strengthens communities, along with many other benefits. This approach is consistent with AISR’s concept of a smart education system – a high-functioning district working with local community partners to provide a comprehensive web of supports for children and families and foster high levels of learning and development. Another characteristic of the CFNs is the high degree of autonomy of schools. But autonomy, per se, is no guarantee of improved performance – it may work well for schools that already have high capacity, but is less successful for struggling schools. Even strong schools have many needs that are beyond their control, and many are under-resourced. Communities welcome chances to pursue some learning opportunities in distant locations, but most of all what they want is strong neighborhood schools. Pursuing innovation and choice, some initiatives have undermined the quality of local schools, particularly in poor neighborhoods. Future investment in school support networks and professional Annenberg Institute for School Reform 1 learning communities should allow for innovations, including technology, that can facilitate exploration and collaboration without putting an excessive burden on the poorest students and on educators and network staff forced to travel far from their neighborhoods. But to effectively address the pernicious achievement and opportunity gaps that plague our educational systems, future investments must focus on supports that will create thriving schools with a firm anchor in thriving communities. 1 The study was fully funded by the Annenberg Institute. About the Study and the the History History of of School SchoolNetworks Networksin inNew NewYork YorkCity City How to structure the New York City school system so that it most effectively administers and supports its schools and students has been a contentious issue for decades – witness the progression of governance and structural changes from community control to decentralization to autonomy to market-based choice. From 2003 to 2012, the administration of Mayor Michael Bloomberg implemented major structural changes in the New York City school system and invested great hopes in the potential of those changes to improve student academic performance. One of the previous administration’s critical structural changes was the formation of associational, rather than geographic, networks of schools. The city system is now organized into fifty-five Children First Networks (CFNs) that provide instructional and operational support to their member schools. Though these networks offer the bulk of the system’s instructional and administrative support to the city’s schools, very little public information about the networks’ student and school composition or their academic effectiveness has been available. Given the recent changes in city government and school system leadership, as well as the need for more information about this relatively new school support structure, the Annenberg Institute for School Reform at Brown University undertook and funded this research study in 2012 to explore several issues critical to the networks’ continued existence. We hope our study informs discussion about the best way to organize school systems to improve outcomes for all the city’s students. We also hope our findings will be useful to other cities across the nation facing the same challenges.1 The study addressed two research questions: • What are the patterns of student demographics – race/ethnicity, poverty, special education, English language learner (ELL) status, and student performance – within and across the school system’s fifty-seven CFNs and their member schools? • What do the CFNs contribute to the academic outcomes of their member schools? 2 Demographics and Performance in New York City's School Networks: An Initial Inquiry Findings: Network Demographics The CFNs vary considerably in terms of member schools’ size, enrollment, grade levels served, and race/ethnicity. When we compared the extent of segregation in the networks to the extent of segregation in the community school districts in 2002, the year before that systemic organizational structure was terminated, we found that the CFNs in 2012 were less racially/ethnically segregated at the elementary and middle school levels than the community school districts were in 2002. This reduction in segregation probably reflects the non-geographic nature of network membership. Findings: Academic Outcomes We analyzed the variations in the CFNs’ compositional factors (network size, mix of schools, and student demographics), and computed a series of academic outcomes across the networks. We analyzed the variance in academic performance across the networks, and considered the relationship between network choice and student academic performance in relation to student demographics and student academic performance. Our findings are: 2 In the month before this study was released, the New York City Schools Chancellor announced a new systemic policy that made school district superintendents responsible for supporting schools and holding them accountable for results and that dissolved the Children First Networks. • Choice of network is associated with school-level academic performance, but not very robustly. • School-level student demographics predict academic performance far more strongly than network membership does. • When key student demographic variables – such as poverty and the percentages of English language learners and students with disabilities – are held constant at the citywide average, network membership has very little effect on student academic performance, except possibly for high school graduation rates.2 Annenberg Institute for School Reform 3 Reflections on National Significance Introduction to the New York City Networks Study by Warren Simmons, Executive Director, Annenberg Institute for Schools Reform at Brown University Large urban districts across the United States face an often-daunting challenge: What kind of system will best administer and support schools with widely differing needs and capacities, ultimately resulting in high achievement for all the city’s students? Does a system better support its residents if it is geographically organized, or should students and schools have broad access to high-quality options outside of their neighborhoods? Should schools have a high degree of autonomy to decide what works best for them, or should accountability and support systems be more centralized? Should systems that hold schools accountable be separate from systems that provide schools with support, or should they be unified? New York City has debated these issues for decades. The latest choice of the administration of Michael Bloomberg and Joel Klein was a system – the Children First Networks (CFNs) – that was non-geographically based, that granted a high level of autonomy to schools, and that separated accountability and support systems. The lessons from this experiment are not only highly relevant to the new administration in New York City, but can also help inform similar discussions in other cities. This report shares findings from a study undertaken on the CFNs by the Annenberg Institute for School Reform (AISR). While some principals and educators have expressed satisfaction with the way the CFNs have supported their needs, this study showed that the networks have not been very successful at meeting the needs of students and communities. They have little impact, on average, on student achievement or school performance. Student demographics – that old, persistent, inequitable gateway to life opportunities – still seems, according to the findings, to be the best predictor of student and school outcomes. The CFNs aimed to break the links between student demographics and academic outcomes, but they have not succeeded in doing that. In this introduction to the report, I would like to take a deeper look at why that might be. Creating the Right Kind of Professional Learning Communities Research supports the effectiveness of some professional learning communities (PLCs) to improve student learning. But what kind of professional networks contribute to student gains? Our study found little evidence-based rationale behind the characteristics of each CFNs – educational level and size of schools served or level of student need – or the way principals choose networks – often by perceived common interests or ties among principals and educators. The interests of administrators and educators must be balanced with the business of schools, which is to educate young people. Network resources should be devoted to and organized around the needs of students and their communities, especially in an era where city budgets are more and more limited. 4 Demographics and Performance in New York City's School Networks: An Initial Inquiry AISR’s two decades of research and experience supporting excellence and equity in urban public education suggest that the most successful school systems have deep ties to the neighborhoods they serve. This helps build the capacity of educators to get to know their students well and reflect their aspirations and needs, allows for the creation of multiple pathways to learning through internships and other local partnerships, encourages parental involvement in support of learning, takes advantage of the cultural capital of communities, and strengthens communities, along with 3 See McAfee and Torre 2015, and http:// many other benefits. Some approaches to school support networks, like promiseneighborhoodsinstitute.org/. Promise Neighborhoods,3 the Strive network,4 and the community 4 See http://www.strivetogether.org/. schools model5 are built around this recognition. This approach is consis5 See http://www.communityschools.org/. tent with AISR’s concept of a smart education system – a high-functioning district working with local community partners to provide a comprehensive web of supports for children and families and foster high levels of learning and development. Such a system places children and families at the center, involves crosssector partnerships, aims at improving a broad set of outcomes for students and families, and involves shared accountability for improving those outcomes. Geographical vs. Non-geographical Networks The tension between neighborhood schools and non-geographically based learning options is not new. Strategies like busing and magnet schools have sought to create more high-quality choices for students and families, but have often met with resistance, since they place greater burdens on the students who must travel the furthest. They also fail to take advantage of community-based assets; make it harder for teachers to get to know their students; and make it harder for parents and community residents to participate in their children’s schooling and hold the educational system accountable. Networks of support for schools that are spread out geographically, like New York’s CFNs, run into similar problems. In the section on the History of School Networks in New York City, this report cites some of the concerns of former chair of the New York City Council’s Education Committee, Robert Jackson, that network staff lose too much time traveling between schools and that parents have no place in their community to engage network staff. The lack of a geographic network shuts out parents – when there is no physical location to go to and the networks cover large geographic distances, there is little visibility, accessibility, and transparency. There is strong evidence that the best professional networks for education are deeply embedded in local communities. Research by the University of Chicago Consortium on Chicago School Research (CCSR) identified strong professional learning communities and parent-community ties as two of the five essential supports and community resources that facilitate school improvement and contribute to improvements in student learning, but note that community resources are often weakest in high-needs neighborhoods (Sebring et al. 2006). Reform efforts that disrupt neighborhood ties are in danger of even further eroding those potential supports. Annenberg Institute for School Reform 5 Research on school reform efforts in New York City and London show that schools do best when the social and political capital of communities are accessed. In a 2007 AISR study of thirteen New York City Schools that succeeded in preparing low-performing students for graduation and college, Carol Ascher and Cindy Maguire (2007) found that the beat-the-odds schools rely on local community assets to maintain high expectations for college: Parents are involved in college planning through workshops on testing, college requirements, and financial aid. Relationships with local community-based organizations provide an array of critical resources, from student internships to help with college essays. (p. viii) The BTO schools collaborated with local community-based organizations, where students were able to participate in service learning and the kinds of extracurricular activities and community service opportunities valued by admissions officers – traditionally more available to middle-class students. (p. 11) Another AISR project, the Transatlantic School Innovation Alliance (TSIA), aimed to improve teaching, learning, and educational leadership by creating a peer network of principals and practitioners in urban secondary schools in the United States and England. A 2011 TSIA report found that “the most successful schools in London are not isolated and separate from their local community and other schools but actively encourage and embrace interaction with others” (Mishook, McAlister & Edge 2011, p. 8). An issue of AISR’s journal Voices in Urban Education (VUE), “A Smart System in London,” examined how Tower Hamlets, a local education authority (equivalent to a U.S. school district), in the TSIA study that serves a high percentage of immigrants and low-income students, has been highly successful across a range of health, social, educational, and economic indicators. Strong and intentional local community partnerships, coordinated neighborhood services, family involvement and training, home visits, and service delivery monitoring are some of the key features of the Tower Hamlets system (Rothman 2008). In contrast, structural efforts to support schools in New York City over the period of the Bloomberg administration, particularly Children First Networks, have in some ways undermined the community assets associated with geographic organization. AISR’s 2007 TSIA report found, for instance: The repeated shifts in the support structure since 2002 may have disrupted informal geographically based networks that previously existed. . . . Principals had no mechanisms for learning from or working with nearby schools facing the same neighborhood issues and serving similar populations. The strategy of replacing large traditional high schools with small themed schools has meant the recruitment of hundreds of new principals and assistant principals spread across the city, a growing proportion of whom are recruited from fields other than education and have few relationships with other school leaders. (Mishook, McAlister & Heck 2011, pp. 13–14) 6 Demographics and Performance in New York City's School Networks: An Initial Inquiry Autonomy: Not a Silver Bullet Another key feature of New York City’s CFNs is the high degree of autonomy granted to schools and the separation of support functions from monitoring and accountability functions. Autonomy, per se, is no guarantee of improved performance – it may work well for schools that already have high capacity, but is less successful for struggling schools. The TSIA report found (Mishook, McAlister & Heck 2011): Network leaders serve at the pleasure of principals and have no formal authority over them. Critics note that that this approach might be insufficient to develop the capacity of new principals and new teachers. . . . While experienced principals may benefit from autonomy, the hundreds of new principals and tens of thousands of new teachers might require more direction and guidance. (p. 13) The multiple reorganizations of that school system have, on the one hand, increased individual school autonomy and accountability, while at the same time dismantling many of the older structures that allowed and even encouraged schools to share knowledge and best practices. This movement from a more collaborative to a more competitive system has produced real tradeoffs. (p. 17) Schools – even high-capacity ones – can’t solve all problems on their own. The BTO study on New York City beat-the-odds schools (Ascher & Maguire 2007) found: The BTO schools we investigated in our qualitative study should be considered strong schools – high administrative capacity to develop and sustain critical academic programming was reflected in high student achievement. Some BTO principals appreciated the increased autonomy they had experienced over the past several years, saying it had allowed them to design curriculum and other interventions to meet the needs of their students. But they didn’t believe that they could solve every problem internally. (p. 14) The New York City Department of Education has decentralized budgetary decision making to the school level. But some BTO schools do not have adequate resources, and state or district mandates interfere with their ability to allocate scarce resources to address school needs. (p. viii) Investing in the Future New York City, like any large city, is composed of neighborhoods with substantial social and cultural differences. And, like other large cities, it has struggled with how to organize its education system to balance the need for schools to support each other with the need for schools to support their students and families. The previous structure of geographically based Community School Districts attempted to locate control in neighborhoods across the city, on the assumption that schools governed locally would build professional capacity more responsive to their communities. That assumption turned out to be wrong. The Children First Networks dissolved the geographical ties among schools on the assumption that this would reduce the influence of demographics on student achievement. That assumption also turned out to be wrong. Annenberg Institute for School Reform 7 What is the solution? AISR’s work leads us to believe that transforming our nation’s urban public schools at scale cannot be achieved without a strong anchor in local communities. Demands that we let students – or educators, or schools – pursue high-quality opportunities no matter where they are located make sense when many students are trapped in low-performing neighborhood schools. But surely behind these demands is the reality that what we really want is to create high-quality opportunities in every neighborhood and respect local values, needs, and aspirations. In the name of innovation and choice, some initiatives have undermined the quality of neighborhood schools, particularly in poor neighborhoods. Many great learning opportunities can still work over distance. Young people from lessprivileged neighborhoods should have ample access to these opportunities, and principals and educators should have the necessary time and resources to engage in lively professional networks with their peers in other locations. Investment in innovation zones should include technology that allows exploration and collaboration without necessarily traveling long distances. But above all, cities, states, the federal government, and their partners, when they are investing in school systems, must focus on the kinds of supports that strengthen neighborhood communities and their schools. Without being closely connected to thriving communities, schools will wither, and even the best ones can only hope to become outposts of learning, lone surviving community assets in ruined landscapes. As a nation, we can do better. 8 Demographics and Performance in New York City's School Networks: An Initial Inquiry About the New York City Networks Study How the New York City school system should be structured to most effectively administer and support its schools and students has been a contentious issue for decades – witness the progression of governance and structural changes from community control to decentralization to autonomy to market-based choice. As part of its structural reorganization of the New York City school system, the administration of Mayor Michael Bloomberg implemented networks of schools to provide instructional and operational support. Many policymakers, school leaders, and frontline practitioners perceive networks as providing more effective school support than the previous community school district structures managed. But many parent leaders, advocates, and elected officials have criticized the lack of network transparency, accessibility, and accountability, and have decried the limited, diffuse, and differentiated nature of supports that networks provide.6 Though the city’s schools are currently organized into fifty-five Children First Networks (CFNs), we know very little about the networks’ student demographics and academic performance, and even less about whatever effects the networks might have on student academic outcomes. Therefore this initial study seeks to explore the following questions: • What are the patterns of student demographics – race/ethnicity, poverty, special education, and English language learner (ELL) status – and student performance within and across the school system’s CFNs? • What do the networks contribute to the academic outcomes of their member schools? Given the recent changes in mayoral and school system leadership, this seems an appropriate moment to investigate the demographics and the achievement outcomes of the school system’s networks. The Annenberg Institute for School Reform at Brown University, building on its years of work in New York City in support of various education initiatives, undertook and funded this research study in 2012 to explore several issues critical to the networks’ efforts. We hope to inform discussions about the best way to organize school systems so that they improve conditions and outcomes for all the City’s students. We also hope our findings will be useful to other cities across the nation facing the same challenges. Annenberg Institute for School Reform 6 At an October 2013 conference sponsored by the Office of the Public Advocate for New York City and the Annenberg Institute for School Reform at Brown University, a recommendation to terminate the network structure received almost unanimous support from the attendees. Conversely, more than a hundred principals wrote then-Mayor-elect Bill de Blasio in November 2013 urging the networks’ continuation (Decker & Wall 2013). 9 History of School Networks in New York City 7 New York City’s high schools were centrally administered. 8 This grant was part of the Annenberg Challenge, a $500 million initiative by Ambassador Walter H. Annenberg consisting of eighteen projects in thirty-five states. The Annenberg Foundation also provided a grant to the Annenberg Institute for School Reform to coordinate Challenge activities. See http:// annenberginstitute.org/ challenge/about/ about.html. 9 10 The New York ACORN chapter has been replaced by the New York Communities for Change. See http://www. nycommunities.org/. If the entire Annenberg Foundation grant had been allocated exclusively to support NYNSR’s member schools, only $35,000 per school would have been available to participating schools in each of the five project years. Less was actually available because of the project’s administrative costs. 10 Until recently, the New York City school system was organized by geography rather than by networks of associated schools. From 1970 to 2003, the city’s elementary and middle schools were administered by thirty-two geographically based community school districts (CSDs).7 Schools within each district varied considerably in student composition, size, educational philosophy, instructional practice, and student outcomes. School district geographic boundaries often yoked together very different schools serving very different neighborhoods. The network as a structural concept was initiated in 1994, through a grant from the Annenberg Foundation of $25 million grant to the New York Networks for School Renewal (NYNSR).8 NYNSR was composed of four school reform organizations – New Visions for Public Schools, the Center for Educational Innovation, the Center for Collaborative Education, and the Association of Community Organizations for Reform Now (ACORN).9 NYNSR’s founding proposal to the Annenberg Foundation pledged to create more than 100 small schools, “organized according to a set of principles that stressed small size, autonomy, personalization, and the formation of professional teaching communities. The proposal envisioned that the participating schools would form networks for mutual support and accountability, and that eventually, a Learning Zone would be established through which successful networks could demonstrate new forms of school organization, administration, and governance for the New York City school system” (Institute for Education and Social Policy 2001, p. 1). Through the resulting five-year (1996–2001) effort, “NYNSR created, restructured, or reorganized almost 140 schools serving almost 50,000 students” (IESP 2001, p. 33). Test scores and graduation rates at NYNSR schools exceeded citywide averages, and longitudinal achievement gains at member schools topped the performance of comparison schools. Other analyses indicated that NYNSR small schools were more cost-effective than comparison schools. (Stiefel et al. 2000, p. 22, 27–39) The 140 NYNSR schools were grouped into networks managed by the four sponsor organizations, and each sponsor distributed discretionary funding from the Annenberg Foundation to their member schools. But the pressures of creating new schools, the limited funding available for the entire effort,10 and the lack of sustained school system support combined to marginalize the networks’ efforts. The concept of a Learning Zone that schools could choose to join, and within which schools would be granted autonomy from administrative constraints in exchange for producing positive student outcomes, survived the NYNSR experiment. Several leaders of successful NYNSR schools helped develop an equivalent zone as the school system was reorganized during the subsequent decade. Demographics and Performance in New York City's School Networks: An Initial Inquiry In 2002, with Michael Bloomberg as New York City’s new mayor, the New York State legislature passed a mayoral control law that transformed the city school system into a city department, thereby abolishing the thirty-two locally elected community school boards and terminating their powers over the community school districts. The new law also severely limited the power of the citywide Board of Education, which was renamed the Panel for Educational Policy.11 Mayor Bloomberg and New York City Department of Education Chancellor Joel Klein initiated massive structural change that consolidated the city system into ten geographic regions, organized primarily to provide instructional support. Each of the ten regions was headed by a superintendent who supervised a team of some ten to twelve local instructional superintendents, who each provided instructional support to ten to twelve schools. Six borough-based Regional Operations Centers were established to help schools deal with logistical, fiscal, personnel, and other management issues. In 2004, Chancellor Klein initiated the Autonomy Zone, a pilot effort similar to NYNSR’s proposed Learning Zone, in which twenty-nine school principals and the directors of three charter schools signed performance agreements that held those schools accountable for meeting student academic performance targets in exchange for the power to make critical decisions at the school level “about staffing, scheduling, curricula, instruction, and assessment that had typically been determined by central office” (Nadelstern 2012). In 2005, the Autonomy Zone grew to include forty-eight schools. In 2006, Chancellor Klein expanded the option of joining the Autonomy Zone to all the system’s schools, and some 330 principals chose to participate. The Autonomy Zone was renamed the Empowerment Schools Project. Participating principals formed networks of approximately twenty-five schools each and hired instructional leaders to guide their networks’ development. Because the Regional Operations Centers that provided fiscal, management, and logistics support were still borough-based, the Empowerment Schools Project created Integrated Service Centers to provide support to member schools. In 2007, the citywide regional structure, through which all the system’s schools had been administered (other than those participating in the Empowerment Schools Project), was dissolved. The system’s principals were asked to join one of three types of school support organizations: • the Empowerment Support Organization, another relabeling of the original Autonomy Zone; • one of four citywide Learning Support Organizations (LSOs) developed within the New York City Department of Education (DOE) and offering assistance, support, and oversight; or • one of six Partnership School Organizations (PSOs) managed by external school reform nonprofits or universities, which also offered varieties of support. Each of these school support organizations grouped their participating schools into networks – essentially teams of instructional coaches supporting groups of schools, sometimes sharing an educational philosophy12 – and schools paid an annual fee to their Annenberg Institute for School Reform 11 Norm Fruchter, one of the authors of this report, serves as a mayoral appointee on the Panel for Educational Policy. 12 “The reason schools initially chose to affiliate with a specific network vary greatly. Many networks remain historically based around an educational philosophy (e.g., project-based learning) or principals who were previously socially networking with their colleagues. A few networks were formed based on geographic proximity; however, most span several boroughs. Some networks were created based on similar student populations (e.g., a large number of the special education high schools are together in one network, as are the schools that target ELL students). Some networks are grade-level specific, with only a few networks spanning kindergarten through grade 12.” (Wohlstetter, Smith & Gallagher, p. 536) “Some networks are organized around a distinct educational philosophy or school model; others around a common set of student needs; and others around strong personal relationships” (Parthenon Group 2013). 11 support organizations to help defray the cost of services provided. The Regional Operations Centers were terminated, but the Integrated Service Centers continued to provide logistical and management support (Childress et al. 2011). 13 14 “Over the past three years (2010–2013), the DOE has closed eight networks due to poor performance and launched six new ones in their place” (Wohlstetter, Smith & Gallagher 2013, p. 5). The city’s Independent Budget Office (IBO) recently published a Schools Brief showing a substantial decrease in school support services from 2002-2003, the school year before the Bloomberg administration eliminated community school districts, to the 2011-2012 school year. In expenditure adjusted for inflation, school support costs reached almost $377 million in 2003, compared to $293 million in 2012, a 22 percent reduction. In terms of dollars per student spent on school support, costs fell from $345 per student in 20022003 to $281 per student in 2011-2012, an almost 19 percent reduction (New York City Independent Budget Office 2014, p. 1). 12 By 2010, all the system’s schools were reorganized into some fifty-five Children First Networks (CFNs) and five citywide clusters that supervised the networks’ efforts. The Integrated Service Centers were dissolved, and the network teams were expanded to provide both instructional and operational supports. The DOE developed evaluation metrics to assess the networks’ efforts, and over time, a small number of networks were disbanded and new networks constituted.13 Schools are permitted to change their networks annually. According to the DOE, less than 8 percent of all the networked schools request such a change each year. Currently, there are fifty-five CFNs across the city school system, each staffed by (New York City Department of Education, n.d.): • a network leader and a deputy; • an instructional improvement unit of five achievement coaches, one exclusively for special education; • five management specialists focused on providing support for operations, human resources and payroll, budget and procurement, data and information technology, and other school logistics; • three student and family services personnel providing support for special education, youth development, ELLs, and other student issues such as attendance, safety, and discipline. These networks are the school system’s major form of instructional and operational support to the city’s schools. The DOE’s Office of School Support monitors the networks’ performance and provides oversight and support for instructional, administrative, operations, finance, personnel, and facilities issues (New York City Department of Education n.d.). Aside from a substantial reduction in the cost of school support services, we know little about how effectively the CFNs have served the city’s schools.14 On October 25, 2012, Robert Jackson, then the chair of the New York City Council’s Education Committee, introduced a hearing on the DOE’s networks by saying that, since the latest reorganization in 2010, “very little information about CFNs has been available despite the fact that CFNs have been the sole support structure for all schools” (City Council, City of New York, 2012). Jackson went on to say: The primary concern has to do with the effectiveness of CFNs as school support structures. Little is known about CFN evaluation, but there appears to be high turnover among network staff. Some networks may be more effective than others, but . . . we’ve heard repeatedly that some network staff spend too much time traveling between schools in networks that are not organized geographically, resulting in less time spent helping schools. Parents also complain that there is no network office in their commu- Demographics and Performance in New York City's School Networks: An Initial Inquiry nity where they can go to meet network staff. We’ve also heard criticism that networks work primarily with school principals and spend very little time working with teachers or School Leadership Teams. . . . A major concern we’ve heard is the lack of network accountability. As one parent leader said, networks are “not held accountable for failures but often celebrated for the successes.” We understand that the DOE has structured CFNs as a choice system, in which principals can switch to a new network if they’re not happy with their current one, but this open marketplace approach itself raises some concerns, as networks are forced to market themselves to schools. Compounding these concerns was the issue of transparency in the DOE’s management of its school support networks. At the same hearing, the DOE’s then–chief academic officer and senior deputy chancellor, Shael Polakow-Suransky, while disputing that networks were not being held accountable, testified that the department should have been more transparent about the networks’ roles (Cromidas 2012). The larger point you’re making – “Have we not done a good job sharing with the public all the information we have and can share?” – is right. The network structure evolved very swiftly from an initial pilot to a systemwide effort. Little research seems to have been conducted about critical implementation issues such as the optimal network size and scale, the most appropriate mix of member schools, and the most effective staffing for school-level instructional support. This study hopes to provide more information and knowledge about the CFNs by exploring two critical issues: • What are the patterns of student demographics – race/ethnicity, poverty, special education, English language learner (ELL) status, and student performance – within and across the school system’s fifty-seven CFNs and their member schools? • What do the CFNs contribute to the academic outcomes of their member schools? Annenberg Institute for School Reform 13 Findings: Network Demographics 15 When this report was released, there were fiftyfive CFN’s. During the 2011-2012 period our study’s data were based on, there were fifty-seven CFNs. 16 Our networks’ data are based on the NYCDOE’s school year 2011-2012 Location Code Generation and Management System (LCGMS) data file released by the Office of Organizational Data. We excluded eighty schools that were new and did not have progress reports for 20112012, and another 120 schools because they were either special education schools in District 75, or charter schools that are not part of the DOE’s network structures. Our final analysis file included 1,452 schools in the fifty-seven networks. 17 These conditions may result in the outlier schools (the minority of schools with a different grade-level structure) experiencing difficulty receiving the most appropriate supports, or being isolated from the majority of the schools in their networks. 14 The CFNs show considerable variance in the number of schools each network serves. Though the average number of member schools across the fifty-seven CFNs15 is almost twenty-five, the range varies from a low of sixteen member schools in one network to a high of thirty-eight in another (see Appendix, Figure A1). This range primarily reflects network decisions. According to the DOE, “No network may support more than thirty-five schools. Growth past thirty schools is at the discretion of the cluster” (New York City Department of Education, n.d.). The student population of each network also varies considerably. Though the average student enrollment across the CFNs is almost 17,000, enrollment rates vary from a high of 49,567 students in the most populous network, to a low of 7,627 students in the least populous16 (see Appendix, Figure A2). The CFNs also vary in the mix of elementary and K–8 schools, middle schools, and high schools in each network. In more than 40 percent of the CFNs, for example, almost all their schools serve a predominant grade-level structure, and only one or two schools serve a different grade-level organization. Almost 30 percent of the CFNs have all but one school serving the same predominant grade level. Network 201, for example, includes thirty-two high schools, but has only one elementary school and one middle school in its membership. Network 205 has twenty-seven elementary schools but only one middle school, while Network 536 has fourteen middle schools and sixteen high schools, but only one elementary school17 (see Appendix, Figure A3). The networks also vary in school size, meaning the student enrollment of their member schools. (see Appendix, Figure A4). In six CFNs, for example, more than 75 percent of their schools are small (fewer than 500 students), while in three networks, more than 50 percent of their schools are large (more than 1,000 students). We found no documentation of how school enrollment, grade levels served, or variations in school size were determined within and across CFNs. The CFNs also vary considerably by race/ethnicity – the percentage of White, Asian, Black and Latino students within each network. Figure 1 displays those CFNs whose percentages of White and Asian students place them in the bottom third of all networks, in terms of the lowest concentrations of both groups of students, as well as those CFNs whose percentages of Black and Latino students place them in the top third of all networks, in terms of the highest concentration of both groups of students. Figure 2 displays the reverse phenomena – networks whose percentages of White and Asian students situate them in the highest third of all networks, as well as networks whose percentages of Black and Latino students situate them in the lowest third of all networks. See Appendix, Figure A5 in the for student race/ethnicity percentages for all 57 CFNs. Demographics and Performance in New York City's School Networks: An Initial Inquiry The CFNs also vary in the percentages of the students in their member schools eligible for free and reduced-price lunch (FRPL) a proxy for poverty, as well as the percentages of English language learners, and students with disabilities (special education) (see Appendix, Figure A6). FIGURE 1 Networks with the Lowest Proportions of White and Asian Students and the Highest Proportions of Black and Latino Students (2012) Network White Asian Black Latino N551 5% 5% 39% 50% N603 4% 4% 41% 50% N608 3% 5% 36% 56% N105 3% 4% 41% 51% CITY AVERAGE 15% 16% 28% 40% NOTE This study draws on a list of New York City schools by network (Location Code Generation and Management System [LCGMS] school-level database) released by the NYCDOE’s Office of Organizational Data. Student demographic and performance variables are aggregated up to the network-level based on this school-level list by network. SOURCE NYCDOE. School Demographics and Accountability Snapshot (CEP 2012). FIGURE 2 Networks with the Lowest Proportions of Black and Latino Students and the Highest Proportions of White and Asian Students (2012) Network White Asian Black Latino N409 41% 20% 7% 31% N405 37% 23% 17% 22% N201 19% 38% 15% 27% CITY AVERAGE 15% 16% 28% 40% SOURCE NYCDOE. School Demographics and Accountability Snapshot (CEP 2012). Annenberg Institute for School Reform 15 18 19 20 The Dissimilarity Index measures how evenly groups are distributed across any given system. Each weighted index represents how dissimilar a racial/ethnic group of students is from the rest of the population within a unit. (Populations Studies Center n.d.) Schools in 2002 and 2012 were categorized into elementary and K–8, middle schools, and high schools based on the DOE’s progress reports’ definition of school types. We calculated the City Index using each school as a unit. We calculated the Community School District Index using each community school district (CSD) as a unit, and we calculated the Network Index using each network as a unit. Note that this comparison is limited. We excluded high schools in 2002, because they were not administered by the community school districts, and because their numbers were far fewer in 2002 than in 2012. We also excluded transfer high schools for similar reasons. 21 The school system’s initial restructuring into ten administrative regions also grouped together dissimilar districts (in terms of segregation indices) into the same region. 22 It is worth noting that, in spite of the reduction in segregation the networks have achieved, the New York City school system is the nation’s third most racially segregated school district (Fessenden 2012). 16 Segregation and the Dissimilarity Index To determine whether systemwide racial/ethnic concentrations of New York City’s students have changed across the past decade, we analyzed the racial/ethnic mix of the school system’s student population in 2002 and compared it to the system’s population mix in 2012. We calculated a Dissimilarity Index, based on methods used to analyze the extent of municipal housing segregation, to compare the extent of change in the school system’s student composition across the past decade (Populations Studies Center n.d.).18 Figure 3 displays the Dissimilarity Index for elementary and K–8 schools, middle schools, and high schools in 2002 and 2012.19 The figure compares the extent of segregation in the city system (City Index) for 2002, the year before the Bloomberg/Klein administration introduced systemwide restructuring, and for 2012, after a decade of systemic change. We also compare the extent of segregation in elementary and middle schools across the community school districts (Community School District Index) in 2002, the last year of their existence, with the extent of segregation in elementary and middle schools across the CFNs (Network School Index) in 2012.20 The numbers in Figure 3 represent the percentage of students who would need to be reassigned to other schools to create a racial/ethnic distribution of students in each school equivalent to the school system’s average racial/ethnic distribution. The high percentages in Figure 3 indicate a high degree of segregation. In the City Index for 2002, for example, more than half the Asian and Black students in elementary, K–8, and middle schools would have to be reassigned to other schools to produce the average citywide racial/ethnic distribution in each school. Lower percentages in Figure 3 indicate somewhat less segregation. The data in Figure 3 (on next page) show several trends. The City Indices suggest that student segregation by race/ethnicity remained roughly the same in 2002 and 2012, except for a small decrease in White student segregation at the elementary/K–8 level, as well as an increase in Asian student segregation at the high school level, in 2012. The data also suggest that the networks in 2012 were less segregated than the community school districts (CSDs) in 2002, at the elementary and middle school levels. This decrease in segregation from CSDs to CFNs probably occurred because the current networks are much less geographically based. The CFNs were purposely created as nongeographic entities, and network selection was based on school choice.21 The relative absence of geographically linked schools within the CFNs may well have produced a lower level of segregation than existed in the pre-2003 CSDs, which as geographic entities reflected the stark neighborhood and housing segregation that has characterized New York City – currently ranked third in the nation in terms of residential segregation (Fry & Taylor 2012).22 Demographics and Performance in New York City's School Networks: An Initial Inquiry Summary of Demographic Findings Our demographic analyses suggest two conclusions. First, several CFNs have very low numbers of one school type (defined by grade level) combined with large numbers of others. Since these outlier schools have few, if any, similar schools to collaborate with, the supports their networks provide may well be limited. Second, though the CFNs are quite segregated, they were less segregated at the elementary and middle school levels in 2012 than the community school districts were in 2002. This reduction probably results from the non-geographic nature of network membership, in which schools are linked by education philosophy, shared curricular concentrations, principals’ prior affiliations (particularly with network support staff), similar student populations, or similar grade organizations, rather than by geography. FIGURE 3 White 69% 63% 56% Asian 58% 54% 47% Black 60% 53% 43% Latino 50% 47% 43% Elementary and K–8 Middle School High School White 65% 63% 57% Asian 60% 57% 53% Black 60% 55% 44% Latino 48% 47% 43% NETWORK INDEX High School 2002 Middle School COMMUNITY SCHOOL DISTRICT INDEX Elementary and K–8 2012 2012 CITY INDEX 2002 CITY INDEX Dissimilarity Index in 2002 and in 2012 Elementary and K–8 Middle School *High School White 49% 52% N/A Asian 47% 48% N/A Black 46% 44% N/A Latino 44% 43% N/A Elementary and K–8 Middle School High School White 40% 44% 36% Asian 34% 37% 38% Black 33% 34% 27% Latino 30% 25% 24% SOURCE NYCDOE. School Demographics and Accountability Snapshot (CEP 2012). * High schools in 2002 were centrally administered and not part of the community school district system. Annenberg Institute for School Reform 17 Findings: Academic Outcomes This section focuses on student academic outcomes across the CFNs, and then explores some of the relationships between student performance and student demographics. The following analyses focus on the academic performance of networks by school level in 2012 (school year 2011-2012), specifically: • fourth grade ELA and math proficiency scores for elementary and K–8 schools; • eighth grade ELA and math proficiency scores for middle schools; • four-year graduation and four-year Aspirational Performance Measure (APM) college readiness rates for high schools. Figures 4–9 display network outcomes in three ways. The solid dots represent the weighted network average score for the academic measure the figure displays. The X’s represent the actual performance rates for each school within each network, which are arrayed and identified across the bottom of the figure. The horizontal line represents the overall average for all the networks citywide. Each network’s academic averages are FIGURE 4 Elementary Schools – Fourth Grade ELA Proficiency Scores (2012) SOURCE NYCDOE. New York State ELA test results (2012). 18 weighted by the number of students tested in ELA and math, or by the cohort size for graduation and college readiness/APM rates.23 Figure 4 indicates that, at the elementary school level, fourth grade ELA scores are dispersed across a roughly equal number of networks above and below a citywide average of 54 percent proficiency. There are some outlier CFNs – Networks 103 and 206 display positive outcomes, while Networks 611, 610, 109, and 608 are grouped at the lower tail of negative outcomes. (A solid square box is a combination of a solid dot – the networks average score – and an individual school score, which means that there is only one school of that level in that network. Thus, Network 201 has only one elementary school in Figures 4 and 5.) 23 Figures A7–A10 in the Appendix display the same data, ordered by high- to low-performing networks, for grades 4 and 8 ELA and math outcomes for 2012, and for APM and graduation rates at the high school level for 2012. The fourth grade math scores in Figure 5 display a higher citywide average at 67 percent, but the network distribution around the citywide average shows a similar pattern. Slightly more CFNs are below, rather than above, that average. The dispersion of individual schools is far more pronounced below the citywide average, which is quite possibly a ceiling effect, and there are few strongly positive outlier networks. See Figure A7 in the Appendix for fourth grade ELA and math data. FIGURE 5 Elementary Schools – Fourth Grade Math Proficiency Scores (2012) SOURCE NYCDOE. New York State math test results (2012). Annen- 19 In Figure 6, the range of outcomes for the CFNs’ eighth grade ELA proficiency scores is wide, and more networks are below, rather than above, the citywide average of 40 percent. Proficiency rates for individual schools are much more dispersed (a band of about 41 percent to 100 percent) above the citywide average, while the schools performing below the citywide average are much more concentrated in a band below the fortieth percentile. FIGURE 6 Middle School – Eighth Grade ELA Proficiency Scores (2012) SOURCE NYCDOE. New York State ELA test results (2012). 20 Demographics and Performance in New York City's School Networks: An Initial Inquiry The eighth grade math proficiency rates in Figure 7 show higher overall performance, with a citywide average of 57 percent. Slightly more networks fall below, rather than above, the citywide average. Schools with below-average proficiency scores are widely dispersed below 57 percent, and several CFNs show particularly poor performance. See Figure A8 in the Appendix for eighth grade ELA and math data. FIGURE 7 Middle Schools – Eighth Grade Math Proficiency Scores (2012) SOURCE NYC DOE. New York State math test results (2012). Annenberg Institute for School Reform 21 The CFNs’ four-year cohort graduation rates, shown in Figure 8, are relatively high, with an overall citywide average of 67 percent. The CFNS are equivalently dispersed above and below the citywide average, with several above-average outcomes in the 80 percent range. There is a wide range of school-level below-average graduation rates (see Figure A9 in the Appendix). In Figure 9, the citywide average for the CFNs’ High School College Readiness Index (APM rate) is 23 percent, with most networks falling below this average, indicating that the overwhelming majority of the city’s high school students are graduating ill-prepared to succeed in college. Twice as many CFNs perform below the 23 percent average than above it. Very few CFNs produce APM rates that exceed or approach 40 percent, and the dispersion of individual schools within or across networks is comparatively limited. A relFIGURE 8 Four-Year High School Graduation Rates (2012) SOURCE NYCDOE. Graduation Results, Cohort 2008 (Class of 2012). 22 Demographics and Performance in New York City's School Networks: An Initial Inquiry atively small number of schools in CFNs score above 60 percent college readiness. Member schools in Networks 201,24 521, 405, and 101 display strong APM performance (see Figure A10 in the Appendix). In summary, network outcomes on the fourth and eighth grade ELA and math tests show curves of roughly similar performance above and below the citywide average, with higher math results. The curve of network graduation outcomes is higher, again with an equivalent dispersion above and below the citywide average. Network college-ready (APM) performance is quite low, and twice as many networks fall below the citywide average than above it. Overall, fewer networks show positive, rather than negative, outcomes. 24 Network 201 includes most of the school system’s specialized, testentry high schools, as well as many other highperforming, over-subscribed high schools. FIGURE 9 High School Aspirational Performance Measure/College Readiness Rates (2012) SOURCE NYCDOE. Regents-based math/ELA Aspirational Performance Measure, Cohort 2008 (Class of 2012). Annenberg Institute for School Reform 23 Assessing Network Impacts We employed a hierarchical linear modeling (HLM) analysis to assess how much networks matter to school-level academic performance. Our analysis used the same set of academic outcomes as in the previous set of figures – fourth and eighth grade state ELA and math test results, high school cohort graduation rates, and college ready (APM) results – to assess how well network membership predicts these academic outcomes. We calculated the extent of variance in those outcomes between networks that was solely associated with network membership as a category. Higher percentages of variance between networks indicate that network membership is strongly associated with school academic performance, while lower percentages of variance indicate that network membership is weakly associated with that performance. Our analysis found that networks matter more for predicting school performance for elementary/K–8 followed by high schools, and matter the least for middle schools. Moreover, none of the percent of variance in academic outcomes at any school level reaches 30 percent, a fairly low threshold. Yet, this percentage, though low, is not negligible; it suggests that network membership can predict academic outcomes at the school level, but not very robustly. Figure 10 displays the between-network variance in academic performance from 2010 to 2012. FIGURE 10 Percent of Variance in Academic Performance between Networks Percent of Variance Performance Measures (2010) (2011) (2012) Percent of grade 4 students proficient in ELA 27% 26% 27% Percent of grade 4 students proficient in math 25% 25% 28% Percent of grade 8 students proficient in ELA 12% 14% 14% Percent of grade 8 students proficient in math 18% 11% 18% Elementary and K–8 School Middle School High School Four-year graduation rate 17% 17% 23% College readiness/APM 23% 21% 21% SOURCES NYCDOE. New York State ELA and math tests results (2010–2012). Graduation Results Cohorts 2006-2008 (Classes of 2010–2012). Regents-based Math/ELA Aspirational Performance Measure for 2010–2012. 24 Demographics and Performance in New York City's School Networks: An Initial Inquiry Selection issues may upwardly bias the extent of variance in academic outcomes between networks that our analysis found. Network membership is not random; it depends on school choice, which is influenced by factors such as principals’ relationships with like-minded colleagues, school affiliations based on educational philosophy or instructional practice, or a range of similar hard-to-measure factors. How much these choice/selection issues affect the percentages of variance across the networks’ academic outcomes is quite difficult to assess, especially given the limited data available to us. Additional research that specified the effects of the relationships underlying network choice might well produce considerably lower percentages of variance in academic outcomes across networks. Because school demographics are traditionally very strong predictors of academic performance, we also examined the extent to which school-level demographics predict network academic outcomes. In Figure 11, the results of our regression model show the percentage of variance for several network performance measures, across elementary and K–8 schools, middle schools, and high schools, explained by student demographics.25 The outcomes in Figure 11 suggest that student demographics predict network academic performance far more strongly than network membership. This is an important finding. The previous mayor and school system leadership implemented major structural interventions, such as grouping schools into networks, to break the links between student demo- 25 The regression model is estimated to predict performance outcomes at different school levels by controlling for the following student variables: enrollment, percentage of FRPL, percentage of White, percentage of Asian, percentage of Black, percentage of Latino, gender, percentage of special education, and percentage of ELL. For middle schools, fourth grade math/ELA proficiencies are also included. For high schools, eighth grade math/ELA proficiency, percentage of self-contained, and percentage of overage are also included. See Figures A11–A16 in the Appendix for the OLS estimates for each regression model. FIGURE 11 Percent of Variance in Academic Performance across Networks Explained by Student Demographics Percent of Variance Performance Measures (2010) (2011) (2012) Percent of grade 4 students proficient in ELA 64% 64% 70% Percent of grade 4 students proficient in math 57% 62% 61% Percent of grade 8 students proficient in ELA 83% 86% 89% Percent of grade 8 students proficient in math 70% 65% 63% Elementary and K–8 School Middle School High School Four-year graduation rate 44% 54% 73% College readiness/APM 78% 80% 87% SOURCES NYCDOE. New York State ELA and math tests results (2010–2012). Graduation Results Cohorts 2006-2008 (Classes of 2010–2012). Regents-based Math/ELA Aspirational Performance Measure for 2010–2012. Annenberg Institute for School Reform 25 graphics and academic outcomes. Yet our findings indicate that the CFNs’ academic outcomes are influenced far more strongly by student and school demographics than by network membership. 26 Our estimated HLM models assume that their group intercepts vary but that the slopes of all the predictors remain constant across networks. The exploratory analyses we conducted confirmed our assumptions. To more precisely assess how much the networks influence academic performance, we developed an HLM analysis that groups schools by network membership, controls for school-level student demographics, and predicts how a hypothetical school with systemwide average student demographics would perform in each network.26 The next series of six figures (12–17) display the results of our model’s application. We use the same six performance outcomes (fourth and eighth grade ELA and math proficiency results, as well as four-year high school cohort graduation rates, and high school college readiness (APM results) as dependent variables. We control for school-level student demographic variables using the same set of variables employed in the regression model in Figure 11 (see footnote 25) as predictors and assign each hypothetical school in each network the same systemwide average demographics. We use network membership as the grouping variable. FIGURE 12 Predicted Network Effects for Elementary Schools – Fourth Grade ELA Proficiency Rates (2012) SOURCE NYCDOE. New York State ELA test results (2012). 26 Demographics and Performance in New York City's School Networks: An Initial Inquiry The outcomes in Figures 12–17 show the predicted school-level academic performance of each network when the demographic variables are held constant at the citywide average. The horizontal line in each figure represents the citywide performance average across all schools and CFNs. The vertical lines display the confidence intervals of the predicted performance for each network, and the middle hollow dot is the point estimate within the predicted performance range of each network. The solid dots represent the actual average performance of each network. Network effects on academic performance measures are represented by the distances between the middle hollow dots and the horizontal line indicating average citywide performance. Positive network effects are indicated when each network’s hollow dot falls above the city average performance line. Negative network effects are indicated when each network’s hollow dot falls below the city average line. In Figure 12, for example, network 111 has a positive effect on its member school’s fourth grade ELA results, while network 610 has a negative effect. Both the predicted network effects on the fourth grade ELA, in Figure 12, and the network effects on fourth grade math proficiency rates, in Figure 13, are tightly clustered around the citywide average, FIGURE 13 Predicted Network Effects for Elementary Schools – Fourth Grade Math Proficiency Rates (2012) SOURCE NYCDOE. New York State math test results (2012). Annenberg Institute for School Reform 27 with no apparent positive or negative outliers. These results indicate that the CFNs have very little, if any, effect on fourth grade ELA and math proficiency rates. We found similar results in the 2010 and 2011 analyses we conducted. Predicted effects for networks’ eighth grade ELA proficiency rates, in Figure 14, are also very tightly dispersed around the citywide average. The results indicate very little, if any, network effects on eighth grade ELA proficiency and no significant positive or negative outlier CFNs. FIGURE 14 Predicted Network Effects for Middle Schools – Eighth Grade ELA Proficiency Rates (2012) SOURCE NYCDOE. New York State ELA test results (2012). 28 Demographics and Performance in New York City's School Networks: An Initial Inquiry Predicted outcomes for CFNs’ eighth grade math proficiency rates, in Figure 15, are slightly less tightly dispersed around the citywide average than predicted eighth grade ELA proficiency rates. More networks display negative effects, but there are no significant positive or negative outliers. FIGURE 15 Predicted Network Effects for Middle Schools – Eighth Grade Math Proficiency Rates (2012) SOURCE NYCDOE. New York State math test results (2012). Annenberg Institute for School Reform 29 The four-year graduation rate, Figure 16, shows higher overall predicted outcomes, with most networks’ performance falling above the citywide average. The pattern of predicted outcomes is similar to the actual pattern of four-year graduation rates, shown previously in Figure 8. FIGURE 16 Predicted Network Effects for High Schools – Four-year Graduation Rates (2012) SOURCE NYCDOE. Graduation Results, Cohort 2008 (Class of 2012). 30 Demographics and Performance in New York City's School Networks: An Initial Inquiry In Figure 17, the predicted outcomes of network APM/college readiness performance are more tightly clustered around the very low citywide average, compared to the more positive clustering above the citywide average of the four-year graduation rate predictions in the previous figure. These APM/college readiness outcomes indicate that the CFNs have very limited, if any, effects on APM at the high school level.27 27 Network 201 includes most of the school system’s specialized, testentry high schools, as well as many other highperforming, over-subscribed high schools. FIGURE 17 Predicted Network Effects for High Schools – Four-Year Aspirational Performance Measure/College Readiness Rates (2012) SOURCE NYCDOE. Regents-based math/ELA Aspirational Performance Measure, Cohort 2008 (Class of 2012). Annenberg Institute for School Reform 31 28 29 We carried out both HLM and regression analyses of the academic outcomes of the 2002 community school districts. The percent of variance in academic outcomes attributable to the community school districts in 2002 was much higher than the percent of variance in academic outcomes attributable to the networks. The findings of our 2002 regression analyses of the contribution of district demographics to academic outcomes were similar to our 2012 findings, confirming the preeminent role of student demographics in shaping both community school district and network academic outcomes (see Figures A17–A20 in the Appendix for the school performances of the school districts in 2002). A recent study by the Research Alliance for New York City Schools identified several successful small high schools, and found that their principals and teachers perceived their schools’ “partnerships with networks as relatively unimportant” to their schools’ effectiveness (Villavicencio & Marinell 2014, p. 14). 32 Summary of Academic Findings The results of our analyses of networks’ variance in academic outcomes suggest that network membership can predict academic outcomes at the school level, but not very robustly, and that student demographics predict academic performance far more strongly than network membership. Moreover, when key demographic variables such as poverty, the percentages of students with disabilities, and English language learners are held constant at the citywide average, network membership seems to have very little effect on member schools’ performance, except possibly for high school graduation rates. Almost all of our HLM analyses show predicted network performance tightly clustered around the citywide average, with neither positive nor negative outliers.28 Our analyses do not identify any “beat-the-odds” CFNs, meaning particularly effective networks whose student outcomes significantly transcend the norm of network performance.29 Our findings indicate that demography is still destiny, in terms of the networks’ overall contributions to schools’ academic outcomes. It is important to stress the limits of our work. Our analyses are neither growth-based nor value-added. Though both our HLM and regression analyses findings are consistent, and robust, for 2010, 2011, and 2012 school years, they do not assess growth in student academic outcomes within and across CFNs. Nor do they address the contributions of particular CFNs to student academic growth in individual schools. Our study does not examine the instructional or administrative practices of particular networks, or identify any specific characteristics of networks other than their demographics and academic outcomes. Our analysis is too coarse-grained to pick up small but important differences across networks, because our effort is essentially a bird’s-eye view of network contributions to student academic outcomes across the city school system. Demographics and Performance in New York City's School Networks: An Initial Inquiry Conclusions The networks are highly variable in terms of size (number of member schools), student enrollment, differing grade levels, and student demographics. Choice at the school level, usually by the principal, seems to determine network membership. Though we found considerable racial/ethnic segregation within and across the networks, the extent of that segregation at the elementary/K–8 and middle school levels decreased from the community school districts in 2002 to the CFNs in 2012. This reduction was most likely a result of the non-geographic nature of the CFNs. Our analyses indicate that network membership can predict academic outcomes at the school level, but not very robustly. We also found that student demographics are a far stronger predictor of school-level academic performance across the networks than network membership. When our analyses controlled for student demographics, we found that the networks had little effect on school-level academic outcomes, except possibly for the four-year high school cohort graduation rates. Demography still seems to determine destiny in the current, network-organized school system. Annenberg Institute for School Reform 33 References Ascher, C., and C. Maguire. 2007. Beating the Odds: How Thirteen NYC Schools Bring Low-Performing Ninth-Graders to Timely Graduation and College Enrollment. Providence, RI: Brown University, Annenberg Institute for School Reform. Childress, S., M. Higgins, A. Ishimaru, and S. Takahashi. 2011. “Managing for Results at the New York City Department of Education.” In Education Reform in New York City: Ambitious Change in the Nation's Most Complex School System, edited by J. A. O’Day, C. S. Bitter, and L. M. Gomez. Cambridge, MA: Harvard University Press. City Council, City of New York. 2012. Transcript of the Minutes of the Committee on Education, Oversight Hearing on New York City’s Networks for School Support (October 25). New York: Ubiqus. Available at http://legistar.council.nyc.gov/LegislationDetail. aspx?ID=1203370&GUID=2D2A5062-242F-4D80-85E5-33AC4E7EFBDF&Options= &Search=. Cromidas, Rachel. 2012. “Tensions Flare as Officials Defend their School Support Systems,” New York: Gotham Schools, http://ny.chalkbeat.org/2012/10/25/tension-flare-asofficials-defend-their-school-support-systems/#.VMU3y8ZeOfs. Decker, G., and P. Wall. 2013. “Fearing Change, Principals Lobby de Blasio to Protect Networks.” New York: Gotham Schools. Fessenden, Ford. 2012. “A Portrait of Segregation in New York City’s Schools,” New York Times, http://www.nytimes.com/interactive/2012/05/11/nyregion/segregation-in-newyork-city-public-schools html?ref=education. Fry, R., and P. Taylor. 2012. The Rise of Residential Segregation by Income. Social and Demographic Trends. Washington, DC: Pew Research Center, http://www. pewsocialtrends.org/files/2012/08/Rise-of-Residential-Income-Segregation-2012.2.pdf. Institute for Education and Social Policy. 2001. Final Report of the Evaluation of New York Networks for School Renewal. New York: Institute for Education and Social Policy, Steinhardt School of Education, New York University. Available at http://files.eric.ed.gov/fulltext/ED464164.pdf. Mishook, J., S. McAlister, and K. Edge. 2011. Peer Networks in School Reform: Lessons from England and Implications for the United States. Providence, RI: Brown University, Annenberg Institute for School Reform. Available at http://annenberginstitute.org/ ?q=publication/peer-networks-school-reform-lessons-england-and-implicationsunited-states. Nadelstern, Eric. 2012. The Evolution of School Support Networks in New York City. Working Paper 2012-2 of the Center on Reinventing Public Education. Seattle, WA: University of Washington Bothell. 34 Demographics and Performance in New York City's School Networks: An Initial Inquiry New York City Department of Education. n.d. School Support. New York: NYCDOE, http://schools.nyc.gov/AboutUs/schools/support/default.htm. http://schools.nyc.gov/AboutUs/schools/support/OSS.htm, and 4B52384E3262/153599/NetworkDirectory.pdf. New York City Independent Budget Office. 2014. As the System for Providing Support Services to Schools Changed, Funding Levels Changed, Too. New York: NYCIBO. Parthenon Group. 2013. An Assessment of the New York City Department of Education School Support Structure. Boston, MA: Parthenon Group. Populations Studies Center. n.d. “Calculation Formula for Segregation Measures, The Index of Dissimilarity.” Racial Residential Segregation Management Project. Michigan: University of Michigan, http://enceladus.isr.umich.edu/race/calculate.html Rothman, R. (Ed.). 2008. Voices in Urban Education no. 21 (Fall). Themed issue: “A Smart System in London.” http://vue.annenberginstitute.org/issues/21 Sebring, P. B., E. Allensworth, A. S. Bryk, J. Q. Easton, and S. Luppescu. 2006. The Essential Supports for School Improvement: Research Report. Chicago: Consortium on Chicago School Research. https://ccsr.uchicago.edu/publications/essential-supportsschool-improvement. Stiefel, L., R. Berne, P. Iatarola, and N. Fruchter. 2000. “High School Size: Effects on Budgets and Performance in New York City,” Educational Evaluation and Policy Analysis 22, no. 1 (Spring): 27–39. Villavicencio, A., and W. H. Marinell. 2014. Inside Success: Strategies of 25 Effective Small High Schools in NYC. New York: The Research Alliance for New York City Schools, Steinhardt School of Culture, Education, and Human Development, New York University. Wohlstetter, P., J. Smith, and A. Gallagher. 2013. “New York City’s Children First Networks: Turning Accountability on its Head,” Journal of Educational Administration 51, no. 4. Annenberg Institute for School Reform 35 Appendix Figure A1. Number of Schools by Network (2012) Figure A2. Student Enrollment by Network (2012) Figure A3. Number of Schools by Network by Different Grades Offered (2012) Figure A4. Percent of Schools by Size by Network (2012) Figure A5. Percent of Students by Race/Ethnicity by Network (2012) Figure A6. Percent of Students who are: Eligible for Free and Reduced Price Lunch, English Language Learners, and in Receipt of Special Education Services by Network (2012) Figure A7. Percent of Fourth Grade Students Scoring Proficient and Above in ELA and Math by Network (2012) Figure A8. Percent of Eighth Grade Students Scoring Proficient and Above in ELA and Math by Network (2012) Figure A9. Percent of Students Graduating in Four Years (2012) Figure A10. Percent of Students Graduating College Ready (2012) Figure A11. Regression Coefficient Estimates for Fourth 4th Grade ELA Proficiency Rates (2012) Figure A12. Regression Coefficient Estimates for Fourth Grade Math Proficiency Rates (2012) Figure A13. Regression Coefficient Estimates for Eighth Grade ELA Proficiency Rates (2012) Figure A14. Regression Coefficient Estimates for Eighth Grade Math Proficiency Rates (2012) Figure A15. Regression Coefficient Estimates for Four-Year Cohort Graduation Rates (2012) Figure A16. Regression Coefficient Estimates for Four-Year Aspirational Performance Measure/College Readiness Rates (2012) Figure A17. Elementary Schools – Fourth Grade ELA Proficiency Rates by Community School District (2002) Figure A18. Elementary Schools – Fourth Grade Math Proficiency Rates by Community School District (2002) Figure A19. Middle Schools – 8th Grade ELA Proficiency Rates by Community School District (2002) Figure A20. Middle Schools – Eighth Grade Math Proficiency Rates By Community School District (2002) 36 Demographics and Performance in New York City's School Networks: An Initial Inquiry FIGURE A1 Number of Schools by Network (2012) Network Schools (N) 30 N607 26 35 31 N403 25 N551 34 32 N536 25 4 N602 33 33 N605 25 5 N201 32 34 N405 24 6 N612 32 35 N411 24 7 N102 31 36 N561 24 8 N109 31 37 N562 24 9 N532 31 38 N603 24 10 N211 30 39 N103 23 11 N404 30 40 N408 23 12 N406 30 41 N563 23 13 N107 29 42 N106 22 14 N202 29 43 N207 21 15 N531 29 44 N401 21 16 N104 28 45 N105 20 17 N204 28 46 N111 20 18 N205 28 47 N410 20 19 N535 28 48 N412 20 20 N101 27 49 N606 20 21 N112 27 50 N609 20 22 N203 27 51 N208 19 23 N533 27 52 N209 19 24 N534 27 53 N402 18 25 N608 27 54 N511 18 26 N611 27 55 N521 18 27 N108 26 56 N206 17 28 N210 26 57 N407 16 29 N604 26 TOTAL 1,452 Network Schools (N) 1 N610 38 2 N409 3 SOURCE NYCDOE, Office of Organizational Data. Location Code Generation and Management System [LCGMS] schoollevel database. Annenberg Institute for School Reform 37 FIGURE A2 Student Enrollment by Network (2012) Network Student Enrollment (N) 1 N201 49,567 2 N405 3 Network Student Enrollment (N) 30 N561 15,527 42,033 31 N410 15,035 N202 34,280 32 N534 15,012 4 N602 25,930 33 N112 14,883 5 N204 25,191 34 N108 14,247 6 N409 24,075 35 N608 13,851 7 N535 22,107 36 N401 13,462 8 N104 21,066 37 N603 13,408 9 N532 21,036 38 N209 13,304 10 N531 20,987 39 N107 13,127 11 N611 20,923 40 N203 13,095 12 N604 20,671 41 N111 12,122 13 N210 20,535 42 N521 11,869 14 N208 19,402 43 N606 11,847 15 N109 19,243 44 N408 11,832 16 N533 18,863 45 N404 11,754 17 N551 18,368 46 N412 11,249 18 N605 17,707 47 N103 11,139 19 N207 17,700 48 N101 10,396 20 N609 17,454 49 N403 10,305 21 N612 17,352 50 N206 10,263 22 N205 16,953 51 N562 10,261 23 N610 16,890 52 N407 10,052 24 N607 16,530 53 N402 9,483 25 N406 16,463 54 N411 8,855 26 N211 16,268 55 N106 8,188 27 N102 15,867 56 N105 8,126 28 N563 15,773 57 N511 7,627 29 N536 15,584 TOTAL 955,137 SOURCE NYCDOE. School Demographics and Accountability Snapshot (CEP 2012). 38 Demographics and Performance in New York City's School Networks: An Initial Inquiry FIGURE A3 Number of Schools by Network by Different Grades Offered (2012) (Each school may offer more than one type of grade pattern) Network ELEM. and K–8 (N) MIDD. Transfer (N) HS. (N) HS. (N) Network ELEM. and K–8 (N) N101 0 23 7 N102 18 5 N103 11 N104 MIDD. Transfer (N) HS. (N) HS. (N) 0 N408 21 1 1 0 7 3 N409 33 2 1 0 7 5 0 N410 18 2 0 0 19 9 2 0 N411 1 12 13 0 N105 0 9 18 0 N412 18 2 0 0 N106 0 2 21 1 N511 4 12 2 1 N107 0 4 18 7 N521 0 10 16 0 N108 5 1 5 16 N531 29 0 0 0 N109 27 4 0 0 N532 20 11 0 0 N111 12 8 0 0 N533 19 8 0 0 N112 1 16 16 1 N534 15 12 0 0 N201 1 1 32 0 N535 17 11 0 0 N202 16 0 13 0 N536 1 14 16 0 N203 26 1 0 0 N551 21 7 8 0 N204 23 5 0 0 N561 2 8 18 3 N205 27 1 0 0 N562 1 4 20 3 N206 10 1 7 0 N563 1 2 17 3 N207 21 0 0 0 N602 19 14 0 0 N208 4 15 1 0 N603 0 7 22 2 N209 18 1 0 0 N604 21 5 0 0 N210 18 8 0 0 N605 21 3 1 0 N211 15 12 5 0 N606 19 1 0 0 N401 19 2 0 0 N607 22 4 0 0 N402 0 6 18 0 N608 5 22 0 0 N403 0 2 16 7 N609 12 8 0 0 N404 0 1 30 0 N610 8 8 22 1 N405 1 5 20 0 N611 7 6 20 0 N406 26 4 0 0 N612 32 0 0 0 N407 12 4 0 0 SOURCE NYCDOE. Progress Reports, 2011-2012. Annenberg Institute for School Reform 39 FIGURE A4 Percent of Schools by Size by Network (2012) (Networks sorted by the percentages of large size Schools) Network Large Sch. Medium Sch. (>1000) (500-1000) Small Sch. (0-499) Network Large Sch. Medium Sch. (>1000) (500-1000) 1 N405 54% 21% 25% 30 N209 11% 74% 16% 2 N201 53% 9% 38% 31 N561 8% 33% 58% 3 N208 53% 42% 5% 32 N610 8% 24% 68% 4 N202 48% 34% 17% 33 N534 7% 37% 56% 5 N609 40% 35% 25% 34 N608 7% 30% 63% 6 N204 36% 43% 21% 35 N112 7% 19% 74% 7 N104 25% 43% 32% 36 N101 7% 7% 85% 8 N410 25% 40% 35% 37 N107 7% 31% 62% 9 N602 24% 55% 21% 38 N404 7% 10% 83% 10 N531 24% 48% 28% 39 N406 7% 37% 57% 11 N101 23% 46% 31% 40 N102 6% 32% 61% 12 N201 19% 65% 15% 41 N206 6% 53% 41% 13 N207 19% 76% 5% 42 N551 6% 53% 41% 14 N401 19% 43% 38% 43 N402 6% 33% 61% 15 N533 19% 56% 26% 44 N511 6% 22% 72% 16 N611 19% 52% 30% 45 N606 5% 55% 40% 17 N563 18% 5% 77% 46 N103 4% 39% 57% 18 N535 18% 54% 29% 47 N603 4% 29% 67% 19 N111 15% 40% 45% 48 N205 0% 82% 18% 20 N532 13% 55% 32% 49 N412 0% 65% 35% 21 N109 13% 48% 39% 50 N211 0% 60% 40% 22 N407 13% 56% 31% 51 N408 0% 52% 48% 23 N612 13% 31% 56% 52 N203 0% 48% 52% 24 N605 12% 64% 24% 53 N403 0% 28% 72% 25 N536 12% 32% 56% 54 N562 0% 25% 75% 26 N607 12% 54% 35% 55 N105 0% 20% 80% 27 N108 12% 27% 62% 56 N411 0% 13% 88% 28 N409 11% 60% 29% 57 N106 0% 5% 95% 29 N521 11% 28% 61% SOURCE NYCDOE. School Demographics and Accountability Snapshot (CEP 2012). 40 Small Sch. (0-499) Demographics and Performance in New York City's School Networks: An Initial Inquiry FIGURE A5 Percent of Students by Race/Ethnicity by Network (2012) (Networks sorted by the percentages of White students) Network White Asian Black Latino Network White Asian Black Latino 1 N604 55% 9% 12% 23% 30 N111 10% 11% 49% 30% 2 N409 41% 20% 7% 31% 31 N521 9% 24% 34% 32% 3 N405 37% 23% 17% 22% 32 N611 9% 9% 53% 29% 4 N103 36% 20% 19% 22% 33 N607 8% 7% 27% 57% 5 N605 35% 22% 23% 20% 34 N209 8% 18% 21% 52% 6 N533 32% 13% 34% 21% 35 N408 8% 5% 35% 51% 7 N102 27% 11% 18% 42% 36 N112 7% 8% 48% 36% 8 N602 26% 19% 40% 14% 37 N106 7% 16% 22% 55% 9 N203 25% 16% 16% 40% 38 N104 7% 7% 21% 65% 10 N609 24% 33% 11% 32% 39 N412 6% 3% 19% 71% 11 N101 23% 22% 17% 36% 40 N402 5% 3% 43% 48% 12 N201 19% 38% 15% 27% 41 N551 5% 5% 39% 50% 13 N207 19% 37% 10% 32% 42 N401 5% 2% 52% 40% 14 N206 18% 24% 19% 38% 43 N108 4% 22% 39% 33% 15 N204 17% 26% 6% 51% 44 N211 4% 8% 55% 32% 16 N107 17% 14% 21% 47% 45 N603 4% 4% 41% 50% 17 N205 15% 37% 34% 13% 46 N404 4% 6% 46% 43% 18 N210 15% 18% 27% 39% 47 N531 4% 20% 45% 29% 19 N563 15% 10% 44% 29% 48 N536 3% 11% 41% 43% 20 N410 14% 19% 16% 50% 49 N608 3% 5% 36% 56% 21 N612 14% 13% 34% 39% 50 N109 3% 3% 24% 69% 22 N406 13% 16% 31% 40% 51 N105 3% 4% 41% 51% 23 N202 12% 17% 17% 54% 52 N534 3% 8% 40% 49% 24 N208 11% 25% 15% 48% 53 N610 3% 4% 45% 49% 25 N535 10% 27% 30% 31% 54 N606 2% 7% 31% 59% 26 N411 10% 5% 30% 54% 55 N511 2% 3% 36% 59% 27 N532 10% 4% 14% 71% 56 N562 2% 3% 36% 60% 28 N403 10% 10% 39% 40% 57 N407 2% 3% 26% 69% 29 N561 10% 26% 26% 37% City Average 15% 16% 28% 40% NOTE Bold percentages represent the highest thirty-third percentile; black percentages represent the middle thirty-third percentile; gray percentages represent the lowest thirty-third percentile. SOURCE NYCDOE. School Demographics and Accountability Snapshot (CEP 2012). Annenberg Institute for School Reform 41 FIGURE A6 Percent of Students who are: Eligible for Free and Reduced Price Lunch, English Language Learners, and in Receipt of Special Education Services by Network (2012) (Networks sorted by the percentages of FRPL) Network Total Enroll. FRPL ELL SPED Network Total Enroll. FRPL ELL SPED 1 N407 10,052 88% 22% 14% 30 N112 14,883 69% 11% 16% 2 N412 11,249 86% 20% 14% 31 N202 34,280 69% 20% 12% 3 N109 19,243 85% 24% 17% 32 N402 9,483 69% 18% 15% 4 N606 11,847 82% 17% 15% 33 N404 11,754 68% 10% 17% 5 N209 13,304 81% 22% 13% 34 N536 15,584 68% 10% 15% 6 N511 7,627 80% 17% 18% 35 N403 10,305 67% 11% 13% 7 N608 13,851 80% 14% 18% 36 N611 20,923 67% 9% 15% 8 N401 13,462 80% 12% 15% 37 N204 25,191 66% 21% 14% 9 N534 15,012 80% 15% 16% 38 N602 25,930 66% 13% 13% 10 N607 16,530 78% 14% 15% 39 N108 14,247 66% 12% 10% 11 N408 11,832 78% 14% 16% 40 N605 17,707 65% 14% 14% 12 N532 21,036 78% 25% 15% 41 N563 15,773 64% 8% 17% 13 N104 21,066 78% 21% 15% 42 N206 10,263 64% 14% 15% 14 N551 18,368 77% 18% 15% 43 N535 22,107 63% 9% 13% 15 N610 16,890 76% 15% 20% 44 N561 15,527 61% 12% 12% 16 N111 12,122 76% 9% 15% 45 N207 17,700 61% 19% 11% 17 N562 10,261 75% 13% 16% 46 N107 13,127 61% 15% 12% 18 N106 8,188 75% 51% 8% 47 N521 11,869 60% 9% 12% 19 N410 15,035 75% 21% 12% 48 N533 18,863 59% 9% 16% 20 N531 20,987 74% 10% 11% 49 N203 13,095 59% 14% 17% 21 N612 17,352 73% 18% 11% 50 N409 24,075 58% 15% 16% 22 N105 8,126 73% 8% 18% 51 N102 15,867 55% 14% 16% 23 N208 19,402 73% 19% 14% 52 N101 10,396 54% 8% 17% 24 N406 16,463 72% 19% 15% 53 N405 42,033 46% 11% 12% 25 N211 16,268 72% 9% 15% 54 N604 20,671 46% 8% 18% 26 N411 8,855 71% 13% 17% 55 N201 49,567 45% 8% 7% 27 N609 17,454 70% 20% 14% 56 N205 16,953 45% 8% 11% 28 N210 20,535 70% 10% 14% 57 N103 11,139 38% 9% 15% 29 N603 13,408 70% 9% 20% City Average 67% 14% 14% SOURCE NYCDOE. School Demographics and Accountability Snapshot (CEP 2012). 42 Demographics and Performance in New York City's School Networks: An Initial Inquiry FIGURE A7 Percent of Fourth Grade Students Scoring Proficient and Above in ELA and Math by Network (2012) Elementary and K–8 Network Schools (N) Gr.4 ELA Gr.4 Math Network Elementary and K–8 Schools (N) Gr.4 ELA Gr.4 Math N201 1 97% 100% N406 26 47% 64% N103 11 85% 94% N531 29 52% 63% N411 1 79% 93% N412 18 49% 62% N206 10 74% 88% N209 18 48% 62% N405 1 64% 84% N532 20 48% 62% N207 21 67% 84% N607 22 48% 62% N561 2 59% 82% N208 4 48% 61% N112 1 74% 81% N211 15 51% 61% N409 33 66% 80% N104 19 43% 59% N205 27 69% 78% N562 1 46% 58% N604 21 67% 77% N536 1 38% 56% N102 18 65% 77% N606 19 41% 56% N602 19 63% 76% N111 12 46% 56% N204 23 57% 76% N551 21 45% 55% N410 18 58% 74% N608 5 37% 54% N605 21 62% 73% N511 4 44% 54% N203 26 57% 71% N401 19 40% 53% N609 12 55% 70% N109 27 36% 53% N535 17 59% 69% N534 15 41% 52% N202 16 53% 69% N407 12 39% 50% N533 19 58% 68% N611 7 26% 41% N612 32 52% 66% N610 8 27% 36% N210 18 54% 66% N563 1 34% 23% N108 5 53% 66% City Average 54% 67% N408 21 48% 64% NOTE City average weighted by the number of students tested. SOURCE NYCDOE. New York State ELA and math tests results (2012). Annenberg Institute for School Reform 43 FIGURE A8 Percent of Eighth Grade Students Scoring Proficient and Above in ELA and Math by Network (2012) Middle Network Schools (N) Gr.8 ELA Gr.8 Math Network Middle Schools (N) Gr.8 ELA Gr.8 Math N201 1 96% 98% N104 9 31% 51% N108 1 87% 94% N112 16 32% 49% N205 1 83% 90% N611 6 36% 49% N106 2 54% 85% N606 1 35% 48% N404 1 76% 81% N401 2 31% 47% N409 2 65% 80% N403 2 32% 46% N605 3 59% 79% N511 12 25% 46% N405 5 73% 77% N408 1 19% 46% N204 5 51% 74% N532 11 24% 45% N609 8 48% 74% N608 22 28% 43% N101 23 57% 70% N402 6 33% 43% N111 8 46% 70% N105 9 31% 43% N521 10 48% 69% N551 7 28% 41% N102 5 57% 67% N411 12 31% 40% N103 7 44% 65% N562 4 29% 40% N107 4 45% 65% N109 4 21% 38% N535 11 49% 64% N406 4 23% 38% N208 15 41% 63% N603 7 28% 37% N206 1 24% 62% N211 12 27% 37% N602 14 46% 62% N407 4 20% 30% N534 12 38% 59% N412 2 22% 29% N604 5 49% 59% N563 2 16% 26% N604 8 41% 57% N203 1 26% 26% N210 8 41% 56% N607 4 14% 25% N561 8 42% 55% N610 8 17% 22% N410 2 29% 53% City Average 40% 57% N536 14 33% 53% NOTE City average weighted by the number of students tested. SOURCE NYCDOE. New York State ELA and math tests results (2012). 44 Demographics and Performance in New York City's School Networks: An Initial Inquiry FIGURE A9 FIGURE A10 Percent of Students Graduating in Four Years (2012) Percent of Students Graduating College Ready (2012) High Network Schools (N) High Network Schools (N) Grad. Rate Grad. Rate 1 N605 1 88% 1 N201 32 49% 2 N409 1 86% 2 N101 7 41% 3 N211 5 84% 3 N521 16 37% 4 N201 32 79% 4 N405 20 35% 5 N104 2 78% 5 N561 18 30% 6 N521 16 77% 6 N409 1 28% 7 N561 18 76% 7 N211 5 28% 8 N107 18 74% 8 N104 2 24% 9 N101 7 73% 9 N107 18 24% 10 N405 20 72% 10 N536 16 24% 11 N103 5 72% 11 N206 7 22% 12 N536 16 71% 12 N202 13 15% 13 N206 7 71% 13 N551 8 15% 14 N105 18 70% 14 N403 16 14% 15 N403 16 69% 15 N611 20 14% 16 N108 5 66% 16 N562 20 14% 17 N411 13 65% 17 N108 5 13% 18 N408 1 64% 18 N103 5 13% 19 N611 20 64% 19 N105 18 12% 20 N404 30 64% 20 N411 13 12% 21 N563 17 64% 21 N563 17 12% 22 N102 7 64% 22 N106 21 12% 23 N562 20 63% 23 N112 16 12% 24 N551 8 62% 24 N605 1 10% 25 N106 21 62% 25 N408 1 10% 26 N202 13 60% 26 N603 22 9% 27 N112 16 59% 27 N402 18 7% 28 N511 2 56% 28 N404 30 7% 29 N402 18 56% 29 N511 2 5% 30 N603 22 55% 30 N610 22 3% 31 N610 22 41% City Average 23% City Average 67% NOTE City averages weighted by the number of students in the 2008 cohort. SOURCES NYCDOE. Graduation Results, Cohort 2008 (Class of 2012). NYCDOE. Regents-based math/ELA Aspirational Performance Measure, Cohort 2008 (Class of 2012). Annenberg Institute for School Reform 45 FIGURE A11 Regression Coefficient Estimates for Fourth Grade ELA Proficiency Rates (2012) NOTE The gray lines represent the confidence intervals of the coefficients. The gray dots represent the coefficient estimates. The black dots are the standardized coefficient estimates. SOURCES NYCDOE. School Demographics and Accountability Snapshot (CEP 2012). NYCDOE. Progress Reports, 2011-2012. 46 Demographics and Performance in New York City's School Networks: An Initial Inquiry FIGURE A12 Regression Coefficient Estimates for Fourth Grade Math Proficiency Rates (2012) NOTE The gray lines represent the confidence intervals of the coefficients. The gray dots represent the coefficient estimates. The black dots are the standardized coefficient estimates. SOURCES NYCDOE. School Demographics and Accountability Snapshot (CEP 2012). NYCDOE. Progress Reports, 2011-2012. Annenberg Institute for School Reform 47 FIGURE A13 Regression Coefficient Estimates for Eighth Grade ELA Proficiency Rates (2012) NOTE The Fourth Grade Math/ELA Index was rescaled to 100 so that the regression coefficients are on a similar scale as the rest of the coefficients. The gray lines represent the confidence intervals of the coefficients. The gray dots represent the coefficient estimates. The black dots are the standardized coefficient estimates. SOURCES NYCDOE. School Demographics and Accountability Snapshot (CEP 2012). NYCDOE. Progress Reports, 2011-2012. 48 Demographics and Performance in New York City's School Networks: An Initial Inquiry FIGURE A14 Regression Coefficient Estimates for Eighth Grade Math Proficiency Rates (2012) NOTE The Fourth Grade Math/ELA Index was rescaled to 100 so that the regression coefficients are on a similar scale as the rest of the coefficients. The gray lines represent the confidence intervals of the coefficients. The gray dots represent the coefficient estimates. The black dots are the standardized coefficient estimates. SOURCES NYCDOE. School Demographics and Accountability Snapshot (CEP 2012). NYCDOE. Progress Reports, 2011-2012. Annenberg Institute for School Reform 49 FIGURE A15 Regression Coefficient Estimates for Four-Year Cohort Graduation Rates (2012) NOTE The Eighth Grade Math/ELA Index was rescaled to 100 so that the regression coefficients are on a similar scale as the rest of the coefficients. The gray lines represent the confidence intervals of the coefficients. The gray dots represent the coefficient estimates. The black dots are the standardized coefficient estimates. SOURCES NYCDOE. School Demographics and Accountability Snapshot (CEP 2012). NYCDOE. Progress Reports, 2011-2012. 50 Demographics and Performance in New York City's School Networks: An Initial Inquiry FIGURE A16 Regression Coefficient Estimates for Four-Year Aspirational Performance Measure/College Readiness Rates (2012) NOTE The Eighth Grade Math/ELA Index was rescaled to 100 so that the regression coefficients are on a similar scale as the rest of the coefficients. The gray lines represent the confidence intervals of the coefficients. The gray dots represent the coefficient estimates. The black dots are the standardized coefficient estimates. SOURCES NYCDOE. School Demographics and Accountability Snapshot (CEP 2012). NYCDOE. Progress Reports, 2011-2012. Annenberg Institute for School Reform 51 FIGURE A17 Elementary Schools – Fourth Grade ELA Proficiency Rates by Community School District (2002) NOTE Community School District numbers are displayed across the bottom of the graph. SOURCE NYSED. New York State Report Cards, 2002. 52 Demographics and Performance in New York City's School Networks: An Initial Inquiry FIGURE A18 Elementary Schools – Fourth Grade Math Proficiency Rates by Community School District (2002) NOTE Community School District numbers are displayed across the bottom of the graph. SOURCE NYSED. New York State Report Cards, 2002. Annenberg Institute for School Reform 53 FIGURE A19 Middle Schools – Eighth Grade ELA Proficiency Rates by Community School District (2002) NOTE Community School District numbers are displayed across the bottom of the graph. SOURCE NYSED. New York State Report Cards, 2002. 54 Demographics and Performance in New York City's School Networks: An Initial Inquiry FIGURE A20 Middle Schools – Eighth Grade Math Proficiency Rates By Community School District (2002) NOTE Community School District numbers are displayed across the bottom of the graph. SOURCE NYSED. New York State Report Cards, 2002. Annenberg Institute for School Reform 55