United States Government Accountability Office Report to the Ranking Member, Subcommittee on Health, Employment, Labor, and Pensions, Committee on Education and the Workforce, House of Representatives July 2015 TEACHER PREPARATION PROGRAMS Education Should Ensure States Identify Low-Performing Programs and Improve InformationSharing GAO-15-598 July 2015 TEACHER PREPARATION PROGRAMS Highlights of GAO-15-598, a report to the Ranking Member, Subcommittee on Health, Employment, Labor, and Pensions, Committee on Education and the Workforce, House of Representatives Education Should Ensure States Identify LowPerforming Programs and Improve InformationSharing Why GAO Did This Study What GAO Found TPPs play a vital role in preparing teachers, including helping them teach to new K-12 college- and career-ready standards recently adopted or under development in all states. Under Title II of the Higher Education Act, states collect information on TPPs and report it to Education, which reports it to the public. Education also administers competitive grant programs related to teacher preparation. In light of new K12 standards and questions about TPP quality, GAO was asked to review TPP, state, and federal efforts. State oversight officials reported that they approve teacher preparation programs (TPP) by assessing the quality of program design and analyzing candidate data such as program graduation rates, according to GAO’s 2014-2015 survey of states and the District of Columbia. However, some states reported that they do not assess whether TPPs are low-performing, as required by federal law. To receive funding under the Higher Education Act, states are required to conduct an assessment to identify TPPs that are low-performing. Seven states reported to GAO that they do not have a process to do so. State officials who reported not having a process in GAO’s survey cited several reasons including that they believed other oversight procedures were sufficient to ensure quality. Education officials told GAO they have not verified states’ processes to identify lowperforming TPPs. In accordance with federal internal control standards, Education should provide reasonable assurance of compliance with applicable laws. If states fail to assess whether TPPs are low-performing, potential teaching candidates may have difficulty identifying low-performing TPPs. This could result in teachers who are not fully prepared to educate children. This report examines: (1) state oversight activities, (2) state and TPP actions related to new K-12 standards, and (3) the extent to which Education shares information about TPP quality. GAO reviewed relevant federal laws and documents, surveyed all state oversight offices (with a 100 percent response rate), and interviewed Education officials and various stakeholders, as well as a nongeneralizable sample of officials in five states with varied approaches to oversight and 14 TPPs in those states. What GAO Recommends Among other things, GAO recommends that Education monitor states to ensure their compliance with requirements to assess whether any TPPs are low-performing and develop mechanisms to share information about TPP quality within the agency and with states. Education agreed with our recommendations. View GAO-15-598. To view the related esupplement, click: GAO-15-599SP. For more information, contact Melissa Emrey-Arras at (617) 788-0534 or EmreyArrasM@gao.gov. Officials in most surveyed states and all 14 of the TPPs GAO interviewed reported making changes to prepare teaching candidates for new state K-12 standards. Thirty-seven states reported providing TPPs with guidance about the new standards and a similar number of states reported adjusting their process for approving TPPs. Most states also required prospective teachers to pass licensing tests that have been modified in response to the new standards. Officials from all of the 14 TPPs GAO interviewed reported making changes that generally fell within the following three categories: (1) increasing subject-matter knowledge of teachers, (2) modifying coursework related to teaching techniques, and (3) using classroom training to provide real world experience. Education missed opportunities to share information about TPP quality internally and with state oversight entities. Federal internal controls standards highlight the value of effective information-sharing with internal and external stakeholders. However, Education does not have mechanisms in place to promote regular, sustained information-sharing among its various program offices that support TPP quality because the workgroup that used to facilitate such informationsharing was discontinued. Without such a mechanism, Education cannot fully leverage information about TPP quality gathered by its various programs. Furthermore, Education’s current efforts to share information about TPP quality with states only reach about a third of states, according to GAO’s survey, although about half of all states reported that they wanted more of such information. Gaps in the agency’s efforts to disseminate information result from information-sharing being left to individual offices’ initiative rather than an agency-wide mechanism. Education officials acknowledged that more could be done to share information with states and other stakeholders. Without such efforts, Education may miss opportunities to support state efforts to improve TPP quality. For example, states may be unaware of good practices identified by Education that could assist them in their oversight. United States Government Accountability Office Contents Letter 1 Background States Assessed TPPs before Approving Them to Prepare Teachers but Some States Reported Not Having a Process to Identify Low-Performing TPPs as Required by Statute Most States Provided Some Support and Oversight As TPPs Modified Their Programs Based on New K-12 Standards Education Missed Opportunities to Support TPP Improvement through Its Data Reporting and Information-Sharing Conclusions Recommendations for Executive Action Agency Comments and Our Evaluation 4 24 32 32 33 Appendix I Objectives, Scope, and Methodology 34 Appendix II Comments from the Department of Education 37 Appendix III GAO Contact and Acknowledgments 40 11 20 Tables Table 1: Examples of Teacher Preparation Program (TPP) Structures Table 2: Summary of Education’s Title II Reporting Templates for States and Institutions of Higher Education (i.e. Colleges and Universities) that Offer Teacher Preparation Programs (TPP) Table 3: Differences in the Sources of Information States Reported Using to Assess Traditional versus Alternative Route Teacher Preparation Programs (TPP) Seeking Approval 16 Figure 1. Information about Teaching Candidates that States Reported Using to Assess Traditional Teacher Preparation Programs (TPP) Seeking Approval 13 5 8 Figures Page i GAO-15-598 Teacher Preparation Programs Figure 2: Information about Graduate Effectiveness that States Reported using to Assess Traditional Teacher Preparation Programs (TPP) Seeking Approval Figure 3: State Reports of Time Spent on Higher Education Act Title II Reports during Academic Year 2013-2014 14 25 Abbreviations CAEP Education Higher Education Act INTASC TPP Council for the Accreditation of Educator Preparation U.S. Department of Education Higher Education Act of 1965, as amended Interstate Teacher Assessment and Support Consortium Teacher preparation program This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Page ii GAO-15-598 Teacher Preparation Programs Letter 441 G St. N.W. Washington, DC 20548 July 23, 2015 The Honorable Jared Polis Ranking Member Subcommittee on Health, Employment, Labor, and Pensions Committee on Education and the Workforce House of Representatives Dear Mr. Polis: Teacher preparation programs (TPP) play a vital role in helping ready the nation’s K-12 teachers for the classroom. The role of TPPs is especially important as all states begin using or developing college- and careerready standards for math and English that require more complex subjectmatter understanding from K-12 students and their teachers than many previous state standards. According to data from the U.S. Department of Education (Education), approximately 500,000 teaching candidates were enrolled in over 2,000 TPPs during the 2012-2013 academic year. 1 Further, the federal government makes a substantial investment in preparing teachers. Students who completed bachelor’s degree requirements between July 1, 2007 and June 30, 2008 and were teaching or considering teaching by June 2009 received an estimated $12.7 billion 1 These estimates are based on the most recent data reported by states under Title II of the Higher Education Act of 1965, as amended, as presented on Education’s website. See https://title2.ed.gov/Public/Home.aspx. Providers of TPPs may be institutions of higher education, such as colleges and universities, or other entities, such as non-profit organizations or public K-12 school districts. The 2,000 TPPs represent the number of providers that offer preparation programs for new teachers. Such providers may offer subprograms that specialize in a given area such as elementary education or high school math. The Title II data estimate that the 2,000 TPP providers offered more than 26,000 sub-programs in academic year 2013-2014. Page 1 GAO-15-598 Teacher Preparation Programs in Federal Pell Grants and federal student loans for their undergraduate and graduate education. 2 States have the primary responsibility for overseeing TPPs, including defining the types of TPPs that may operate in a given state and reviewing and approving TPP operations. States must also comply with federal reporting requirements under Title II of the Higher Education Act of 1965, as amended (Higher Education Act), which include assessing whether any TPPs are low-performing and reporting a list of lowperforming and at-risk TPPs to Education and the public. 3 Education plays an indirect role in supporting TPPs through various methods. In particular, Education collects and disseminates the data reported by states about TPPs, such as the number of teaching candidates enrolled in TPPs, pass rates for the tests candidates are required to take to be licensed to teach, and states’ procedures for identifying low-performing TPPs. Education also administers several competitive grant programs that fund TPP reforms, such as the Teacher Quality Partnership Grant Program. Given the importance of TPPs for ensuring that new teachers are ready to teach millions of public school children as states adopt new K-12 standards, and the amount of federal aid that supports teacher 2 These estimates are based on GAO analysis of the data from U.S. Department of Education, National Center for Education Statistics, 2008/09 Baccalaureate and Beyond Longitudinal Study. Individuals included in the sample completed requirements for a bachelor’s degree between July 1, 2007 and June 30, 2008 and may have also enrolled in graduate training programs related to teaching or other professions between 2008 and 2012. They were interviewed between July 2008 and June 2009 regarding whether they had taught, were preparing to teach, were considering teaching, or had no plans to teach. The study also tracked the amount of federal student loans the individuals received through 2012 and Federal Pell Grant funding through 2008, which may have been used to pay for undergraduate TPPs, graduate TPPs, or other types of degrees offered by colleges or universities. TPPs that do not participate in federal financial aid programs— which may include those operated by an entity that is not an institution of higher education (i.e., college or university)—would not be included in these estimates. 3 Title II of the Higher Education Act, as amended by the Higher Education Opportunity Act, is codified at 20 U.S.C. §§ 1021-1041. Each institution of higher education that conducts a TPP and enrolls students receiving federal student aid is required to annually report certain information to the state and the general public, and each state that receives funds under the Higher Education Act must report annually to Education and the general public certain information on the TPPs in the state. See 20 U.S.C. §§ 1022d-1022f for state and institutional reporting requirements. For the purposes of this report, we refer to these requirements collectively as Higher Education Act Title II reporting requirements. Page 2 GAO-15-598 Teacher Preparation Programs candidates attending many of these programs, you and Representative George Miller, in his role as Ranking Member of the House Committee on Education and the Workforce, asked us to review oversight and support provided to TPPs. This report examines: 1. how states oversee TPPs; 2. what states and select TPPs are doing to prepare new teachers for new K-12 standards; and, 3. the extent to which Education’s data reporting and other informationsharing support and encourage high quality TPPs. 4 To obtain information about all three objectives, we conducted a survey of all 50 states and the District of Columbia, obtaining a 100 percent response rate. The surveys and a more complete tabulation of the results can be viewed at GAO-15-599SP. To obtain more detailed examples about states’ and TPPs’ actions and experiences, we conducted case studies of five states: Arizona, California, Maine, Tennessee and Virginia. We selected these states because they represent a variety of sizes (i.e., numbers of TPPs operating in the state) and approaches to overseeing TPPs (including how they use data about graduate effectiveness to evaluate TPP performance), and to reflect different types of college- and career-ready K-12 standards. For each case study state, we reviewed documents and interviewed officials from the state agency or agencies that oversee TPPs, two to three TPPs, and at least one K-12 school district. In total, we interviewed 14 TPPs in our 5 case study states. We ensured that this non-generalizable sample varied in a number of ways, including whether they were located within or outside colleges and universities; were alternative route or traditional; and offered in-person or online coursework. We also interviewed U.S. Department of Education officials and contractors, teacher preparation researchers, organizations that represent TPPs, companies that test teaching candidates for state licensure, and other stakeholders. In addition, we reviewed relevant federal laws, regulations, and other documents; analyzed data collected pursuant to the Higher Education Act Title II reporting requirements; and compared Education’s actions to federal standards for internal controls and leading practices for data-driven program management. We also reviewed the reliability of select data fields of the Higher Education Act 4 This report focuses on TPPs that prepare teachers for an initial teaching license, and not other types, such as advanced licenses. Page 3 GAO-15-598 Teacher Preparation Programs Title II reports published by Education regarding the number of TPPs that states reported identifying as low-performing or at risk of being lowperforming by: (1) reviewing agency documents, (2) interviewing relevant agency officials, and (3) testing the accuracy of these reporting fields for a non-generalizable sample of states. We determined that Education’s compilation of state data about the number of low-performing TPPs was sufficiently reliable to include in our report. We conducted our work from March 2014 to July 2015 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background Structure of TPPs TPPs prepare teaching candidates to employ effective teaching techniques and gain real-world experience in the classroom. TPPs take many forms and may be operated by a variety of organizations (see table 1). For example, the structure of TPPs can vary widely, from “traditional” TPPs such as four-year undergraduate programs with student teaching requirements, to “alternative route” TPPs such as those wherein candidates serve as a classroom teacher while concurrently completing their coursework. Page 4 GAO-15-598 Teacher Preparation Programs Table 1: Examples of Teacher Preparation Program (TPP) Structures Types of TPP structures Examples of variation in TPP structures Organization administering the TPP • • • • “Traditional” TPP offered by a college or university (public, non-profit, or for-profit); “Alternative route” TPP offered by a college or university (public, non-profit, or for-profit); “Alternative route” TPP offered by an organization operated outside of a college or university; or “Alternative route” TPP offered by a public K-12 school district. Specialization • TPPs may be composed of multiple sub-programs that focus on a specific area of teaching such as elementary general education or high school mathematics. Length and timing of TPP • 4 year undergraduate program; 2 year post-baccalaureate program; 1 or 2 year master’s program; or Summer institute with periodic follow-up courses. Subject-matter courses taught in subject-specific departments of a college or university (e.g. the math department) or Candidate required to demonstrate subject-matter knowledge prior to TPP enrollment, through previous coursework or content examinations. • • • Method of providing subjectmatter content • • Structure and content of classroom training • • Format • • • “Student teaching” where a candidate teaches K-12 students for all or part of the day with support from the official teacher (i.e., ”teacher of record”) and TPP supervisors, or Serving as the “teacher of record” with support from TPP staff. All coursework and supervision of classroom training conducted in person; Most coursework online with an in-person supervisor for the classroom training; or Most or all coursework online, with primary classroom training supervision occurring via videorecording or other electronic communication. Source: GAO analysis of Education’s 2013 Higher Education Act Title II report and documents and interviews from select teacher preparation programs.   GAO-15-598 Note: This table illustrates the variation among TPPs and is not intended to be exhaustive. State Oversight State oversight responsibilities related to TPPs may be held by one or more state agencies, including the state department of education, the state board of education, or a state independent standards board. States have discretion in how they conduct oversight of TPP quality, by, for example: • defining the types of TPPs that may operate in the state, such as undergraduate or post-baccalaureate TPPs, or alternative route TPPs; • reviewing and approving individual TPPs to operate and periodically assessing them for renewal; Page 5 GAO-15-598 Teacher Preparation Programs College- and CareerReady Standards for K-12 Students • assessing whether any TPPs in the state are low-performing, as required under the Higher Education Act, using criteria of the state’s choosing; 5 and, • setting licensing requirements that teaching candidates must satisfy that often include completing an approved TPP and passing licensing tests that assess subject-matter knowledge or other skills. 6 States are also responsible for adopting academic content standards for K-12 students. 7 To address concerns about inadequately prepared students, all states are now using or developing academic standards that are explicitly tied to college and career preparation (referred to in this report as new K-12 standards). In 2010, the Council of Chief State School Officers and the National Governors Association spearheaded the effort to help states develop common college- and career-ready standards for grades K-12 in math and English, which resulted in the Common Core State Standards. As of the beginning of the 2014-15 academic year, 44 states and the District of Columbia were using the Common Core Standards that were developed and published in 2010, and the remaining states were using or developing their own college- and career-ready standards. 5 20 U.S.C. § 1022f(a). A small proportion of states also determine whether TPPs are high performing. According to our survey of all state oversight agencies, 9 states had such a process for some or all TPPs at the beginning of the 2014-2015 academic year. 6 Some states use the terms “credential” or “certificate” instead of “license.” For the purposes of this report, we will use the term “license” to refer to licenses, credentials or certificates. 7 Among other requirements, to receive funding under Title I of the Elementary and Secondary Education Act of 1965, as amended, states are required to adopt challenging academic content and student achievement standards for all public elementary and secondary school children in mathematics, reading or language arts, and science. 20 U.S.C. § 6311(b). Page 6 GAO-15-598 Teacher Preparation Programs Federal Reporting Requirements and Support Education does not have direct oversight authority over TPPs. 8 The two main ways it influences the quality of TPPs are (1) implementing the Higher Education Act Title II reporting requirements and (2) awarding and administering several competitive grants. Higher Education Act Title II Reporting Requirements Title II of the Higher Education Act requires states and institutions of higher education (referred to in this report as “colleges and universities”) 9 that conduct TPPs to annually report specific information. 10, 11 States and most colleges or universities that offer TPPs submit the required data using reporting templates developed by Education (see table 2). 8 However, Education oversees compliance with Title IV of the Higher Education Act (which authorizes federal student aid programs) for participating institutions of higher education, within which some TPPs are housed. As part of its responsibilities under Title IV, Education also recognizes accreditors of institutions of higher education and accreditors of programs, departments, or schools that are part of an institution, which may include organizations that accredit TPPs. According to officials, in reviewing applications for recognition, Education focuses on factors such as an accreditation body’s internal controls, conflict of interest policies, and whether the body’s procedures and standards are widely accepted within the field. It was outside the scope of this report to review Education’s oversight of Title IV. For more information about accreditation of institutions of higher education, see GAO, Higher Education: Education Should Strengthen Oversight of Schools and Accreditors, GAO-15-59 (Washington D.C.: Dec. 22, 2014). 9 In this report we use the term “college or university” to mean “institution of higher education” as defined by the act. 10 See 20 U.S.C. §§ 1022d -1022f. Education officials stated that they are authorized to impose a fine on a college or university for failure to provide required information, but they have not used this authority to date. 20 U.S.C. § 1022d(a)(3). 11 Some entities operating TPPs are not institutions of higher education. The Higher Education Act does not require Title II reporting by entities operating TPPs that are not institutions of higher education. However, according to Education officials, a state may require these entities to cooperate with it in providing information the state needs to prepare the state report. Page 7 GAO-15-598 Teacher Preparation Programs Table 2: Summary of Education’s Title II Reporting Templates for States and Institutions of Higher Education (i.e. Colleges and Universities) that Offer Teacher Preparation Programs (TPP) Reporting sections Program information Description of information reported by a b colleges and universities that offer TPPs Description of information reported by states Descriptive information about TPPs such as Compilation of information about TPPs in the state and the gender and race of candidates; the their participants. academic majors of program completers; the number of candidates that completed the program in a given year; and the average number of hours required for student teaching. Annual goals and assurances Description of institutions’ annual goals for increasing the number of teachers trained in shortage areas and institutions’ progress toward those goals. Certification that institutions are in compliance with various statutorily-defined assurances. Indication that TPPs in the state have provided the various statutorily-defined assurances, such as that program preparation responds to the identified needs of local school districts where the program completers are likely to teach; and that prospective general education teachers are prepared to teach students from specified populations, including students with disabilities and limited English proficient students. (The state report card does not explicitly reference “annual goals.”) Initial teacher credential requirements Not applicable Description of each teaching credential (such as a license) issued by the state. State teacher standards and criteria for certification or licensure Not applicable Yes or no questions about the state’s standards for teacher certification or licensure. Pass rates and scaled scores TPP participants’ pass rates for each test used by the state for teacher certification or licensure. A compilation of the pass rates from TPPs on each state certification/licensure test, as well as information such as the minimum passing score for each test. Alternative routes to teacher certification or licensure Not applicable Description of, and requirements for, alternative route TPPs approved by the state. Criteria for assessing the performance of teacher preparation programs in the state Not applicable Information about the criteria the state uses to assess the performance of TPPs, such as whether the state used criteria from national organizations. Low-performing TPPs Information about whether TPPs have been approved or accredited and whether they have been designated by the state as lowperforming. Description of the state’s criteria and procedures for identifying TPPs that are low-performing or at risk of being low-performing; and a list of TPPs that have been so identified. Use of technology Description of how TPPs prepare teachers to integrate technology effectively into curricula and instruction, and use it effectively to collect, manage, and analyze data to improve teaching and learning. Description of activities that prepare teachers to integrate technology effectively into curricula and instruction, and use it effectively to collect, manage, and analyze data to improve teaching and learning. Teacher training Description of how TPPs prepare teachers to effectively teach students with disabilities and students who are limited English proficient. Not applicable Page 8 GAO-15-598 Teacher Preparation Programs Reporting sections Description of information reported by b a colleges and universities that offer TPPs Description of information reported by states Shortages of highly qualified teachers Not applicable Description of the extent to which TPPs are addressing shortages of highly qualified teachers in the state’s public schools. Efforts to improve teacher quality Not applicable Description of state efforts to improve the quality of the current and future teaching force. Source: GAO analysis of Department of Education Higher Education Act Title II reporting templates.   GAO-15-598 Notes: This table presents a summary of the templates developed by the Department of Education for reporting purposes and is not intended to provide an exhaustive list of the Higher Education Act Title II reporting requirements for states or institutions of higher education. For the statutory reporting requirements, see 20 U.S.C. §§ 1022d-1022f. a See https://title2.ed.gov/Public/TA/IHEReportCard.pdf for a copy of the reporting template most colleges and universities offering TPPs used to complete reporting requirements in 2014. Not all colleges and universities use this template; they may also provide information using a different format if their state so chooses. b See https://title2.ed.gov/Public/TA/StateReportCard.pdf for a copy of the reporting template states used to complete reporting requirements in 2014. Education is responsible for ensuring that states and colleges and universities offering TPPs provide the required information. The agency also compiles and disseminates the information to the public in annual reports, webpages and data spreadsheets. Education contracts with a private research organization (Westat) to provide states and colleges and universities with technical assistance in collecting the required information and to assist the agency in compiling, analyzing, and publishing the resulting data. On December 2, 2014, Education published a notice of proposed rulemaking, which among other things, proposed to modify the Title II reporting requirements. 12 The proposed rule was available for public comment through February 2, 2015. As part of this rulemaking effort, Education has also proposed revisions to the templates for reporting the Title II data. Competitive Grant Programs Education administers several competitive grant programs that provide funding for TPP reforms. Of these competitive grants, the largest that is focused specifically on improving TPP quality is the Teacher Quality 12 Teacher Preparation Issues, 79 Fed. Reg. 71,820 (Dec. 3, 2014). Page 9 GAO-15-598 Teacher Preparation Programs Partnership Grant program. 13 In September 2014, Education selected 24 partnerships, including TPPs and partnering school districts, to receive a combined $35 million in Teacher Quality Partnership grant funds to improve teacher preparation primarily for science, technology, engineering, and math teachers. The Transition to Teaching grant program also provides grants to recruit and retain teachers in high-need schools and encourage the development and expansion of alternative route TPPs. 14 Aside from these two grant programs, applicants for grants from the Race to the Top Fund, Investing in Innovation Fund, and Supporting Effective Educator Development program may also choose to develop proposals related to teacher preparation programs and activities, among other topics. 15 For example, Investing in Innovation grants funded 25 projects related to TPPs out of the 143 projects the program has funded during fiscal years 2010-2014. 13 20 U.S.C. §§ 1022-1022c. Education also administers grant programs that fund teacher preparation efforts that serve specific student populations. For example, the Indian Education Professional Development Grants program provided approximately $1.8 million in new grants for 2014. Two personnel development grant programs related to special education are designed to assist state agencies in reforming and improving their systems for teacher preparation, among other things, in order to improve results for children with disabilities. The Improving Teacher Quality state formula grants may also be used to support TPPs, among other uses. For more details on Education’s grants for teacher quality programs, which include teacher preparation, see GAO, Teacher Quality: Sustained Coordination among Key Federal Education Programs Could Enhance State Efforts to Improve Teacher Quality. GAO-09-593 (Washington, DC: July 6, 2009). 14 20 U.S.C. §§ 6681-6684. Most recently, 30 Transition to Teaching grants were awarded in fiscal year 2011. The program provides grants, with project periods up to 5 years, to state and local educational agencies, or for-profit organizations, non-profit organizations, or institutions of higher education collaborating with state or local educational agencies. 15 The Race to the Top fund supports state reforms to K-12 education including efforts targeted towards developing effective teachers and leaders. Education officials told us that the second award phase was the most recent phase that was likely to include activities related to teacher preparation and the agency awarded $3.4 billion in fiscal year 2010 to phase two awardees. Education awarded $129 million in fiscal year 2014 from the Investing in Innovation Fund which provides competitive grant to applicants, such as partnerships between school districts or non-profits, in order to expand the implementation of, and investment in, innovative practices related to improving student achievement among other goals. Education awarded $39 million in Supporting Effective Educator Development Grants in fiscal year 2013—the most recent year for which the agency awarded new grants. This program provides grants to national non-profit organizations for evidence-based projects to recruit, select, and prepare or provide professional enhancement activities for teachers, principals, or both. Page 10 GAO-15-598 Teacher Preparation Programs States Assessed TPPs before Approving Them to Prepare Teachers but Some States Reported Not Having a Process to Identify Low-Performing TPPs as Required by Statute Almost All States Assessed TPP Program Design and Information about Candidates and More Than Half Assessed TPPs Based on Graduates’ Effectiveness All states reported that they review traditional TPPs before approving them to prepare new teachers and may renew approval on a periodic basis. 16 To do this, nearly all states reported that they review TPP program design and data about candidates before approving them to prepare new teachers, and more than half also use one or more types of information to assess graduates’ effectiveness as teachers. 17 To assess program design, 49 states and the District of Columbia reported in our survey that they assess whether TPPs seeking approval are meeting standards for program quality. 18 For example, these standards may specify that teaching candidates should be trained to identify the 16 Our case study states renewed approval for TPPs every 5 to 7 years. Title II does not prescribe how states are to approve TPPs. 17 We define program design to mean syllabi or other course material, clinical experiences, and interviews with faculty and staff. Data about candidates includes: licensure assessment pass rates; graduation and completion rates; the results of teaching candidates performance assessments from pre-service clinical practice; and, surveys concerning TPP graduates’ satisfaction with the preparation they received from their program. We consider information assessing graduate effectiveness to include: surveys of principals and other district personnel regarding their satisfaction of recent TPP graduates; K-12 student assessment results to measure teacher effectiveness; and, teacher evaluation results for recent TPP graduates. 18 These standards may be state-developed standards for TPPs or teachers, or standards developed by others. For more information, see questions 8a – 8k in the accompanying esupplement, GAO-15-599SP. Page 11 GAO-15-598 Teacher Preparation Programs appropriate teaching techniques for particular learning needs or achieve a particular threshold of subject matter knowledge. States also reported that they typically reviewed program design for traditional TPPs by reviewing syllabi or other course material (43 states), and interviewing TPP faculty or staff (41 states). 19 To conduct these reviews, states reported that they may use teams made up of peers from other TPPs, state staff, national accreditation organization staff, or a combination, and make determinations using professional judgment. 20 For example, when conducting these reviews, 43 states reported that they consider information collected by an external TPP accreditation organization— such as the Council for the Accreditation of Educator Preparation (CAEP)—for at least some traditional TPPs. In addition to reviewing program design, nearly all states reported in our survey that they examine data about teaching candidates. Most states reported using data about the proportion of candidates who obtain a teaching license (48 states) and the proportion of candidates who graduate (29 states) as part of their approval process for all traditional TPPs (see fig. 1). 21 19 Unless otherwise noted, survey results are out of 51 total responses (50 states plus the District of Columbia). 20 In our survey, 38 states provided information about the number of TPP providers they approved, conditionally approved, or denied during the 2013-2014 academic year. In total, they reported approving a combined 345 TPP providers, placing a total of 5 under conditional approval, and denying approval for a total of 2 TPP providers. The other states reported they did not have information available about such decisions. 21 In addition to reviewing data when a TPP is up for initial approval or renewal, some states may also review data intermittently. Our case study states used varying approaches to assess data on candidates in between formal approval and renewal decisions. For example, one case study state had each TPP choose the data they use to measure effectiveness. In contrast, another case study state verified that TPPs met a minimum licensing test pass rate. Page 12 GAO-15-598 Teacher Preparation Programs Figure 1. Information about Teaching Candidates that States Reported Using to Assess Traditional Teacher Preparation Programs (TPP) Seeking Approval Notes: These results are based on our survey of all 50 states and the District of Columbia. In some instances, the wording of the original survey question has been modified for brevity and to remove technical terminology. See the related e-supplement, GAO-15-599SP, for the original language. Additionally, for information about the teaching candidate data that states used as part of their approval process for alternative route TPPs, see questions 9a, 9f, 9h, and 9i in the related esupplement. a “Pre-service assessments” could include evaluations of candidates’ performance as student teachers. More than half of states reported that they also review certain information about graduate effectiveness when assessing TPPs for approval or renewal. States that incorporate this information reported doing so in several ways. Most commonly, 30 states reported using surveys that assessed principals’ and other district personnel’s satisfaction with recent traditional TPP graduates. Fifteen states reported assessing traditional TPPs based on other outcomes data such as the test scores (i.e., K-12 student assessment results) of public school students taught by recent TPP graduates (see fig. 2). 22 For example, one of our case study states 22 The data described here pertain to how states use effectiveness information to assess traditional TPPs. For more information about how states use effectiveness data to assess alternative route TPPs, see questions 9b- 9e and 9g in the accompanying e-supplement, GAO-15-599SP. Page 13 GAO-15-598 Teacher Preparation Programs uses such data to help TPPs identify potential problem areas in the training they provide to teaching candidates. Officials in this state told us they used these data to help a TPP identify shortcomings in its social science program. Figure 2: Information about Graduate Effectiveness that States Reported using to Assess Traditional Teacher Preparation Programs (TPP) Seeking Approval Notes: These results are based on our survey of all 50 states and the District of Columbia. In some instances, the wording of the original survey question has been modified for brevity and to remove technical terminology. See the related e-supplement, GAO-15-599SP, for the original language. Additionally, for information about the effectiveness data that states used as part of their approval process for alternative route TPPs, see questions 9b- 9e and 9g in the related e-supplement. As of the 2014-2015 academic year, at least 10 additional states reported that they planned to begin using graduate effectiveness information or expand their current use of such information as part of their approval process, according to our case study state interviews and several survey responses. For example, officials in Tennessee told us that they currently review some effectiveness data in their approval process and plan to begin reviewing recent TPP graduates’ teacher evaluation results in the future. Additionally, officials in Arizona told us they plan to begin Page 14 GAO-15-598 Teacher Preparation Programs reviewing data on graduate effectiveness as part of their approval process. They told us that the impetus behind adding this new data is to better align the state’s TPP quality standards with recommended requirements presented by CAEP, so that TPPs that already have CAEP approval may receive an expedited state approval process. 23 States and TPPs reported challenges collecting information on graduates’ effectiveness. Officials from 3 of our 5 case study states and 7 of the 14 TPPs we spoke to said that collecting this type of data is difficult. For example, state oversight offices or TPPs would need to obtain key information about TPP graduates—such as performance evaluations or employer survey responses—from local districts and several of these officials noted districts may be difficult to identify or not be willing to provide such information. Officials in one of our case study states noted that it was especially challenging to obtain data on teachers who work in another state or in private schools. As shown in figure 2 above, states more commonly reported using information about the effectiveness of graduates teaching in public schools within their state versus information about those teaching in private schools or in other states. Some States Use Different Types of Information to Approve Alternative Route TPPs versus Traditional TPPs When deciding whether to approve or renew TPPs for operation, 22 states reported using fewer sources of information for alternative route TPPs compared to traditional TPPs. For example, eight states reported that they assessed alternative route TPPs against state developed standards less frequently than traditional TPPs and seven states reported using observations of alternative route TPP courses or experiences less frequently (see table 3). 23 By the fall of 2016, CAEP expects to begin evaluating TPPs against new standards that measure the effectiveness of new teachers and the performance of the students they teach. Page 15 GAO-15-598 Teacher Preparation Programs Table 3: Differences in the Sources of Information States Reported Using to Assess Traditional versus Alternative Route Teacher Preparation Programs (TPP) Seeking Approval Number of states that review the information less frequently for alternative route TPPs than for traditional TPPs Number of states that review the information less frequently for traditional TPPs than for alternative TPPs 16 0 Assessment of TPP against external teacher and/or TPP standards 8 0 Assessment of TPP against state developed standards for TPPs 8 0 8 0 Licensure assessment pass rates 7 0 Observations of courses or clinical experiences during a site visit to the TPP 7 1 Assessment of TPP alignment with state’s K-12 academic content standards 6 0 Assessment of TPP against state developed teaching standards 6 0 Data about TPPs (such as TPP enrollment or performance information) 6 0 Interviews with TPP faculty, staff, and/or administrators 6 1 Sources of information Consideration of decisions made, or evidence collected, by the Council for the Accreditation of Educator Preparation, and/or other relevant professional associations Results of pre-service assessments for teacher candidates a Source: GAO analysis of survey of state oversight entities GAO-15-598 Notes: These results are based on our survey of all 50 states and the District of Columbia. In some instances, the wording of the original survey question has been modified for brevity and to remove technical terminology. This table only includes the ten information sources for which states most frequently reported differences between what they required from traditional and alternative route TPPs. For more information about the data that states used as part of their approval process for alternative route and traditional TPPs, see the related e-supplement, GAO-15-599SP. a ”Pre-service assessments” could include evaluations of candidate’s performance as student teachers. The differences in how states approve alternative route TPPs compared to traditional TPPs may be a consequence of several factors, including how states define alternative route TPPs and differences in state requirements for alternative route and traditional TPPs. For example, some alternative route TPPs may not include a student teaching requirement, so information about student teachers’ performance would not be relevant for making approval decisions. Additionally, in response to one of our survey questions, a representative from one state explained that the requirements for alternative route and traditional TPPs are Page 16 GAO-15-598 Teacher Preparation Programs different because the requirements for traditional TPPs are set by state regulations while some alternative TPPs may be approved through other mechanisms. Some States Reported Not Having a Process for Identifying Low-Performing TPPs, and Education Does Not Verify State Procedures Some states reported not having a process for identifying low-performing TPPs, as required under the Higher Education Act. In order to receive funds under the Higher Education Act, states are required to conduct an assessment to identify low-performing TPPs in the state and provide Education with an annual list of such programs and any programs at risk of being placed on the list. 24 States have flexibility in the criteria they use to identify low-performing TPPs, but are required to describe those criteria to Education as part of their annual Title II reports. This provides an avenue for states to assess TPP quality and make their determinations public. However, officials from seven states reported in our survey that they did not have a process for identifying low-performing TPPs in academic year 2014-2015. 25 In response to our follow-up inquiries, state officials who reported not having a process in our survey told us they believed their other oversight procedures are sufficient to ensure quality without having a process to identify low-performing TPPs or that they were in the middle of changing their state’s process for identifying low-performing TPPs, among other reasons. For example, officials in 2 of the 7 states that reported they do not have a process to identify low-performing programs in our survey told us having a process for identifying low-performing TPPs was not necessary to ensure that all TPPs are performing sufficiently. While it is possible that all TPPs are meeting states’ performance criteria and do not merit a low-performance designation, states are still required to conduct an assessment. Officials from two other states told us that they are developing or planning to develop a new process for identifying low- 24 20 U.S.C. § 1022f(a). States are required to assist any programs identified as lowperforming through the provision of technical assistance. For purposes of this report, when we refer to “low-performing” programs, we generally also include those programs at risk of being low-performing. Education’s December 2014 proposed rule would require states to report TPP performance using at least four performance levels: low-performing, at-risk, effective, and exceptional. It also proposes establishing required areas for states to consider in identifying low-performing or at-risk TPPs. See 79 Fed. Reg. 71,820 (Dec. 3, 2014). 25 The number of TPPs in these states ranged from 10 to 45 TPPs and the number of students enrolled in these states ranged from approximately 1,700 to 42,000 for a combined total of 72,000 in academic year 2012-2013. Page 17 GAO-15-598 Teacher Preparation Programs performing TPPs, but do not have a process in the interim. Education officials told us that while states can choose to change their process from time to time, they are expected to use the previous process until the new one is implemented. Education does not verify whether states use the process they describe in their Title II reports to identify low-performing programs or ensure that all states have such a process. According to Standards for Internal Control in the Federal Government, an agency’s management should provide reasonable assurance of compliance with applicable laws and regulations. 26 In addition, under the Higher Education Act, Education has responsibility for ensuring the quality of the data submitted in Title II reports. 27 Education officials told us that state officials are expected to certify the accuracy of the Title II data they submit to Education and Education reviews the state reports for obvious instances of noncompliance. However, agency officials were not aware that two states did not describe a process to identify low-performing programs in their most recent Title II reports. 28 Education officials also said that Education does not verify that states are in fact implementing the procedures they describe in their Title II reports, due to financial constraints. All seven states that reported in our survey that they did not have a process for identifying low-performing TPPs described a process to Education in the Title II report that was reported in October, 2014. 29 Without a monitoring process to verify the accuracy of this information in state reports, Education may miss instances of noncompliance. If states are failing to 26 GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999). 27 Education is required to establish regulations to ensure the reliability, validity, integrity, and accuracy of the data submitted in the institutional and state report cards. 20 U.S.C. § 1022d(c). Education is also required to ensure that states and institutions of higher education use fair and equitable methods in reporting the data. 20 U.S.C. § 1022g(a). 28 In response to our inquiry, officials from one of these states told us that they do not approve any programs that do not meet all of the states’ content and teaching standards and, therefore, would not approve any low-performing TPPs. According to the other state’s most recent Title II report, the state has only one TPP. In our survey, that state reported that they assess low-performance on a case- by-case basis. 29 In its guidance on reporting information about states’ processes for identifying lowperforming TPPs, Education asks states to include the most recently available information without specifying a date. In our survey, conducted from November 2014 to March 2015, we asked states to report data for academic year 2014-2015. Page 18 GAO-15-598 Teacher Preparation Programs comply with the federal requirement to conduct an assessment to identify low-performing TPPs, struggling TPPs may not receive the technical assistance they need and potential teaching candidates or hiring school districts will have difficulty identifying struggling TPPs. This may impact the quality of training provided to new teachers and result in their being inadequately prepared to educate children. The majority of states (43) reported to us that they had a process for identifying low-performing TPPs or TPPs that were at risk of becoming low-performing. The most common criteria they used to identify lowperforming TPPs were failure to meet the state’s TPP or teaching standards (used by 35 states) and denial or conditional approval during the state approval or renewal process (used by 34 states). 30 Fewer states used teacher evaluations (9) or student assessments (8) to identify lowperforming TPPs. According to data that states submitted as part of their Title II report, 6 states identified one or more TPPs as low-performing and 13 states identified one or more TPPs as at risk of becoming low-performing in 2013 or 2014. 31 When TPPs are identified as low-performing, the Higher Education Act requires that states provide them with technical assistance. 32 Of the 6 states that identified low-performing TPPs in 2013 or 2014, all reported in our survey that they provided technical support and informed the TPPs of their status. Most also publicized this status (4) and increased their monitoring of the TPPs (5). 30 For more information, see questions 18a- 18r in the accompanying e-supplement, GAO-15-599SP. 31 Two of the six states that identified low-performing programs also identified programs at risk of becoming low-performing. U.S. Department of Education, At-Risk and LowPerforming Programs by State for 2013 and 2014, in the Title II Data Tools, accessed April 2015, https://title2.ed.gov/Public/DataTools/Tables.aspx. 32 20 U.S.C. § 1022f(a). Page 19 GAO-15-598 Teacher Preparation Programs Most States Provided Some Support and Oversight As TPPs Modified Their Programs Based on New K-12 Standards New K-12 Standards Led Most States to Offer TPPs Information, Adjust Their Approval Process, and Use Modified Licensing Tests As states shifted to new K-12 standards, most reported taking steps to help TPPs prepare prospective teachers to teach lessons aligned with the new standards. To help TPPs understand the standards, 37 states reported in our survey that they provided TPPs with written resources, information sessions, or both. 33 All of our five case study states offered TPPs information about the standards using various approaches. For example, one state convened a half-day workshop that included information about the new standards, related changes to TPP oversight processes, and examples of how TPPs might choose to adapt to the new standards. Officials then posted a recording of the session and additional resources about the standards online. Three states reported inviting TPPs to K-12 conferences that discussed the new standards. Officials in one of those states said that this approach allowed them to foster communication and coordination among K-12 districts and TPPs. Apart from offering information, most states reported that they modified their oversight activities to verify that TPPs were aligning with new K-12 standards. In particular, 34 states reported deliberately modifying their TPP approval process to assess such alignment, by, for example, modifying state standards for TPP quality to align them with new K-12 33 Specifically, 19 states reported organizing or funding TPP attendance at information sessions that were specifically targeted to them, 22 states reported organizing or funding TPP attendance at information sessions that were primarily designed for K-12 school districts, and 32 states reported providing TPPs with written resources about the new K-12 standards. For more information, see questions 31e, 31f and 31g in our e-supplement, GAO-15-599SP. Page 20 GAO-15-598 Teacher Preparation Programs standards. 34 Twelve other states did not report modifying their approval process for this specific reason, but did report assessing some or all TPPs against standards for TPP quality that may nonetheless provide information about alignment. For example, the Interstate Teacher Assessment and Support Consortium (InTASC) standards are commonly used standards for TPP quality and were designed to align with the Common Core State Standards. 35 According to our survey, two states reported placing a TPP under conditional approval due in part to limited alignment with new K-12 standards during the 2013-2014 academic year, and no states reported denying approval for this reason. Fewer states reported modifying their process for identifying lowperforming TPPs to assess alignment with new K-12 standards. Specifically, in our survey, 27 states reported taking steps to modify the process for identifying low-performing TPPs. For example, California reported modifying its process for approving TPPs to assess alignment with new K-12 standards. It also continued its previous practice of identifying TPPs as low-performing if they receive conditional renewal decisions. In our survey, no states reported identifying a TPP as lowperforming during the 2013-2014 academic year because of limited alignment with K-12 standards. States also used modified licensing tests designed to assess individual teaching candidates’ preparation for the new K-12 standards, according to officials from the companies that develop the tests. Two national companies—Educational Testing Service and Pearson—developed tests related to math and English language arts and reported that such tests are used by 43 states. The testing companies both reported modifying those tests to align with new K-12 standards, although we did not independently evaluate the extent of this alignment. 36 Among the 8 states 34 These states reported in our survey that they took one or more of the following steps to help TPPs prepare teaching candidates to teach to the college- and career-ready standards: (a) modified state teaching and/or TPP standards or (b) modified the state’s TPP approval process to include assessment of the TPP’s alignment. See question 31 in our e-supplement, GAO-15-599SP. 35 The INTASC standards were developed by the Council of Chief State School Officers. 36 Within the Educational Testing Service, these tests include the Praxis Core Academic Skills for Educators tests and Praxis Content tests in mathematics and English language arts. Within Pearson, these tests include National Evaluation Series tests in reading, writing and mathematics. The 43 states used one or more of these exams, according to testing company records. Page 21 GAO-15-598 Teacher Preparation Programs that do not use such tests, 4 reported in our survey that they modified their licensure requirements in other ways in response to the new K-12 standards. 37 For example, all four of these states contract with testing companies to design customized tests and may request revisions to align with the new K-12 standards. Pearson officials described working with one such state to modify its custom tests to align with new K-12 standards by adding more questions that measure teaching candidates’ ability to teach non-fiction texts and address how to help hypothetical K12 students understand complex subject-matter information. To Varying Degrees, Select TPPs Took Steps to Incorporate New K-12 Standards All 14 TPPs we interviewed made changes that ranged from large-scale reforms to more modest modifications. 38 The changes generally fell within the following three categories: (1) increasing subject-matter knowledge, (2) modifying coursework related to teaching techniques, and (3) using classroom training to provide real world experience. Examples of such changes, and related challenges, are listed below. • Increasing subject-matter knowledge: Officials from 11 of 14 TPPs described changing coursework or coursework requirements to ensure that teaching candidates had sufficient subject-matter knowledge to teach to the new K-12 standards. The three TPPs that did not make such changes were graduate-level programs or otherwise required teaching candidates to obtain a bachelor’s degree before attending the TPP and officials said most or all subject-matter knowledge should be obtained prior to starting the TPP. The TPPs that did make changes sometimes coordinated with other departments, such as math and English. For example, one TPP began offering courses that were co-taught by subject-area and TPP 37 Respondents from 2 of the 8 states reported that the state did not make modifications to the licensing requirements to align with new K-12 standards and the respondents from two states reported that they did not know whether the state had modified the licensing requirements for this reason. 38 For example, officials from two TPPs described completely redesigning their programs as a direct result of the more rigorous standards and those from two others said that the standards helped them target and expand reforms that were already underway when the new K-12 standards were introduced. Conversely, one TPP provided faculty and teaching candidates with web links to external resources and waited to make changes to its courses until they were scheduled for revision. An official from another TPP who described the TPP’s changes as relatively modest stated that the new K-12 standards were well-aligned with best practices for teaching and, therefore, supported the TPP’s existing approach. Page 22 GAO-15-598 Teacher Preparation Programs faculty. Officials from a few TPPs stated that some academic departments were more receptive to modifying their curriculum than others, due to department priorities or other factors. For example, officials from one TPP observed that the new K-12 standards for English require the participation of teachers from multiple academic disciplines, but TPP faculty members who were not in the English department were sometimes reluctant to modify their courses accordingly. • Modifying coursework related to teaching techniques: Officials from all but one of the 14 TPPs we contacted described modifying coursework related to teaching techniques. For example, officials from one TPP said that candidates should learn teaching techniques that focus on collaboration and officials from another said that it was important for candidates to make connections between different subject areas. Officials from five TPPs also said the new K-12 standards led them to start or expand courses on teaching techniques that are subject-matter specific. • Using classroom training to provide real world experience: Officials from all 14 TPPs we spoke with highlighted the importance of providing candidates with ample opportunities to apply the new K-12 standards in real classrooms and receive feedback or support from mentor teachers or TPP staff. For example, officials from one TPP and a school district said that mentor teachers can be important role models for teaching candidates because they can illustrate how to apply new teaching techniques and adapt to changing expectations. Officials from 9 of the 14 TPPs described assessing candidates’ preparedness for the new K-12 standards when reviewing their performance or soliciting district feedback about teaching candidates’ performance. However, officials from half of the TPPs we interviewed acknowledged the difficulty of training new teachers in real classroom settings. In particular, officials from 6 TPPs said that school districts are training veteran teachers in the new standards at varying rates, and officials from several TPPs observed that teaching candidates may not always be paired with a veteran teacher who knows the standards as well as the teaching candidate. As states continue to implement the new K-12 standards, several TPPs we spoke with said that they planned to make further modifications. For example, one TPP that made a number of modifications to its program recently surveyed faculty and administrators to evaluate its efforts and identify any ongoing needs. In addition, many states are beginning to adopt assessments to measure K-12 students’ performance on the new Page 23 GAO-15-598 Teacher Preparation Programs standards, and officials from five TPPs said it will be important to incorporate information about these assessments into their programs. Education Missed Opportunities to Support TPP Improvement through Its Data Reporting and InformationSharing Some Title II Data Have Not Been Useful to States and TPPs, but Education Has Not Assessed Whether Less Useful Elements Should be Eliminated The current Title II data requirements may be of limited use in helping to improve the quality of TPPs and Education has not taken steps to evaluate whether any of them should be eliminated. Each state collects Title II information from colleges and universities that offer TPPs in the state and submits information about TPPs and some information about state oversight processes to Education annually. For example, they report the number and demographics of teaching candidates enrolled in and completing TPPs. 39 As described below, states, TPPs, and other stakeholders often reported to us that, while they may use some data elements, others are not useful. This difference between the Title II information states and colleges and universities are required to report about TPPs and the information they and other stakeholders ultimately use to make decisions is contrary to leading practices for data-driven 39 Title II requires states to report the number of students in each TPP, disaggregated by race, ethnicity, and gender, as well as certain information on the “number of teachers prepared.” See 20 U.S.C. § 1022d(b)(1)(G)(ii), (b)(1)(H). As previously mentioned, the Higher Education Act does not require reporting by entities that are not institutions of higher education. However, states may also collect information from TPPs which are not located in institutions of higher education in order to complete their state reports. Page 24 GAO-15-598 Teacher Preparation Programs management, which state that measures should be selected based on their relevance and ability to inform decisions by key stakeholders. 40 Moreover, state oversight entities are important potential users of the Title II data, but they reported mixed views about whether the data were useful for their oversight, even though it takes a relatively large number of staff hours to prepare the data. In our survey, states most frequently reported spending between 21-100 staff hours completing the annual Title II statelevel reports and another 21-100 hours assisting with institution and program-level Title II reports. (See fig. 3). Figure 3: State Reports of Time Spent on Higher Education Act Title II Reports during Academic Year 2013-2014 Note: These results are based on time estimates for state staff and contractors in our survey of all 50 states and the District of Columbia. 40 GAO, Managing for Results: Enhancing Agency Use of Performance Information for Management Decision Making, GAO-05-927 (Washington, D.C.: Sept. 9, 2005) and GAO, Results-Oriented Government: GPRA Has Established a Solid Foundation for Achieving Greater Results, GAO-04-38 (Washington, D.C.: Mar. 10, 2004). Page 25 GAO-15-598 Teacher Preparation Programs After devoting this time to the task, 5 states said the Title II data were “very useful,” 19 said they were “moderately useful,” and 25 states said they were “neither useful nor not useful,” “slightly useful,” or “not useful.” States most frequently reported using the Title II data to inform the approval process or inform state agency discussions about proposed TPP requirements. Very few states reported using the data to inform state funding decisions or to provide information to school districts that are hiring teachers from TPPs. 41 Nearly every state reported some cases where certain requirements may not be helpful. For example, less than half the states told us that they use the goals and assurances section of the Title II reports, which includes information about institutions’ progress toward their goals for addressing teacher shortage areas. Further, 48 states told us either that they are not using some sections of the Title II reports or that they already collect most of the useful Title II data elements through other mechanisms. These results suggest that, even among states that find the Title II data generally useful, states are frequently required to complete some reporting requirements that they report are not contributing to their oversight activities. Most of the TPPs and K-12 districts and several other stakeholders with whom we spoke questioned the usefulness of some Title II data to themselves and other stakeholders. Officials from 6 of the 14 TPPs with whom we spoke reported spending at least a month completing the annual Title II reports. Yet, 8 of the 14 TPPs told us that very little of the Title II data was useful to them for assessing the performance of their own programs. Officials from several TPPs also expressed confusion about the overarching goal of the Title II reporting requirements. The TPP officials said they have not seen any indication that other stakeholders, such as state and federal regulators or prospective teaching candidates, were using the Title II data to inform decisions. Further, none of the officials in the six K-12 school districts we spoke with said they use the Title II data when comparing TPP performance or recruiting new teachers. Similarly, seven researchers and stakeholder organizations we spoke with questioned the usefulness of some of the Title II data for research purposes. 41 Three states reported that they used the data to inform state funding decisions and 6 states reported that they used it to provide information to school districts that are hiring teachers from TPPs. See our associated e-supplement more details about the results of our survey, GAO-15-599SP. Page 26 GAO-15-598 Teacher Preparation Programs Education officials said they have to continue to collect the current data, due to statutory requirements. According to Education officials, the agency can require states to submit new Title II data elements, but they do not have authority to remove existing data elements that are required by statute. Education described some benefits to the existing data, including that it provides a robust picture of the demographics of teaching candidates in each state, as well as the number of individuals who complete a program. However, Education also noted in the preamble to its December 2014 proposed rule that “data that are collected and reported have not led to an identification of significant improvements in teacher preparation program performance in part because the data are not based on meaningful indicators of program effectiveness.” 42 Education officials told us they have not conducted a study to inform Congress or the agency about whether any current reporting requirements are not useful. Therefore, Education has an incomplete picture of the usefulness of various data fields and states and colleges and universities will continue submitting data that are time-consuming to gather and may not be contributing to state oversight, TPP improvement, or the public’s knowledge about TPPs. Education Has Not Identified and Publicized Key Limitations in How Data Should Be Interpreted Education permits states to report some Title II data in different ways to account for their differing approaches to overseeing TPPs and the differing structures of the TPPs themselves. This affects the consistency and clarity of the data that Education ultimately disseminates. Examples of key elements that may vary include: • Alternative route TPPs: States may use different definitions of alternative route TPPs when reporting information such as the number of alternative route TPPs in their state and the number of teaching 42 79 Fed. Reg. 71,820, 71,823 (Dec. 4, 2014). Among other things, Education’s proposed rule would modify the Title II reporting requirements to establish required indicators, such as student learning outcomes, for states to use in assessing and reporting on the performance of their TPPs. Page 27 GAO-15-598 Teacher Preparation Programs candidates who enroll in or complete such TPPs each year. 43 For example, according to Education officials, one of the top teacherproducing states defines some TPPs as traditional TPPs that most states would consider alternative route TPPs. Education officials said that such decisions are within the states’ authority, but have repercussions for the consistency of national Title II data. • Teaching candidate enrollment: TPPs define when a teaching candidate is formally enrolled and these definitions may range from when a teaching candidate first takes a course to after they have completed other requirements such as a certain sequence of courses. Moreover, one large online TPP defines most of its teaching candidates as enrolled in the state where the TPP is headquartered, even though they live throughout the country and most likely plan to teach in other states. Consequently, the Title II report lists that state as one of the nation’s top teacher-producing states, while state officials told us that in fact the state faces teacher shortages. • Program Completers: Education’s Title II guidance defines a TPP completer as someone who has met all the educational or training requirements in a state-approved course of study for initial teacher certification or licensure. However, this definition allows for states or TPPs to choose whether they require their students to take and pass all state licensing tests before they can complete all of those educational or training requirements. As a result, different definitions of completer can lead to inconsistent data and make it difficult to make comparisons across TPPs or obtain a national picture regarding TPP completers. These varying ways of calculating and reporting key Title II data have persisted despite Education’s efforts to clarify guidance and improve reporting tools. Such efforts, according to Education and its Title II 43 Title II of the Higher Education Act specifies various information that states must report both for traditional TPPs and for “alternative routes to teacher certification or licensure in the State (including any such routes operated by entities that are not institutions of higher education);” however, alternative routes are not further defined by the statute. See, e.g., 20 U.S.C. § 1022d(b)(1)(E). According to Education guidance, for Title II reporting purposes, an “alternative route to a teaching credential” is a teacher preparation pathway that primarily serves candidates that are the teacher of record in a classroom while participating in the route. Alternative routes to a teaching credential are defined as such by the state. According to the guidance, in general, a “traditional” TPP is one that primarily serves undergraduate students without prior teaching or work experience and leads at least to a bachelor’s degree. Page 28 GAO-15-598 Teacher Preparation Programs contractor, have included clarified definitions, new guidance documents, some on-site technical assistance visits, and additional data checks in the online Title II reporting system. Several states and TPPs with whom we spoke praised Education’s efforts to facilitate the Title II submission process, noting particularly that they have received excellent technical assistance when they have questions about the process. However, states and TPPs also noted remaining challenges, such as how to interpret the Title II requirements within the context of each state or TPP’s specific circumstances. Education identified some potential data inconsistencies, such as differences in state definitions of alternate route TPPs, in its most recent Title II annual report, which presented information about the 2009-2010 academic year. 44 However, it has not provided similar information about limitations of data it has disseminated since then. In more recent Title II data in published spreadsheets and on its website, Education does not include clarifications about potentially inconsistent data elements, such as alternative route TPPs or definitions of enrollees and completers. Education officials told us they did not include such explanatory material in these other formats because they considered the explanations in the previous annual report to be sufficient. However, these officials also noted that the agency may consider adding such material in the future. By not providing the explanatory material, the data may be potentially misleading and make it difficult for users to compare across states or programs. In addition, by not providing these explanations, Education’s approach is not consistent with federal internal control standards, which require that pertinent information should be identified, captured, and distributed in a form that permits users to perform their duties efficiently. 45 44 Department of Education, Office of Postsecondary Education, Preparing and Credentialing the Nation’s Teachers: The Secretary’s Ninth Report on Teacher Quality. (Washington, D.C. April, 2013) This is the most recent Title II annual report that Education has produced, as required by 20 U.S.C. § 1022d(d). It is based on data collected in 2011, which covers the 2009-2010 academic year. 45 GAO/AIMD-00-21.3.1 Page 29 GAO-15-598 Teacher Preparation Programs Education Has Taken Few Steps to Share Information about Teacher Preparation Program Quality Internally and with States Education has made few efforts to share expertise about TPP quality among its offices. Various offices and programs within Education influence TPP quality, including the Office of Postsecondary Education, which administers Higher Education Act Title II reporting requirements, and several offices that administer competitive grants. However, the agency does not have mechanisms in place to promote regular, sustained information-sharing among these offices. Education officials said the agency and its Title II contractor occasionally create custom Title II data runs for Education program offices, but there is no systematic effort to share Title II data within the agency. Further, Education officials in one office that administers competitive grants related to TPP quality described discussing grant results internally, but did not systematically discuss TPP quality with staff in other offices and programs. Education formerly convened a teacher quality workgroup on a regular basis that included opportunities to share information across the agency regarding issues related to TPP quality. Education officials noted that such workgroups, particularly when operating out of high-level offices in the agency, have been very helpful for systematically sharing information across offices. However, the teacher quality workgroup has been inactive since the office in which it was housed reorganized in the fall of 2014 and the agency has not subsequently resumed these information-sharing efforts. This represents a missed opportunity to use relevant information to bolster the effectiveness of several Education programs, such as Race to the Top and Supporting Effective Educator Development, which fund TPP improvements, among other priorities. Federal internal controls standards emphasize that effective information-sharing efforts are those that flow broadly across an agency in a form that is helpful for those who need it to carry out their responsibilities. 46 Without such mechanisms to promote information-sharing, programs and offices within Education may not have access to clear and useful information about TPP quality. Furthermore, Education’s efforts to support or enhance TPP quality reach a limited number of states. For example, according to Education officials, several technical assistance and research entities and the office that administers competitive grants such as the Teacher Quality Partnership Grant Program have recently undertaken research or disseminated good practices about teacher preparation. However, in our survey, only about one third of states reported receiving information from Education about 46 GAO/AIMD-00-21.3.1 Page 30 GAO-15-598 Teacher Preparation Programs TPP oversight or enhancing TPP quality, and about half of all states said they would like additional support from Education on this topic. 47 This suggests that Education is also missing opportunities to support states that could use relevant research and assistance from Education to enhance TPP quality. Additionally, 15 states reported in our survey that they oversee TPPs through an independent standards board, oversee the licensing of new teachers through such boards, or both. Among the eight such states that responded to our follow-up inquires, officials from four expressed concern about their access to Education resources for TPP improvements because they are independent from the primary state educational agency that has formalized relationships with Education’s technical assistance providers. 48 Gaps in the agency’s efforts to disseminate information result from information- sharing being left to individual offices’ initiative rather than an agency-wide mechanism, and Education officials noted that more could be done to share information with states and other stakeholders. Education officials also noted that sharing information with states can be challenging for some competitive grant programs, because program funds are not always available for Education to use for national activities, including providing technical assistance to non-grantees. However, this also underscores the importance of Education systematically leveraging existing resources to disseminate knowledge about enhancing TPP quality. Federal internal controls standards emphasize the importance of agencies ensuring adequate information-sharing with key stakeholders. 49 Without such an approach, Education may be missing opportunities to support state efforts to enhance TPP quality. For example, states may be unaware of information about good practices for TPP quality that could assist them in their oversight. 47 These states reported in our survey that they would like more of one or more of the following types of assistance: dissemination of information on best practices; one-on-one discussions (via phone, teleconference or in person); and/or, group conference calls, webinars or in-person sessions. 48 In follow-up to our survey, we contacted all 15 states that oversee TPPs through an independent standards board, oversee new teacher licensing through such a board, or both, and 8 responded. 49 GAO/AIMD-00-21.3.1 Page 31 GAO-15-598 Teacher Preparation Programs Conclusions TPPs serve a vital role in preparing new teachers—and thereby, K-12 students—for future success. The recent shift to college- and careerready K-12 standards further highlights the importance of such programs. Education is responsible for collecting and disseminating Higher Education Act Title II data and administering grant programs that provide an opportunity to contribute to TPPs’ continuous improvement. However, unless Education ensures that all states assess whether TPPs are lowperforming as required by the Higher Education Act, low-performing TPPs may not be identified, potentially resulting in new teachers being illprepared to teach K-12 students. Further, if Higher Education Act Title II reports include data elements that are not useful, states and colleges and universities that offer TPPs will expend unnecessary effort on collecting information that is unlikely to improve TPP quality. Among reporting requirements that are useful, if Title II reports do not include important limitations on how the data should be used, policymakers and practitioners could draw incorrect conclusions based on the data. Finally, without increasing information-sharing within the Department and with states, Education may miss opportunities to disseminate information that could enhance TPP quality. Recommendations for Executive Action We recommend that the Secretary of Education take the following four actions: 1. Develop a risk-based, cost-effective strategy to verify that states are implementing a process for assessing whether any teacher preparation programs are low-performing. 2. Study the usefulness of Title II data elements for policymakers and practitioners, and, if warranted, develop a proposal for Congress to eliminate or revise any statutorily-required elements that are not providing meaningful information. 3. Identify potential limitations in the Title II data and consistently disclose these limitations in the reports, websites, and data tables the agency uses to distribute the results. This could include more detailed information about data elements where definitions vary substantially from state to state or teacher preparation program to teacher preparation program. 4. Develop and implement mechanisms to systematically share information about teacher preparation program quality with relevant Department of Education program offices and states (including state Independent Standards Boards). Page 32 GAO-15-598 Teacher Preparation Programs Agency Comments and Our Evaluation We provided a draft of the report to the Department of Education for review and comment. Education’s comments are reproduced in Appendix II. Education agreed with our four recommendations. Regarding our first recommendation, Education noted that its proposed regulations include new requirements for how states report on TPP performance, including whether any TPPs are low-performing. Education anticipates that its final regulations will guide the agency’s future efforts to monitor states’ processes for identifying any low-performing TPPs. While it finalizes its regulations, Education also plans to work closely with select states to help ensure they comply with Title II requirements related to identifying lowperforming TPPs. We believe that interim monitoring will be important, particularly if the regulations are not finalized prior to the next Title II reporting cycle. Education also agreed with our other three recommendations, stating that the agency would: examine statutorilydefined Title II reporting requirements and make recommendations to Congress to remove or revise requirements, as warranted; identify potential limitations in the Title II data and disclose such limitations in its reports, websites, and data tables; and, enhance information-sharing about TPP quality within Education and relevant state agencies. We are sending copies of this report to the appropriate congressional committees, the Secretary of the Department of Education, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (617) 788-0534 or EmreyArrasM@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix III. Sincerely yours, Melissa Emrey-Arras Director, Education, Workforce, and Income Security Issues Page 33 GAO-15-598 Teacher Preparation Programs Appendix I: Objectives, Scope, and Methodology Appendix I: Objectives, Scope, and Methodology The objectives of this study were to answer the following questions: (1) How do states oversee teacher preparation programs (TPP); (2) What are states and select TPPs doing to prepare new teachers for new K-12 standards; and (3) To what extent does the U.S. Department of Education’s (Education) data reporting and other information-sharing support and encourage high quality TPPs? We used a variety of methods to examine all three objectives. We reviewed relevant federal laws and regulations; federal internal controls standards; and leading practices for program management. We also reviewed documents from Education, select states, and research organizations. We also conducted a web-based survey of the state entities responsible for overseeing TPPs in all 50 states and the District of Columbia (see below for more information about this survey). In addition, we conducted interviews with a wide range of stakeholders including Education officials and contractors; companies responsible for administering state licensing tests; the national accreditation organization for teacher preparation programs; researchers; as well as organizations representing teachers, teacher preparation programs, officials who head state departments of education, and governors. We also conducted case studies in 5 states. In each of these states we interviewed officials from state oversight entities, officials from two or three TPPs, and officials in at least one K-12 school district (see below for more information about these case study interviews). Finally, we analyzed data collected to fulfill Higher Education Act Title II reporting requirements and Education’s related reports and technical assistance guidance (see below for more information about this analysis). We conducted our work from March 2014 to July 2015 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Survey of 50 States and the District of Columbia To identify how states oversee TPPs, modified their practices in response to new K-12 standards, and used Higher Education Act Title II data and Department of Education information about TPPs, we surveyed state oversight entities from all 50 states and the District of Columbia, achieving a 100 percent response rate. The survey was administered from November 2014 to March 2015. The survey used self-administered, Page 34 GAO-15-598 Teacher Preparation Programs Appendix I: Objectives, Scope, and Methodology electronic questionnaires that were posted on the Internet. We sent the survey to the official responsible for submitting each state’s Higher Education Act Title II reporting requirements and requested that this person consult with other officials in order to provide an official state response. We reviewed state responses and followed up by e-mail or telephone with select states for additional clarification and obtained corrected information for our final survey analysis. We also published survey responses in an e-publication supplemental to this report, “Teacher Preparation Programs: Survey of State Entities that Oversee Teacher Preparation Programs, an E-Supplement to GAO-15-598” (GAO-15-599SP, July, 2015). The quality of survey data can be affected by nonsampling error. Nonsampling error includes variations in how respondents interpret questions, respondents’ willingness to offer accurate responses, and data collection and processing errors. We included steps in developing the survey, and collecting, editing, and analyzing survey data, to minimize such nonsampling error. In developing the Web survey, we pretested draft versions of the instrument with state officials in four states and consulted with officials from the Department of Education to check the clarity of the questions and the flow and layout of the survey. On the basis of the pretests, we made revisions to the survey. Further, using a webbased survey also helped remove error in our data collection effort. By allowing state officials to enter their responses directly into an electronic instrument, this method automatically created a record for each state official in a data file and eliminated the errors associated with a manual data entry process. In addition, the program used to analyze the survey data was independently verified to ensure the accuracy of this work. Case Studies To obtain more detailed information about how states oversee teacher preparation; the steps that states and TPPs have taken in response to new K-12 standards; and, state and TPP views about Higher Education Act Title II reporting requirements and Department of Education grants and information-sharing related to TPPs, we conducted case studies of five states: Arizona, California, Maine, Tennessee and Virginia. We selected this non-generalizable sample of states because they represent a variety of sizes (i.e., numbers of TPPs operating in the state) and approaches to overseeing TPPs (including how they use data about graduate effectiveness to evaluate TPP performance) and to reflect different types of college- and career-ready K-12 standards. As part of these case studies, we reviewed state documents and interviewed officials from the state oversight agency, two to three TPPs, and at least Page 35 GAO-15-598 Teacher Preparation Programs Appendix I: Objectives, Scope, and Methodology one K-12 school district. In total, we interviewed 14 TPPs. We ensured that this non-generalizable sample of TPPs varied in a number of ways including whether they were located within or separate from colleges and universities; were alternative route or traditional; and offered in-person or online coursework. We also ensured that the TPPs represented a range of sizes and urban, suburban, and rural locations. We selected K-12 districts that worked with these TPPs, by providing student teaching experiences, hiring graduates, or both. Interviews with state, TPP, and K12 district officials from Arizona, California and Maine were conducted in person and interviews with officials from Tennessee and Virginia were primarily conducted by telephone. Analysis of Higher Education Act Title II Reports To assess the extent that Education’s data reporting supports and encourages high quality TPPs, we reviewed select state Title II reports; Education’s reporting templates for the state reports and institutional and program reports; Education’s December 2014 proposed rule; 1 technical assistance documents developed by Education and its contractor; and Education’s reports, data spreadsheets and web pages that compile data from Title II state reports. We interviewed Education officials and contractors responsible for collecting, analyzing and reporting such data. We also compared Education’s reports, spreadsheets and web pages— including their definitions and descriptions of the appropriate use of reported data—against federal standards for internal controls and leading practices for data-driven program management. 2 We also reviewed the reliability of fields in Education’s Title II data spreadsheets regarding the number of TPPs that states reported identifying as low-performing or at risk of being low-performing by: (1) reviewing agency documents, (2) interviewing relevant agency officials, and (3) testing the accuracy of these reporting fields for a non-generalizable sample of states. We determined that Education’s compilation of state data about TPPs that have been identified as low-performing was sufficiently reliable to include in our report. 1 Teacher Preparation Issues, 79 Fed. Reg. 71,820 (Dec. 3, 2014). 2 GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999). GAO, Managing for Results: Enhancing Agency Use of Performance Information for Management Decision Making, GAO-05-927 (Washington, D.C.: Sept. 9, 2005) and GAO, Results-Oriented Government: GPRA Has Established a Solid Foundation for Achieving Greater Results, GAO-04-38 (Washington, D.C.: Mar. 10, 2004). Page 36 GAO-15-598 Teacher Preparation Programs Appendix II: Comments from the Department of Education Appendix II: Comments from the Department of Education Page 37 GAO-15-598 Teacher Preparation Programs Appendix II: Comments from the Department of Education Page 38 GAO-15-598 Teacher Preparation Programs Appendix II: Comments from the Department of Education Page 39 GAO-15-598 Teacher Preparation Programs Appendix III: GAO Contact and Acknowledgments Appendix III: GAO Contact and Acknowledgments GAO Contact Melissa Emrey-Arras, (617) 788-0534, EmreyArrasM@gao.gov Staff Acknowledgments In addition to the contact named above, Scott Spicer (Assistant Director); Barbara Steel-Lowney and Daren Sweeney (analysts-in-charge); Lucas Alvarez, and Hedieh Fusfield made key contributions to this report. Also contributing to this report were: Deborah Bland, Kate Blumenreich, Joanna Chan, Sarah Cornetto, Holly Dye, Kirsten Lauber, Ashley McCall, Sheila McCoy, Jennifer McDonald, John Mingus, Mimi Nguyen, and Tom James. (131303) Page 40 GAO-15-598 Teacher Preparation Programs GAO’s Mission The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO’s website (http://www.gao.gov). Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to http://www.gao.gov and select “E-mail Updates.” Order by Phone The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, http://www.gao.gov/ordering.htm. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO Connect with GAO on Facebook, Flickr, Twitter, and YouTube. Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. Visit GAO on the web at www.gao.gov. To Report Fraud, Waste, and Abuse in Federal Programs Contact: Website: http://www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov Automated answering system: (800) 424-5454 or (202) 512-7470 Congressional Relations Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 5124400, U.S. Government Accountability Office, 441 G Street NW, Room 7125, Washington, DC 20548 Public Affairs Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, DC 20548 Please Print on Recycled Paper.