Measuring up: Canadian Results of the OECD PISA Study The Performance of Canada’s Youth in Science, Reading and Mathematics 2015 First Results for Canadians Aged 15 Measuring up: Canadian Results of the OECD PISA Study The Performance of Canada’s Youth in Science, Reading, and Mathematics 2015 First Results for Canadians Aged 15 Authors Kathryn O’Grady, Council of Ministers of Education, Canada Marie-Anne Deussing, Employment and Social Development Canada Tanya Scerbina, Council of Ministers of Education, Canada Karen Fung, Council of Ministers of Education, Canada Nadia Muhe, Council of Ministers of Education, Canada The Council of Ministers of Education, Canada (CMEC) was formed in 1967 by the jurisdictional ministers responsible for education to provide a forum in which they could discuss matters of mutual interest, undertake educational initiatives cooperatively, and represent the interests of the provinces and territories with national educational organizations, the federal government, foreign governments, and international organizations. CMEC is the national voice for education in Canada and, through CMEC, the provinces and territories work collectively on common objectives in a broad range of activities at the elementary, secondary, and postsecondary levels. Through the CMEC Secretariat, the Council serves as the organization in which ministries and departments of education undertake cooperatively the activities, projects, and initiatives of particular interest to all jurisdictions. One of the activities on which they cooperate is the development and implementation of pan-Canadian testing based on contemporary research and best practices in the assessment of student achievement in core subjects. Note of appreciation The Council of Ministers of Education (Canada) would like to thank the students, teachers, and administrators whose participation in the Programme for International Student Assessment ensured its success. The quality of your commitment has made this study possible. We are truly grateful for your contribution to a pan-Canadian understanding of educational policy and practices in science, reading, and mathematics of 15-year-olds. Council of Ministers of Education, Canada 95 St. Clair West, Suite 1106 Toronto, Ontario M4V 1N6 Telephone: (416) 962-8100 Fax: (416) 962-2800 E-mail: cmec@cmec.ca © 2016 Council of Ministers of Education, Canada ISBN 978-0-88987-239-4 Ce rapport est également disponible en français. Table of Contents Introduction.................................................................................................................7 The Programme for International Student Assessment............................................................. 7 Why did Canada participate in PISA? ......................................................................................... 8 What is PISA 2015?..................................................................................................................... 8 Objectives of the report............................................................................................................ 11 Chapter 1: Canadian Students’ Performance in Science in an International Context......................................................................13 Defining science........................................................................................................................ 13 PISA achievement results by proficiency levels in science....................................................... 16 Canadian students achieve a high level of proficiency in science............................................ 18 PISA achievement results by average scores in science........................................................... 19 Canadian students perform well in science in a global context............................................... 19 There are marked variations between provinces .................................................................... 22 Canadian results in science are characterized by relatively high levels of equity.................... 24 In Canada, science results show significant differences by the school system’s language ..... 25 There is no gender gap in science in Canada overall................................................................ 26 The mean performance of Canadian students in science has remained stable over time...... 29 Summary................................................................................................................................... 30 Chapter 2: Canadian Students’ Reading and Mathematics Performance in an International Context......................................................................31 Defining reading and mathematics........................................................................................... 31 Canadian students perform well in reading and mathematics in a global context................. 32 Most provinces performed at or above the OECD average in reading and mathematics....... 35 Across Canada, differences in reading and mathematics performance are seen between students attending majority-language school systems and those attending minority-language systems in reading and mathematics......................................................... 37 Canadian girls outperformed boys in reading, while Canadian boys outperformed girls in mathematics.......................................................................................................................... 38 Canadian students’ performance in reading remained relatively stable over time while performance in mathematics stabilized between 2012 and 2015................................. 39 Summary................................................................................................................................... 41 Conclusion.................................................................................................................43 Overview of results................................................................................................................... 43 Final statement......................................................................................................................... 45 Appendix A: PISA 2015 sampling procedures, exclusion rates, and response rates...47 Appendix B: PISA 2015 data tables............................................................................52 3 List of Tables Introduction.................................................................................................................7 Table 1 Overview of PISA 2015..................................................................................... 10 Chapter 1: Canadian Students’ Performance in Science in an International Context......................................................................13 Table 1.1 Table 1.2 Table 1.3 Table 1.4 Table 1.5 Table 1.6 Table 1.7 Table 1.8 Table 1.9 Table 1.10 Table 1.11 Competencies of science.................................................................................. 14 Content knowledge of science......................................................................... 15 Procedural and epistemic knowledge of science............................................. 16 PISA 2015 Science proficiency levels — Summary description....................... 17 Countries performing better than or as well as Canada – Science.................. 22 Provincial results in science relative to the Canadian average........................ 23 Estimated average overall science scores, by province and language of the school system......................................................................................... 25 Summary of differences in provincial results between language systems in science competency, knowledge, and content area subscales................... 26 Distribution of students on the overall science scale by proficiency level and gender............................................................................................... 27 Summary of gender differences in average science scores for Canada and the provinces............................................................................................. 28 Comparison of performance in science in PISA 2006–2015, Canada and the provinces............................................................................................. 30 Chapter 2: Canadian Students’ Reading and Mathematics Performance in an International Context..............................................................................31 Table 2.1 Table 2.2 Table 2.3 Table 2.4 Table 2.5 Table 2.6 Countries performing better than or as well as Canada in reading and mathematics.............................................................................................. 32 Provincial results in reading and mathematics relative to the Canadian average............................................................................................. 35 Estimated average reading and mathematics scores, by province and language of the school system......................................................................... 37 Summary of gender differences in average reading and mathematics scores for Canada and the provinces............................................................... 38 Comparison of performance in reading in PISA 2009, 2012, and 2015, Canada and the provinces................................................................................ 40 Comparison of performance in mathematics in PISA 2012 and 2015, Canada and the provinces................................................................................ 40 Appendix A: PISA 2015 sampling procedures, exclusion rates, and response rates...45 Table A.1a Table A.1b Table A.2 4 PISA 2015 student exclusion rate..................................................................... 48 PISA 2015 student exclusion rate by type of exclusion.................................... 48 PISA 2015 school and student response rates................................................. 51 Appendix B: PISA 2015 data tables............................................................................52 Table B.1.1 Table B.1.2 Table B.1.3 Table B.1.4 Table B.1.5 Table B.1.6 Table B.1.7 Table B.1.8 Table B.1.9 Table B.1.10 Table B.1.11 Table B.1.12 Table B.1.13 Table B.1.14 Table B.1.15 Table B.1.16 Table B.1.17 Table B.1.18 Table B.2.1 Table B.2.2 Table B.2.3 Table B.2.4 Percentage of students at each proficiency level for countries, economies, and provinces: SCIENCE................................................................ 52 Estimated average scores and confidence intervals for countries, economies, and provinces: SCIENCE................................................................ 54 Estimated average scores and confidence intervals for Canada and the provinces: SCIENCE BY COMPETENCY SUBSCALES.................................... 55 Estimated average scores and confidence intervals for Canada and the provinces: SCIENCE BY KNOWLEDGE SUBSCALES..................................... 56 Estimated average scores and confidence intervals for Canada and the provinces: SCIENCE BY CONTENT SUBSCALES........................................... 57 Variation in student performance for countries, economies, and provinces: SCIENCE.................................................................................... 58 Estimated average scores by language of the school system for Canada and the provinces: SCIENCE............................................................................. 60 Estimated average scores by language of the school system for Canada and the provinces: SCIENCE BY COMPETENCY SUBSCALES............................. 61 Estimated average scores by language of the school system for Canada and the provinces: SCIENCE BY KNOWLEDGE SUBSCALES.............................. 62 Estimated average scores by language of the school system for Canada and the provinces: SCIENCE BY CONTENT SUBSCALES.................................... 63 Estimated average scores by gender for Canada and the provinces: SCIENCE............................................................................. 64 Proportion of males and females who performed below Level 2 and at Levels 5 and 6, PISA 2015, Canada and the provinces: SCIENCE.......... 64 Estimated average scores by gender for Canada and the provinces: SCIENCE BY COMPETENCY SUBSCALES.................................... 65 Estimated average scores by gender for Canada and the provinces: SCIENCE BY KNOWLEDGE SUBSCALES..................................... 66 Estimated average scores by gender for Canada and the provinces: SCIENCE BY CONTENT SUBSCALES........................................... 67 Comparisons of performance, PISA 2006, 2009, 2012, and 2015, Canada and the provinces: SCIENCE................................................................ 68 Proportion of students who performed below Level 2 and at Levels 5 and 6, in PISA 2006 and 2015, Canada and the provinces: SCIENCE .............. 68 Gender differences in student performance, PISA 2006 and 2015, Canada and the provinces: SCIENCE................................................................ 69 Estimated average scores and confidence intervals for countries, economies, and provinces: READING.............................................................. 70 Estimated average scores and confidence intervals for provinces, countries and economies: MATHEMATICS ...................................................... 71 Variation in student performance for countries, economies, and provinces: READING.................................................................................. 72 Variation in student performance for countries, economies, and provinces: MATHEMATICS ........................................................................ 74 5 Table B.2.5 Table B.2.6 Table B.2.7 Table B.2.8 Table B.2.9a Table B.2.9b Table B.2.10a Table B.2.10b Table B.3.1 Estimated average scores by language of the school system for Canada and the provinces: READING......................................................... 76 Estimated average scores by language of the school system for Canada and the provinces: MATHEMATICS................................................ 76 Estimated average scores by gender for Canada and the provinces: READING................................................................................... 77 Estimated average scores by gender for Canada and the provinces: MATHEMATICS.......................................................................... 77 Comparisons of performance, PISA 2000, 2003, 2006, 2009, and 2012, Canada and the provinces: READING............................................. 78 Comparisons of performance, PISA 2009 and 2012, Canada and the provinces: READING............................................................................ 78 Comparisons of performance, PISA 2003, 2006, 2009, and 2012, Canada and the provinces: MATHEMATICS..................................................... 79 Comparisons of performance, PISA 2012, Canada and the provinces: MATHEMATICS.......................................................................... 79 Multiple comparisons of achievement for countries, economies, and provinces: SCIENCE.................................................................................... 80 List of Figures Chapter 1: Canadian Students’ Performance in Science in an International Context......................................................................13 Figure 1 Main features of the PISA 2015 science framework........................................... 14 Figure 1.1 Distribution of students by proficiency level on the overall science scale — Canada, provinces, and OECD............................................................................. 18 Figure 1.2 Estimated average scores and confidence intervals for countries and provinces: Science............................................................................................... 20 Figure 1.3 PISA 2015 Science – Difference between high and low achievers, Canada, provinces, and OECD........................................................................................... 24 Figure 1.4 PISA Canadian results over time, 2006–2015 science overall............................ 29 Chapter 2: Canadian Students’ Reading and Mathematics Performance in an International Context..............................................................................31 Figure 2.1 Estimated average scores and confidence intervals for countries and provinces: Reading....................................................................................... 33 Figure 2.2 Estimated average scores and confidence intervals for countries and provinces: Mathematics............................................................................... 34 Figure 2.3 PISA 2015 Reading: Difference between high and low achievers, Canada, provinces, and OECD..... 36 Figure 2.4 PISA 2015 Mathematics: Difference between high and low achievers, Canada, provinces, and OECD..... 36 6 Introduction The skills and knowledge that individuals bring to their jobs, to further studies, and to our society play an important role in determining our economic success and our overall quality of life. Today’s knowledge-based economy is driven by advances in information and communication technologies, by reduced trade barriers, and by the globalization of markets that have changed the type of knowledge and skills that the future economy requires. There is a demand for a strong set of foundational skills upon which further learning can be built. Education systems play a central role in building this strong base. Students leaving secondary education without a strong foundation may experience difficulty accessing the postsecondary education system or the labour market and they may benefit less when learning opportunities are presented later in life. Without the tools needed to be effective learners throughout their lives, these individuals with limited skills risk economic and social marginalization. Governments in industrialized countries have devoted large portions of their budgets to provide high-quality schooling. Given these investments, they are interested in the relative effectiveness of their education systems. To address these issues, member countries of the Organisation for Economic Co-operation and Development (OECD), along with partner countries and economies,1 developed a common tool to improve their understanding of what makes young people — and entire education systems — successful. This tool is the Programme for International Student Assessment (PISA). It measures the extent to which youth, at age 15, have acquired some of the knowledge and skills that are essential for full participation in modern societies. The Programme for International Student Assessment PISA is a collaborative effort among member countries of the OECD. PISA is designed to provide policyoriented international indicators of the skills and knowledge of 15-year-old students and to shed light on a range of factors that contribute to successful students, schools, education systems, and learning environments.2 It measures skills that are generally recognized as key outcomes of the educational process. The assessment focuses on young people’s ability to use their knowledge and skills to meet real-life challenges. These skills are believed to be prerequisites for efficient learning in adulthood and for full participation in society. Information gathered through PISA enables a thorough comparative analysis of the performance of students near the end of their compulsory education. PISA also permits exploration of the ways that achievement varies across different social and economic groups and the factors that influence achievement within and among countries. Over the past decade, PISA has brought significant public and educational attention to international assessments and related studies by generating data to enhance policy-makers’ ability to formulate decisions based on evidence. Canadian provinces have used information gathered from PISA, along with other sources of information such as the Pan-Canadian Assessment Program (PCAP),3 other international assessments, as well as their own provincial assessment programs, to inform various education-related initiatives. In Canada, PISA is carried out through a partnership between Employment and Social Development Canada (ESDC) and the Council of Ministers of Education, Canada (CMEC). The word countries will be used to denote countries and economies. OECD, PISA 2015 assessment and analytical framework: Science, reading, mathematic and financial literacy (Paris: OECD, 2016). Available at http:// dx.doi.org/10.1787/9789264255425-en. 3 Council of Ministers of Education, Canada, PCAP-13 2007 report on the assessment of 13-year-olds in reading, mathematics, and science (Toronto: CMEC, 2008). 1 2 7 The project began in 2000 and focuses on the capabilities of 15-year-olds as they near the end of compulsory education. It reports on scientific, mathematic, and reading literacy every three years and provides a more detailed look at one of those domains in the years when it is the major focus. Why did Canada participate in PISA? Canada’s continued participation in PISA stems from many of the same questions that motivate other participating countries and economies. In Canada, provinces and territories responsible for education invest significant public resources in the provision of elementary and secondary education and Canadians are interested in the outcomes of compulsory education provided to their youth. How can resources be directed to the achievement of higher levels of knowledge and skills upon which lifelong learning is founded and to potentially reduce social inequality in life outcomes? Elementary and secondary education systems play a key role in providing students with the knowledge and skills that form an essential foundation necessary to further develop human capital — either through participation in the workforce, postsecondary education, or lifelong learning. Previous studies based on PISA data have shown the relationship between strong skills in the core subject areas at age 15 and outcomes in later life. For example, results from the Youth in Transition Survey (YITS) show that there is a strong association between reading proficiency and education attainment.4 Canadian students in the bottom quartile of PISA reading scores were much more likely to drop out of secondary school and less likely to have completed a year of postsecondary education than those in the high quartile of reading score. In contrast, Canadian students in the top PISA level (Level 5) of reading performance were twenty times more likely to go to university that those in the lowest PISA level (at or below Level 1).5 Questions about educational effectiveness can be partly answered with data on the average performance of Canada’s youth in key subject areas. However, two other questions with respect to equity can be answered only by examining the distribution of competencies: who are the students at the lowest levels of achievement? Do certain groups or regions appear to be at greater risk? These are important questions because, among other things, acquisition of knowledge and skills during compulsory schooling influences access to postsecondary education, eventual success in the labour market, and the effectiveness of continuous, lifelong learning. What is PISA 2015? In 2015 the sixth cycle of PISA was completed and it focuses on scientific literacy. While science was also assessed in previous PISA cycles, the domain was the major focus only in 2006. Students who participated in PISA 2015 entered primary school at about the same time as the PISA 2006 survey so the 2015 results provide an opportunity to relate policy changes to changes in learning outcomes using the benchmarks set by the original 2006 survey when science was also the major focus of assessment. With an emphasis on science in 2015, PISA reports on scientific literacy as well as three “competency” subscales related to explaining phenomena scientifically, evaluating and designing scientific enquiry, and interpreting data and evidence scientifically. Comparing country performance is based on knowledge of science content as well as procedural and epistemic knowledge of science. “Content knowledge” refers to knowledge of facts, concepts, ideas, and theories about the natural world that science has established, while “procedural knowledge” refers to the knowledge of the practices and concepts on which empirical enquiry is based. “Epistemic knowledge” refers to an understanding of the role of specific constructs and defining features essential to the process of knowledge building in science. The three main areas OECD, Pathways to success: How knowledge and skills at age 15 shape future lives in Canada (Paris: OECD, 2010); OECD, Learning beyond fifteen: Ten years after PISA (Paris: OECD, 2012). Available at https://www.oecd.org/canada/49893598.pdf. 5 OECD, Pathways to success. 4 8 of science knowledge are physical systems, living systems, and Earth and space systems.6 As minor domains in PISA 2015, reading and mathematics are measured at only an overall, rather than detailed, level and as such are not reported by performance level or subscales. PISA 2015 also includes a collaborative problem solving and a financial literacy assessment for those countries that decided to participate in the computer-based assessment.7 Recognizing the pervasiveness of computer-based tools in the workplace and in everyday life in the 21st century, PISA 2015 assessed all subjects for the first time via computer, although paper-based assessment instruments were provided for countries that chose not to test their students by computer, albeit for reading, mathematics, and science trend items only. Prior to PISA 2015, the assessment was implemented through a paper-based format although the 2009 reading framework and the 2012 mathematics and problem-solving frameworks included electronic assessments and expanded the definition of “PISA literacies” beyond what can be measured by a traditional paper-and-pencil test. Seventy-two countries participated in PISA 2015, including all 35 OECD countries.8 Between 5,000 and 10,000 students aged 15 from at least 150 schools were typically tested in each country. In Canada, approximately 20,000 15-year-olds from about 900 schools participated across the ten provinces.9 The large Canadian sample was required to produce reliable estimates representative of each province and for both French- and English-language school systems in Nova Scotia, New Brunswick, Quebec, Ontario, Manitoba, Alberta, and British Columbia. PISA was administered in English and in French according to the respective school system. The 2015 PISA assessment was administered in schools during regular school hours in April and May 2015. The assessment was a two-hour computer-based test. Students also completed a 35-minute student background questionnaire providing information about themselves and their home, while school principals completed a 20-minute questionnaire about their schools. As part of PISA 2015, international options could also be implemented. Canada chose to add a one-hour financial literacy assessment as well as a five-minute paper-based questionnaire to collect information on the attitudes of 15-year-old students toward trades; however, only some provinces chose to participate in these options. An overview of PISA 2015 is given in the table below. It includes information on participants, test design and administration, and national and international options. OECD, PISA 2015 assessment and analytical framework, p. 19. Results of the collaborative problem-solving and financial literacy components will be released in 2017. 8 OECD countries include Australia, Austria, Belgium, Canada, Chile, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Israel, Italy, Japan, Korea, Latvia, Luxembourg, Mexico, the Netherlands, New Zealand, Norway, Poland, Portugal, Slovak Republic, Slovenia, Spain, Sweden, Switzerland, Turkey, United Kingdom, and United States. Partner countries and economies are: Albania, Algeria, Argentina, Beijing, Shanghai, Jiangsu, Guangdong (BSJG)–China, Brazil, Bulgaria, Chinese Taipei, Colombia, Costa Rica, Croatia, Cyprus, Dominican Republic, Georgia, Hong Kong–China, Indonesia, Jordan, Kazakhstan, Kosovo, Lebanon, Lithuania, Macao–China, Malaysia, Malta, Moldova, Montenegro, Peru, Qatar, Republic of Macedonia, Romania, Russian Federation, Singapore, Thailand, Trinidad and Tobago, Tunisia, United Arab Emirates, Uruguay, and Vietnam. 9 No data were collected in the three territories or in First Nations schools. Further information on sampling procedures and response rates for Canada can be found in Appendix A. 6 7 9 Table 1 Overview of PISA 2015 International Canada Participating countries/ provinces • 72 countries and economies • 10 provinces Population • Youth aged 15 • Same Number of participating students • Between 5,000 and 10,000 per country with some exceptions for a total of around 510,000 students • Approximately 20,000 students Domains • Major: science • Minor: reading and mathematics • Computer-based collaborative problem solving • Same Languages in which the test was administered • 47 languages • English and French International assessment • Two hours of direct assessments of science, reading, mathematics, and collaborative problem solving • Thirty-five-minute contextual questionnaire administered to youth • Twenty-minute school questionnaire administered to school principals • Same International options • Ten-minute optional questionnaire on information technology and communications familiarity administered to students • Ten-minute optional questionnaire on educational career administered to students • Twenty-minute optional questionnaire administered to parents • One-hour optional assessment of financial literacy • Thirty-minute optional teacher questionnaire • One-hour optional assessment of financial literacy in Newfoundland and Labrador, Prince Edward Island, Nova Scotia, New Brunswick, Ontario, Manitoba, and British Columbia. National options • Other options were undertaken in a limited number of countries • Five minutes of additional questions administered to students regarding their attitudes towards trades in Newfoundland and Labrador, Prince Edward Island, New Brunswick-English sector, Manitoba, Saskatchewan, and British Columbia. 10 Objectives of the report This report provides the initial results from the PISA 2015 assessment for Canada and the provinces. It presents the national and provincial results in science, reading, and mathematics and complements the information presented in the PISA 2015 International report.10 It also compares results to other participating countries and economies and across Canadian provinces. Chapter 1 provides information on the performance of Canadian 15-year-old students on the PISA 2015 assessment in science. Chapter 2 presents results on the performance of Canada and the provinces in the minor domains of reading and mathematics. The major findings and opportunities for further study are discussed in the conclusion. The PISA 2015 International report is released in two volumes. Results presented in this report correspond to results presented in PISA 2015 Results: Excellence and Equity in Education, Volume I (Paris: OECD 2016). 10 11 Chapter 1 Canadian Students’ Performance in Science in an International Context The results of student performance on the science assessment are presented in this report in two ways: as the percentage of students attaining proficiency levels and as overall average scores. The performance of 15-yearolds for science overall is described in terms of seven PISA proficiency levels for Canada and the provinces. The average scores for science overall are then compared to those from the other countries and economies that participated in PISA 2015. Results are presented for Canada overall and by province, both for science overall and by the subscales of science (competencies and knowledge areas). Then the performance of students enrolled in anglophone and francophone school systems is presented for those provinces in which the two groups were sampled separately. This chapter also compares Canadian students’ performance in science by gender. Given that science was assessed as a major domain for a second time in PISA (the first time was in 2006), change in science performance over time will also be discussed. Defining science Science education in primary and secondary school should prepare students so that by the time they leave school they can understand and engage in discussions about the science and technology-related issues that shape our world. They should also have the skills needed to participate in higher education in fields related to science if they wish to. Most current curricula for science education are designed on the assumption that an understanding of science is so important that the subject should be a central feature in every young person’s education.11 In the PISA context, science refers to “scientific literacy” which is defined as the ability to engage with sciencerelated issues, and with the ideas of science, as a reflective citizen. A scientifically literate person is willing to engage in reasoned discourse about science and technology, which requires the competencies to: explain phenomena scientifically, evaluate and design scientific enquiry, and interpret data and evidence scientifically.12 The science framework was originally developed for PISA 2006 and has kept its essential features in 2015 which allows participating countries to report on trends in performance over time. However, two major improvements were made to the 2006 framework: 1) “knowledge about science” has been defined more clearly and split into two components — procedural knowledge and epistemic knowledge; and 2) the move from a paper-based to a computer-based assessment. These two elements do not jeopardize the possibility of reporting on trends in science performance because they expand the information already available in PISA 2006. For PISA assessment purposes, the domain of science is divided into three competencies (explain phenomena scientifically, evaluate and design scientific enquiry, and interpret data and evidence scientifically); two knowledge types (content and procedural/epistemic); and three areas of knowledge of science (physical systems, living systems, and Earth and space systems). PISA 2015 also measured students’ interest in and awareness of science and environmental issues as well as their perceived value of scientific approaches. OECD, PISA 2015 Results: Excellence and Equity in Education, Volume 1 (Paris: OECD, 2016) OECD, PISA 2015 assessment and analytical framework: Science, reading, mathematic and financial literacy, (Paris: OECD, 2016), available at http:// dx.doi.org/10.1787/9789264255425-en. 11 12 13 The main features of the PISA 2015 science framework are presented in the following illustration.13 Figure 1 Main features of the PISA 2015 science framework Competencies Contexts ▪ Personal ▪ Local/national ▪ Global Requires individuals to display ▪ Explain phenomena scientifically ▪ Evaluate and design scientific enquiry ▪ Interpret data and evidence scientifically How an individual does this is influenced by Attitudes ▪ Interest in science ▪ Valuing scientific approaches to enquiry ▪ Environmental awareness Knowledge ▪ Content ▪ Procedural ▪ Epistemic Three competencies are used in PISA 2015 to describe how a scientifically literate person engages in issues and ideas related to science. The competencies appear in Table 1.1. Table 1.1 Competencies of science Explain phenomena scientifically — being able to recognize, offer, and evaluate explanations for a range of natural and technological phenomena by demonstrating the ability to: • • • • • Recall and apply appropriate scientific knowledge. Identify, use, and generate explanatory models and representations. Make and justify appropriate predictions. Offer explanatory hypotheses. Explain the potential implications of scientific knowledge for society. Evaluate and design scientific enquiry — being able to describe and appraise scientific investigations and propose ways of addressing questions scientifically by demonstrating the ability to: • • • • • Identify the question explored in a given scientific study. Distinguish questions that could be investigated scientifically. Propose a way of exploring a given question scientifically. Evaluate ways of exploring a given question scientifically. Describe and evaluate how scientists ensure the reliability of data, and the objectivity and generalizability of explanations. Interpret data and evidence scientifically — being able to analyze and evaluate data, claims, and arguments in a variety of representations and draw appropriate scientific conclusions by demonstrating the ability to: • • • • • Transform data from one representation to another. Analyze and interpret data and draw appropriate conclusions. Identify the assumptions, evidence, and reasoning in science-related texts. Distinguish between arguments that are based on scientific evidence and theory and those based on other considerations. Evaluate scientific arguments and evidence from different sources (e.g., newspapers, the Internet, journals).* * Adapted from Figures 2.4 a, b, and c in OECD, PISA 2015 assessment and analytical framework. OECD, PISA 2015 assessment and analytical framework, p. 23. 13 14 Each of the scientific competencies requires some content knowledge (knowledge of theories, explanatory ideas, information, and facts), but also an understanding of how such knowledge has been derived (procedural knowledge) and of the nature of that knowledge (epistemic knowledge). For PISA 2015, content knowledge was classified according to the three broad content areas central to the disciplines. Although their definitions and delineations may vary, these are very consistent with the way provincial curricula14 as well as pan-Canadian15 and other international assessments16 are organized. Descriptions of these content areas appear in Table 1.2. Table 1.2 Content knowledge of science Physical systems that require knowledge of: • • • • • • structure of matter (e.g., particle model, bonds) properties of matter (e.g., changes of state, thermal and electrical conductivity) chemical changes of matter (e.g., chemical reactions, energy transfer, acids/bases) motion and forces (e.g., velocity, friction) and action at a distance (e.g., magnetic, gravitational, and electrostatic forces) energy and its transformation (e.g., conservation, dissipation, chemical reactions) interactions between energy and matter (e.g., light and radio waves, sound and seismic waves) Living systems that require knowledge of: • cells (e.g., structures and function, DNA, plant and animal) • the concept of an organism (e.g., unicellular and multicellular) • humans (e.g., health; nutrition; subsystems such as digestion, respiration, circulation, excretion, and reproduction and their relationship) • populations (e.g., species, evolution, biodiversity, genetic variation) • ecosystems (e.g. food chains, matter, and energy flow) • biosphere (e.g., ecosystem services, sustainability) Earth and space systems that require knowledge of: • • • • • • structures of the Earth systems (e.g., lithosphere, atmosphere, hydrosphere) energy in the Earth systems (e.g., sources, global climate) change in Earth systems (e.g., plate tectonics, geochemical cycles, constructive and destructive forces) Earth’s history (e.g., fossils, origin, and evolution) Earth in space (e.g., gravity, solar systems, galaxies) the history and scale of the universe and its history (e.g., light year, Big Bang theory).* * Adapted from Figure 2.5 in OECD, PISA 2015 assessment and analytical framework. For the reporting of knowledge in PISA 2015, procedural knowledge and epistemic knowledge were combined into one category. The examples listed in the table below convey the general features of the types of procedural and epistemic knowledge addressed in the assessment. For updated science curricula, please visit official jurisdictional Web sites. See K. O’Grady, and K. Hume, PCAP 2013: Report on the pan-Canadian assessment of science, reading, and mathematics (Toronto: Council of Ministers of Education, Canada, 2014). 16 See I. Mullis, M. Martin, G. Ruddock, C. O’Sullivan, and C. Preuschoff. TIMSS assessment frameworks (Chestnut Hill, MA: Boston College, 2009), available at http://timssandpirls.bc.edu/timss2015/frameworks.html. 14 15 15 Table 1.3 Procedural and epistemic knowledge of science Procedural knowledge requires an understanding of how scientific knowledge is derived. It includes: • the concept of variables, including dependent, independent, and control variables; • concepts of measurement, for example, quantitative (measurements), qualitative (observations), the use of a scale, categorical and continuous variables; • ways of assessing and minimizing uncertainty, such as repeating and averaging measurements; • mechanisms to ensure the replicability (closeness of agreement between repeated measures of the same quantity) and accuracy of data (the closeness of agreement between a measured quantity and a true value of the measure); • common ways of abstracting and representing data using tables, graphs, and charts, and using them appropriately; • the control-of-variables strategy and its role in experimental design or the use of randomized controlled trials to avoid confounded findings and identify possible causal mechanisms; and • the nature of an appropriate design for a given scientific question, for example, experimental, field-based, or patternseeking. Epistemic knowledge requires an understanding of the nature of knowledge in science. It involves the constructs and defining features of science: • the nature of scientific observations, facts, hypotheses, models, and theories; • the purpose and goals of science (to produce explanations of the natural world) as distinguished from technology (to produce an optimal solution to human need), and what constitutes a scientific or technological question and appropriate data; • The values of science, for example, a commitment to publication, objectivity, and the elimination of bias; and • the nature of reasoning used in science, for example, deductive, inductive, inference to the best explanation (abductive), analogical, and model-based. Epistemic knowledge requires a recognition of these constructs’ and features’ role in justifying the knowledge produced by science. That is: • how scientific claims are supported by data and reasoning in science; • the function of different forms of empirical enquiry in establishing knowledge, their goal (to test explanatory hypotheses or identify patterns), and their design (observation, controlled experiments, correlational studies); • how measurement error affects the degree of confidence in scientific knowledge; • the use and role of physical, system, and abstract models and their limits; • the role of collaboration and critique, and how peer review helps to establish confidence in scientific claims; and • the role of scientific knowledge, along with other forms of knowledge, in identifying and addressing societal and technological issues.* * Adapted from Figures 2.6 and 2.7 in OECD, PISA 2015 assessment and analytical framework. PISA achievement results by proficiency levels in science PISA developed useful benchmarks relating a range of average scores in science to levels of knowledge and skills that are measured by the assessment. Although these levels are not linked directly to any specific program of study in science, they provide an overall picture of students’ accumulated understanding at age 15. PISA science literacy is expressed on a seven-level proficiency scale in which tasks at the lower end of the scale (Level 1) are deemed easier and less complex than other tasks at the higher end (Level 6). This progression in task difficulty/ complexity applies to both the overall science scale and for each competency and knowledge area. Table 1.4 provides a summary description of the tasks that students are able to do at the seven proficiency levels for overall science along with the corresponding lower limit for the level. It is assumed that students classified at a given proficiency level can perform most of that tasks at that level as well as those at the lower levels. Proficiency level achievement is reported for only the major domain in the Canadian report of the PISA 2015 assessment. 16 Table 1.4 PISA 2015 Science proficiency levels — Summary description* Percentage of students able to perform tasks at this level or above Level Lower score limit 6 707.93 1.1% of students across the OECD and 2.0% in Canada Students at Level 6 of the PISA science assessment are able to successfully complete the most difficult PISA items. At Level 6, students can: • draw on a range of interrelated scientific ideas and concepts from the physical, life, Earth, and space sciences, link different information sources and representations, and move flexibly among them; • use content, procedural, and epistemic knowledge to offer explanatory hypotheses of novel scientific phenomena, events, and processes or to make predictions; • discriminate between relevant and irrelevant information and draw on knowledge external to the normal school curriculum when interpreting data and evidence; • distinguish between arguments that are based on scientific evidence and theory and those based on other considerations; and • evaluate competing designs of complex experiments, field studies, or simulations and justify their choices. 5 633.33 7.7% of students across the OECD and 12.4% in Canada At Level 5, students can: • use abstract scientific ideas or concepts to explain unfamiliar and more complex phenomena, events, and processes involving multiple causal links; • apply more sophisticated epistemic knowledge to evaluate alternative experimental designs and justify their choices and use theoretical knowledge to interpret information or make predictions; and • evaluate ways of exploring a given question scientifically and identify limitations in interpretations of data sets, including sources and the effects of uncertainty in scientific data. 4 558.73 26.7% of students across the OECD and 38.4% in Canada At Level 4, students can: • use more complex or more abstract content knowledge, which is either provided or recalled, to construct explanations of more complex or less familiar events and processes; • conduct experiments involving two or more independent variables in a constrained context; • justify an experimental design, drawing on elements of procedural and epistemic knowledge; and • interpret data drawn from a moderately complex data set or less familiar context, draw appropriate conclusions that go beyond the data, and provide justifications for their choices. 3 484.14 54.0% of students across the OECD and 68.7% in Canada At Level 3, students can: • draw upon moderately complex content knowledge to identify or construct explanations of familiar phenomena; • construct explanations with relevant cueing or support in less familiar or more complex situations; • draw on elements of procedural or epistemic knowledge to carry out a simple experiment in a constrained context; and • distinguish between scientific and nonscientific issues and identify the evidence supporting a scientific claim. 2 409.54 78.8% of students across the OECD and 88.9% in Canada Level 2 is considered the baseline level of science proficiency that is required to participate fully in modern society. At Level 2, students can: • draw on everyday content knowledge and basic procedural knowledge to identify an appropriate scientific explanation, interpret data, and identify the question being addressed in a simple experimental design; • use basic or everyday scientific knowledge to identify a valid conclusion from a simple data set; and • demonstrate basic epistemic knowledge by being able to identify questions that could be investigated scientifically. 1a 334.94 94.5% of students across the OECD and 98.0% in Canada At Level 1a, students can: • use basic or everyday content and procedural knowledge to recognize or identify explanations of simple scientific phenomenon; • undertake structured scientific enquiries with no more than two variables with support; • identify simple causal or correlational relationships and interpret graphical and visual data that require a low level of cognitive demand; and • select the best scientific explanation for given data in familiar personal, local, and global contexts. 1b 260.54 99.4% of students across the OECD and 99.9% in Canada At Level 1b, students can: • use basic or everyday scientific knowledge to recognize aspects of familiar or simple phenomenon; and • identify simple patterns in data, recognize basic scientific terms, and follow explicit instructions to carry out a scientific procedure. Task characteristics * Adapted from OECD, PISA 2015 Results: Excellence and Equity in Education. Note: Level 1 and Level 1a are used interchangeably. Level 1b is also referred to as below Level 1. 17 Canadian students achieve a high level of proficiency in science In PISA 2015, 89 per cent of Canadian students and 79 per cent of students in OECD countries performed at or above Level 2 in science, which is the baseline level of science proficiency (Appendix B.1.1). Across provinces, the percentage of Canadian students at or above the baseline level of performance ranges from 83 per cent in Saskatchewan and Manitoba to over 90 per cent in Quebec, Alberta, and British Columbia (Figure 1.1). By contrast, 11 per cent of Canadian students did not reach the baseline Level 2 in science, compared with 21 per cent for the OECD. More than 60 countries had a higher proportion of students performing at the lower level compared to Canada. Provincially, there is a lot of variability among the provinces. Quebec (8%), Alberta (9%), and British Columbia (9%) had a lower proportion of low achievers; Manitoba and Saskatchewan had a higher (17%) proportion of low achievers in science. At the higher end of the PISA science scale, 12 per cent of Canadian students performed at Level 5 or above compared to 8 per cent performing at this level for the OECD. Although this is a higher proportion of students than in most other countries participating in PISA, seven countries and economies (Beijing, Shanghai, Jiangsu, Guangdong [BSJG]–China, Estonia, New Zealand, Australia, the Netherlands, United Kingdom, and Korea) had a similar proportion of students performing at Level 5 or above as Canada did, while four had a statistically higher proportion (Singapore, Chinese Taipei, Japan, and Finland). At the provincial level, the proportion of students achieving at this higher level is 10 per cent or more in Nova Scotia, Quebec, Ontario, Alberta, and British Columbia. Across the OECD, 6 per cent of 15-year-olds did not achieve Level 1 while this proportion was 2 per cent in Canada. Provincially, 4 per cent of students in Manitoba did not achieve Level 1, compared to 1 percent of students in Quebec, Alberta, and British Columbia. Figure 1.1 Distribution of students by proficiency level on the overall science scale — Canada, provinces, and OECD Newfoundland and Labrador Prince Edward Island Nova Scotia New Brunswick Quebec Ontario Manitoba Saskatchewan Alberta British Columbia Canada OECD Below Level 2 Level 2 Level 3 Level 4 Level 5 and 6 Note: Percentages may not add up to 100 due to rounding. Results for the province of Quebec in this table should be treated with caution because of a possible non-response bias (see Appendix A for further details). 18 PISA achievement results by average scores in science The PISA scores for science are expressed on a scale with an average or mean of 500 points for the OECD countries and a standard deviation of 100. This average was established in 2006 and reestablished at 493 in 2015.17 This means that approximately two-thirds of all students in OECD countries scored between 393 and 593 (i.e., within one standard deviation of the average) on this PISA 2015 assessment. International studies such as PISA summarize student performance by comparing the relative standing of countries based on their average test scores. This approach can be misleading because there is a margin of error associated with each score (see note below). When interpreting average performances, only those differences between countries that are statistically significant should be taken into account. A note on statistical comparisons Because PISA’s goal is to report results on the skills of 15-year-old students, a random sample of 15-yearold students was selected to complete PISA. The averages (for mean scores and for proficiency-levels proportions) were computed from the scores of random samples of students from each country and not from the population of students in each country. Consequently, it cannot be said with certainty that a sample average has the same value as the population average that would have been obtained had all 15-year-old students been assessed. A degree of error is associated with the scores describing student performance because these scores are estimated based on student responses to test items. A statistic, called the standard error, is used to express the degree of uncertainty associated with sampling error and measurement error. The standard error can be used to construct a confidence interval that provides a means of making inferences about the population averages and proportions in a manner that reflects the uncertainty associated with sample estimates. A 95 per cent confidence interval is used in this report and represents a range of plus or minus about two standard errors around the sample average. Using this confidence interval, it can be inferred that the population mean or proportion would lie within the confidence interval in 95 out of 100 replications of the measurement, using different samples randomly drawn from the same population. When comparing scores among countries, provinces, or population subgroups, the degree of error in each average should be considered to determine whether averages are significantly different from each other. Standard errors and confidence intervals may be used as the basis for performing these comparative statistical tests. Such tests can identify, with a known probability, whether actual differences are likely to be observed in the populations being compared. For example, when an observed difference is significant at the .05 level, it implies that the probability is less than .05 that the observed difference could have occurred because of sampling or measurement error. When comparing countries and provinces, extensive use is made of this type of statistical test to reduce the likelihood that differences resulting from sampling or measurement errors will be interpreted as real. Only statistically significant differences at the .05 level are noted in this report, unless otherwise stated. If the confidence intervals overlap, an additional test of significance (t-test) was conducted to determine whether the difference was statistically significant. In case of multiple t-tests, no corrections were made to reduce the false positive, or Type-I error rate. When comparing results over time, the standard error includes a linking error to account for the fact that different cohorts of students have been tested over time with a test that also varied slightly over time. Canadian students perform well in science in a global context Overall, Canadian 15-year-old students achieved a mean score of 528 which is 35 points over the OECD average. As Figure 1.2 illustrates, Canada was outperformed by Singapore, Japan, and Estonia, ranking third (along with Finland) among OECD countries and fourth (along with Chinese Taipei, Finland, Macao–China, Vietnam, Hong Kong–China, and BSJG–China) among all 72 participating countries and economies. Further details on the interpretation of change over time are provided in tables in separate sections of this report. 17 19 Figure 1.2 Estimated average scores and confidence intervals for countries and provinces: Science 300 350 400 450 500 550 Alberta British Columbia British Columbia Above the Canadian average Quebec CANADA At the Canadian average Ontario Nova Scotia Below the Canadian average Prince Edward Island New Brunswick Newfoundland and Labrador Manitoba Saskatchewan 95% Confidence interval Estimated average score Countries 600 and provinces Singapore Alberta British Columbia Japan Quebec Estonia Chinese Taipei Finland Macao-China CANADA Vietnam Ontario Hong Kong-China BSJG-China Nova Scotia Korea Prince Edward Island New Zealand Slovenia Australia United Kingdom Germany The Netherlands New Brunswick Newfoundland and Labrador Switzerland Ireland Belgium Denmark Poland Portugal Manitoba Norway United States Saskatchewan Austria France Sweden Czech Republic Spain Latvia Russian Federation Luxembourg Italy Hungary Lithuania Croatia Iceland Israel Malta Slovak Republic Greece Chile Bulgaria United Arab Emirates Uruguay Romania Cyprus Moldova Albania Turkey Trinidad and Tobago Thailand Costa Rica Qatar Colombia Mexico Montenegro Georgia Jordan Indonesia Brazil Peru Lebanon Tunisia Republic of Macedonia Kosovo Algeria Dominican Republic Average S.E. 556 541 539 538 537 534 532 531 529 528 525 524 523 518 517 516 515 513 513 510 509 509 509 506 506 506 503 502 502 501 501 499 498 496 496 495 495 493 493 493 490 487 483 481 477 475 475 473 467 465 461 455 447 446 437 435 435 433 428 427 425 425 421 420 418 416 416 411 411 409 403 401 397 386 386 384 378 376 332 (1.2) (4.0) (4.3) (3.0) (4.7) (2.1) (2.7) (2.4) (1.1) (2.1) (3.9) (3.9) (2.5) (4.6) (4.5) (3.1) (5.4) (2.4) (1.3) (1.5) (2.6) (2.7) (2.3) (4.5) (3.2) (2.9) (2.4) (2.3) (2.4) (2.5) (2.4) (4.7) (2.3) (3.2) (3.1) (2.4) (2.1) (3.6) (2.3) (2.1) (1.6) (2.9) (1.1) (2.5) (2.4) (2.7) (2.5) (1.7) (3.4) (1.6) (2.6) (3.9) (2.4) (4.4) (2.4) (2.2) (3.2) (1.4) (2.0) (3.3) (3.9) (1.4) (2.8) (2.1) (1.0) (2.4) (2.1) (1.0) (2.4) (2.7) (2.6) (2.3) (2.4) (3.4) (2.1) (1.2) (1.7) (2.6) (2.6) Note: OECD countries appear in italics. The OECD average was 493, with a standard error of 0.4. The results of Argentina, Kazakhstan, and Malaysia are excluded because of insufficient coverage to ensure comparability (see Appendix B.1.2 for these results). See Appendix B.3.1 for further comparisons between provinces and participating countries. Results for the province of Quebec in this table should be treated with caution because of a possible non-response bias (see Appendix A for further details). 20 When interpreting provincial and international results, it should be kept in mind that PISA students were aged between 15 years and 3 months and 16 years and 2 months in participating countries. In Canada, 88 per cent of students were at the Grade 10 (Secondary 4) level and they achieved a mean score of 532. Grade 9 students (10 per cent) achieved a mean score of 501. Small proportions of students participating in PISA 2015 were in lower or higher grades. Table 1.5 lists those countries performing significantly better than or equally as well as Canada on the overall science scale (with all remaining countries that took part in PISA 2015 being statistically below the Canadian average). Science results are also reported for each competency, knowledge, and content area subscale. Students’ facility at applying science to problems and issues is dependent on skills inherent in all three competencies. A closer analysis of results in each category can help inform policy-level discussions, curricular emphasis, or teaching practice. Canadian results by scientific competency are similar, with an average score of 530 in evaluating and designing scientific enquiry and in explaining phenomena scientifically, and 525 in interpreting data and evidence scientifically. Across OECD countries, students scored 493 in all three competency subscales. Only Singapore achieved a higher average score than Canada in evaluating and designing scientific enquiry while Singapore and Japan outperformed Canada in explaining phenomena scientifically, and Singapore, Japan, Estonia, Chinese Taipei, and Macao–China outperformed Canada in interpreting data and evidence scientifically. Canadian students achieved an average score of 528 in both the content and the procedural and epistemic knowledge subscales. Across OECD countries, students scored 493 on both knowledge subscales. Content knowledge was further reported for each of three important systems in science. At the Canadian level, there was no significant difference in achievement across the three system subscales. The scores across OECD countries varied slightly: living systems (492), physical systems (493), and Earth and space systems (494). 21 Table 1.5 Countries performing better than or as well as Canada – Science Better than Canada* As well as Canada* Singapore, Japan, Estonia Chinese Taipei, Finland, Macao–China, Vietnam, Hong Kong–China, BSJG–China Explain phenomena scientifically Singapore, Japan Chinese Taipei, Finland, Estonia, Macao–China, Hong Kong–China, BSJG–China Evaluate and design scientific enquiry Singapore Japan, Estonia, Finland, Macao–China, Chinese Taipei, Hong Kong–China Interpret data and evidence scientifically Singapore, Japan, Estonia, Chinese Taipei, Macao–China Finland, Korea, Hong Kong–China, BSJG–China Content Singapore, Japan, Chinese Taipei Finland, Estonia, Macao–China, Hong Kong–China, BSJG–China, Procedural and epistemic Singapore, Japan, Estonia Macao–China, Chinese Taipei, Finland, Hong Kong–China Physical systems Singapore, Japan, Estonia, Finland Macao–China, Chinese Taipei, Hong Kong–China, BSJG–China, Living systems Singapore, Japan Chinese Taipei, Estonia, Finland, Macao–China, Hong Kong–China Earth and space systems Singapore, Japan, Estonia Finland, Chinese Taipei, Macao–China, Hong Kong–China, Korea Science overall Science – Competency subscales Science – Knowledge subscales Science – Content area subscales * Differences in scores are statistically significant only when confidence intervals do not overlap. If the confidence intervals overlap, an additional test of significance was conducted to determine whether the difference was statistically significant. Countries performing as well as Canada have a confidence interval that overlaps that of Canada. There are marked variations between provinces At the provincial level, 15-year-old students in Quebec, Alberta, and British Columbia performed above the Canadian average in overall science, with average scores of 537, 541, and 539, respectively. Only Singapore (556) had higher achievement than these three jurisdictions. Students in Ontario performed at the Canadian average while the other provinces were below the Canadian average. With the exception of Manitoba and Saskatchewan which scored at the OECD average, all provinces scored above the OECD average in science (Appendix B.1.2). An analysis of results by scientific competencies also reveals provincial differences. As presented in Table 1.6 and Appendix B.1.3, Alberta was above the Canadian mean score and Ontario was at the mean score for all three scientific competencies. Quebec students achieved above the Canadian mean for the competencies of evaluating and designing scientific enquiry and interpreting data and evidence scientifically and at the Canadian mean for explaining phenomena scientifically. British Columbia students achieved above the Canadian mean for the competencies of explaining phenomena scientifically and interpreting data and evidence scientifically and at the Canadian mean for evaluating and designing scientific enquiry. Students in Nova Scotia performed at the 22 Canadian average in interpreting data and evidence scientifically. Students in all other provinces were below the Canadian average for all three scientific competencies. There were also provincial differences in performance between the different knowledge subscales. Students in Alberta and British Columbia achieved scores above the Canadian average in both knowledge subscales while students in Ontario performed at the Canadian average in both knowledge subscales. Quebec students achieved above the Canadian average in procedural and epistemic knowledge and at the Canadian average in content knowledge. Students in Prince Edward Island scored at the Canadian average in content knowledge but below the Canadian average on the procedural and epistemic knowledge subscale. All other provinces were below the Canadian average on both knowledge subscales (Table 1.6 and Appendix B.1.4). When it came to the different content areas at the provincial level, Alberta performed better than the Canadian average in all three content areas, while British Columbia performed better than the Canadian average in the living systems subscale and Quebec in physical systems and Earth and space systems. Students in British Columbia performed at the Canadian average in physical systems and Earth and space systems, while students in Quebec performed at the Canadian average in living systems. Students in Ontario performed at the Canadian average in all three content areas, while students achieved as well as the Canadian average in both physical and living systems in Prince Edward Island and Nova Scotia (Table 1.6 and Appendix B.1.5). Table 1.6 Provincial results in science relative to the Canadian average Better than Canada* As well as Canada* Quebec, Alberta, British Columbia Ontario Explain phenomena scientifically Alberta, British Columbia Quebec, Ontario Evaluate and design scientific enquiry Quebec, Alberta Ontario, British Columbia Interpret data and evidence scientifically Quebec, Alberta, British Columbia Nova Scotia, Ontario Content Alberta, British Columbia Prince Edward Island, Quebec, Ontario Procedural and epistemic Quebec, Alberta, British Columbia Ontario Physical systems Quebec, Alberta Prince Edward Island, Nova Scotia, Ontario, British Columbia Living systems Alberta, British Columbia Prince Edward Island, Nova Scotia, Quebec, Ontario Earth and space systems Quebec, Alberta Ontario, British Columbia Science overall Science – Competency subscales Science – Knowledge subscales Science – Content area subscales * Differences in scores are statistically significant only when confidence intervals do not overlap. If the confidence intervals overlap, an additional test of significance was conducted to determine whether the difference was statistically significant. Results for the province of Quebec in this table should be treated with caution because of a possible non-response bias (see Appendix A for further details). 23 Canadian results in science are characterized by relatively high levels of equity Another way of studying differences in achievement is to look at the distribution of scores within a population. The difference between the mean score of students at the 90th percentile and those at the 10th percentile is often used as a proxy for equity in educational outcomes whereby the relative distribution of scores or the gap that exists between students with the highest and lowest levels of performance within each jurisdiction is examined. Figure 1.3 shows the difference in average scores between lowest achievers and highest achievers in science in Canada and the provinces. For Canada overall, those in the highest decile scored 240 points higher compared to those in the lowest decile. This compares to 247 across OECD countries. At the provincial level, the largest gap can be observed in Ontario (less equity) and the smallest in Prince Edward Island (more equity). Although high-achieving countries tend to have a larger gap, high achievement does not necessarily come at the cost of equity. Notably, Singapore and Japan achieved higher average scores comparable to Canada (556 and 538 respectively) (Appendix B.1.2) but only Japan has similar equity levels as seen by the difference in the achievement gap (271 and 243 respectively) (Appendix B.1.6). Figure 1.3 PISA 2015 Science – Difference between high and low achievers, Canada, provinces, and OECD Difference between the 90th and 10th percentiles* Prince Edward Island 223 Quebec 226 Saskatchewan 229 Nova Scotia 235 British Columbia 237 New Brunswick 237 Alberta 238 Manitoba 239 Newfoundland and Labrador 240 Canada 240 Ontario 246 OECD 247 300 350 400 450 500 550 600 * Jurisdictions are ordered from the least to the most difference between the two groups. 650 Percentiles of performance 10th 25th 75th Mean score with 95% confidence interval Note: Results for the province of Quebec in this table should be treated with caution because of a possible non-response bias (see Appendix A for further details). 24 700 90th In Canada, science results show significant differences by the school system’s language In seven Canadian provinces (Nova Scotia, New Brunswick, Quebec, Ontario, Manitoba, Alberta, and British Columbia), the sample was sufficiently large to allow for separate reporting for students in the anglophone and francophone school systems.18 No difference between the two language systems in science performance was found in Canada overall or in New Brunswick and British Columbia, as shown in Table 1.7 and Appendix B.1.7. The remaining provinces show a statistically different performance on the overall science scale between the anglophone and the francophone school systems. Students in the majority-language system (students in the anglophone school systems in Nova Scotia, Ontario, Manitoba, and Alberta and students in the francophone school system in Quebec) performed better than their counterparts in the minority-language system. The same pattern is found for the scientific competencies and knowledge subscales and for two content area subscales, living and physical systems. For Earth and space systems, there is higher achievement in francophone school systems for Canada overall (Table 1.8 and Appendices B.1.8 to B.1.10). Table 1.7 Estimated average overall science scores, by province and language of the school system Anglophone school system Francophone school system Difference between systems* Average S.E. Average S.E. Score difference S.E. Nova Scotia 518 (4.6) 477 (7.3) 42 (8.7) New Brunswick 508 (5.7) 502 (4.9) 6 (7.1) Quebec 514 (3.5) 540 (5.3) -26 (6.2) Ontario 526 (4.1) 486 (4.2) 39 (5.4) Manitoba 501 (5.0) 473 (6.9) 28 (8.3) Alberta 541 (4.1) 504 (8.9) 37 (10.6) British Columbia 539 (4.3) 532 (15.8) 7 (15.9) Canada 526 (2.2) 533 (4.7) -7 (5.0) * Results in bold indicate a statistical difference between the two systems. A negative difference means that the result for the francophone school system is higher. The Canadian results include students from all provinces. Results for the province of Quebec in this table should be treated with caution because of a possible non-response bias (see Appendix A for further details). These results suggest that policy-makers may wish to analyze provincial results more closely, given that some of the largest differences between the majority- and the minority-language school systems amount to between 26 and 42 points for overall science scale and between 22 and 49 points on the PISA competency, knowledge, and content area subscales. Within anglophone school systems, students in French Immersion programs completed the science component in English. 18 25 Table 1.8 Summary of differences in provincial results between language systems in science competency, knowledge, and content area subscales Significantly higher* performance in anglophone school system Significantly higher* performance in francophone school system Nova Scotia, Ontario, Manitoba, Alberta Quebec Canada, New Brunswick, British Columbia Nova Scotia, Ontario, Manitoba, Alberta Quebec Canada, New Brunswick, British Columbia Physical systems Living systems Nova Scotia, Ontario, Manitoba, Alberta Quebec Canada, New Brunswick, British Columbia Earth and space systems Nova Scotia, Ontario, Manitoba, Alberta Canada, Quebec New Brunswick, British Columbia No significant differences between school systems Science – Competency subscales Explain phenomena scientifically Evaluate and design scientific enquiry Interpret data and evidence scientifically Science – Knowledge subscales Content Procedural and epistemic Science – Content area subscales * Differences in scores are statistically significant only when confidence intervals do not overlap. If the confidence intervals overlap, an additional test of significance was conducted to determine whether the difference was statistically significant. Results for the province of Quebec in this table should be treated with caution because of a possible non-response bias (see Appendix A for further details). There is no gender gap in science in Canada overall Policy-makers have an interest in reducing gender disparities in education. Student motivation in school can have a significant impact on their later career choices and earning prospects. In science overall, there was no difference in average achievement scores between boys and girls in Canada and the provinces (Appendix B.1.11). There was a small gender gap in OECD countries: boys outperformed girls in science by four points on average in PISA 2015, with much variability between participating countries: in over 20 countries, girls outperformed boys in science while boys outperformed girls in a similar number of countries. In Canada, there was a higher proportion of boys than girls performing at the highest levels of proficiency (Levels 5 and 6) as well as at the lowest levels of proficiency (below Level 2) in science. Provincially, more boys than girls performed at the highest levels of proficiency in Newfoundland and Labrador and Quebec while no gender differences were observed in any of the provinces at the lowest levels of proficiency (Table 1.9 and Appendix B.1.12). 26 When looking at the different scientific competency, knowledge, and content area subscales, we see that performance was remarkably similar between 15-year-old boys and girls in Canada in the science competency of interpreting data and evidence scientifically and for all three content area subscales: physical systems, living systems, and Earth and space systems (Table 1.10). Girls outperformed boys in evaluating and designing scientific enquiry and in procedural and epistemic knowledge while boys outperformed girls in explaining phenomena scientifically and in content knowledge. Table 1.9 Distribution of students on the overall science scale by proficiency level and gender Levels 5 and 6 Percentage of girls is significantly higher* than percentage of boys Percentage of boys is significantly higher* than percentage of girls No significant differences in the percentage of boys and girls Canada, Newfoundland and Labrador, Quebec Prince Edward Island, Nova Scotia, New Brunswick, Ontario, Manitoba, Saskatchewan, Alberta, British Columbia Below Level 2 Percentage of girls is significantly higher* than percentage of boys Percentage of boys is significantly higher* than percentage of girls No significant differences in the percentage of boys and girls Canada all provinces * Differences in percentages at proficiency levels are statistically significant only when confidence intervals do not overlap. If the confidence intervals overlap, an additional test of significance was conducted to determine whether the difference was statistically significant. Gender differences at the provincial level for science overall and by subscale are shown in Table 1.10 and in appendices B.1.13 to B.1.15. 27 Table 1.10 Summary of gender differences in average science scores for Canada and the provinces Girls performed significantly higher* than boys Boys performed significantly higher* than girls Science overall No significant differences between boys and girls Canada, all provinces Science – Competency subscales Explain phenomena scientifically Evaluate and design scientific enquiry Canada, Newfoundland and Labrador, Quebec, Saskatchewan, Alberta, British Columbia Canada, Prince Edward Island, Ontario Prince Edward Island, Nova Scotia, New Brunswick, Ontario, Manitoba Newfoundland and Labrador, Nova Scotia, New Brunswick, Quebec, Manitoba, Saskatchewan, Alberta, British Columbia Interpret data and evidence scientifically Canada, all provinces Science – Knowledge subscales Content Procedural and epistemic Canada, Newfoundland and Labrador, Quebec, Saskatchewan Canada, Ontario Prince Edward Island, Nova Scotia, New Brunswick, Ontario, Manitoba, Alberta, British Columbia Newfoundland and Labrador, Prince Edward Island, Nova Scotia, New Brunswick, Quebec, Manitoba, Saskatchewan, Alberta, British Columbia Science – Content area subscales Physical systems Quebec Canada, Newfoundland and Labrador, Prince Edward Island, Nova Scotia, New Brunswick, Ontario, Manitoba, Saskatchewan, Alberta, British Columbia Living systems Canada, all provinces Earth and space systems Canada, all provinces * Differences in scores are statistically significant only when confidence intervals do not overlap. If the confidence intervals overlap, an additional test of significance was conducted to determine whether the difference was statistically significant. Results for the province of Quebec in this table should be treated with caution because of a possible non-response bias (see Appendix A for further details). Table 1.10 28 The mean performance of Canadian students in science has remained stable over time PISA 2015 provides the fourth assessment of science since 2006 when the first full assessment of science took place. As a result, PISA 2015 enables countries and provincial education systems to compare their own performance over time between 2006 and 2015. This important information can inform educational policy and instructional practices. While this section looks at changes over time, performance differences should be interpreted with caution. It is possible to compare changes in student performance over time in each PISA domain because a number of common test questions are used in each survey. However, the limited number of such common test items used increases the chances of measurement error. To account for this, an extra error factor, known as the linking error, is introduced into the standard error. The standard errors with linking errors should be used whenever comparing performance across assessments (but not when comparing results across countries/ economies or subpopulations within a particular assessment).19 Consequently only those changes that are indicated as statistically significant should be considered. In Canada, as well as across the OECD countries, science performance did not change between 2006 and 2015. However, there were changes in performance in some of the 57 countries that participated in both PISA 2006 and PISA 2015. In six countries (Qatar, Portugal, Macao–China, Romania, Norway, and Colombia) science performance improved on a statistically significant basis, while in 14 countries, science performance declined between the baseline year and 2015. No changes were observed in the remaining countries. In 2006, Canada’s average performance in science was at its highest with a score of 534 points: Canada ranked third, after Finland (563) and Hong Kong–China (542). Since then, Canadian results have remained very stable with average scores of 529, 525, and 528 points in 2009, 2012, and 2015 respectively (Figure 1.4). Although the lack of improvement is a cause for closer analysis, it is important to note that a significant change in science performance is observed only between 2006 and 2012. Compared to the baseline, there is no significant change between 2006 and 2009 or between 2006 and 2015. Figure 1.4 PISA Canadian results over time, 2006–2015 science overall Note: Difference compared with baseline (2006). See OECD, PISA 2015 Results: Excellence and Equity in Education, for information on linking errors. 19 29 Provincially, no significant change in science achievement was observed in most provinces, with the exception of Newfoundland and Labrador, Manitoba, and Saskatchewan where the average score decreased by approximately 20 points (Table 1.11 and Appendix B.1.16). Table 1.11 Comparison of performance in science in PISA 2006–2015, Canada and the provinces 2006 2009 2012 Average 526 509 Standard error 2.5 2.7 Nova Scotia 520 2.5 523 3.7 516 4.6 517 6.3 New Brunswick 506 2.3 501 3.5 507 4.4 506 6.3 Quebec 531 4.2 524 4.1 516* 4.8 537 6.5 Ontario 537 4.2 531 4.2 527 5.6 524 6.0 Manitoba 523 3.2 506* 4.7 503* 4.8 499* 6.5 Saskatchewan 517 3.6 513 4.5 516 4.6 496* 5.5 Alberta 550 3.8 545 5.0 539 5.8 541 6.0 British Columbia 539 4.7 535 4.8 544 5.3 539 6.2 Canada 534 2.0 529 3.0 525* 4.0 528 4.9 Newfoundland and Labrador Prince Edward Island Average 518 495* Standard error 4.0 3.5 2015** Average 514* 490* Standard error 5.0 4.4 Average 506* 515 Standard error 5.5 7.0 * Significant difference compared with baseline (2006). The standard error of measurement includes a linking error to account for the comparison of results over time between the baseline (2006) and subsequent years. ** Results for the province of Quebec in this table should be treated with caution because of a possible non-response bias (see Appendix A for further details). At the Canadian level, the proportion of low-performing (below Level 2) 15-year-old students remained stable in science between 2006 and 2015; however, the proportion of students achieving below Level 2 has increased in Newfoundland and Labrador and Manitoba. The proportion of students achieving Levels 5 and 6 also remained unchanged over the 2006–to–2015 period although provincially, the proportion decreased in Newfoundland and Labrador, Manitoba, and Saskatchewan (Appendix B.1.17). With the exception of Newfoundland and Labrador where a gender gap in science achievement favoured girls in 2006, there have been no significant differences between girls and boys across Canada and in the other provinces over time in science (Appendix B.1.18) Summary Canada continues to perform well in science, with close to 90 per cent of Canadian students reaching the baseline level of science proficiency required to participate fully in modern society (Level 2) while almost one in ten students reached Levels 5 or 6. Globally, Canada ranked third among OECD countries and fourth among all participating countries and economies. In spite of these strong results, PISA 2015 results in scientific literacy also suggest that there is cause for some concern. Almost one in ten Canadian students do not meet the benchmark level of science proficiency, a proportion which has not changed since the baseline year in 2006, and students in minority-language settings achieve lower results in science compared to their counterparts in majority-language settings. 30 Chapter 2 Canadian Students’ Reading and Mathematics Performance in an International Context This chapter presents the overall results of the PISA 2015 assessments in the minor domains of reading and mathematics. For each domain, the performance of 15-year-old students across Canada and in the 10 provinces is compared to the performance of 15-year-olds from the other countries that participated in PISA 2015. Next, it examines the performance of students enrolled in anglophone and francophone school systems for those provinces where the two groups were sampled sufficiently. This is followed by a comparison between the performance of boys and girls in Canada and the provinces. Changes over time are discussed. Defining reading and mathematics Since reading and mathematics were minor domains in PISA 2015, there were fewer assessment items in these two areas compared to the major domain of science. As a result, PISA 2015 allows for only an update on overall performance in reading and mathematics, and not on their sub-domains. Additionally, although paper-based assessments were provided for countries that chose not to test their students by computer, in Canada, computer was the primary mode of delivery for all domains in PISA 2015. Because the computer-based assessments of reading and mathematics were an optional domain in PISA 2012 and were not taken by all countries, they are not part of the reading and mathematical literacy trends. With an emphasis on functional knowledge and skills that allow active participation in society, PISA defines reading and mathematics like this:20 • Reading literacy (hereafter referred to as reading) is an individual’s capacity to understand, use, reflect on, and engage with written texts, to achieve one’s goals, develop one’s knowledge and potential, and participate in society. • Mathematical literacy (hereafter referred to as mathematics) is an individual’s capacity to formulate, employ, and interpret mathematics in a variety of contexts. It includes reasoning mathematically and using mathematical concepts, procedures, facts, and tools to describe, explain, and predict phenomena. It assists individuals to recognize the role that mathematics plays in the world and to make the well-founded judgments and decisions needed by constructive, engaged, and reflective citizens. As is the case for science, the scores for reading and mathematics are expressed on a scale with an average among OECD countries of 500 and a standard deviation of 100. This average was established in the year in which the domain became the main focus of the assessment (2000 for reading and 2003 for mathematics). Approximately two-thirds of the students in OECD countries scored between 400 and 600 (i.e., within one standard deviation of the average). Because participating countries and performance have changed over time, the OECD average scores for reading and mathematics in PISA 2015 differ slightly from 500. OECD, PISA 2015 assessment and analytical framework: Science, reading, mathematic and financial literacy (Paris: OECD 2016), available at http://dx.doi. org/10.1787/9789264255425-en. 20 31 Canadian students perform well in reading and mathematics in a global context One way to summarize student performance and compare the relative standing of countries is by examining their average test scores. However, simply ranking countries based on their average scores can be misleading because there is a margin of uncertainty associated with each score. As discussed in Chapter 1, when interpreting average performances, only those differences between countries that are statistically significant should be noted. On average, Canadian 15-year-olds performed well in reading and mathematics (Table 2.1 and Figures 2.1 and 2.2). Canadian students had an average score of 527 in reading and 516 in mathematics, well above the OECD average of 493 and 490, respectively. Table 2.1 shows the countries that performed significantly better than or the same as Canada in reading and mathematics. The averages of the students in all the remaining countries were significantly below those of Canada. Among the 72 countries that participated in PISA 2015, only one outperformed Canada in reading while six outperformed Canada in mathematics. Table 2.1 Countries performing better than or as well as Canada in reading and mathematics Better than Canada* As well as Canada* Reading Singapore Hong Kong–China, Finland, Ireland Mathematics Singapore, Hong Kong–China, Macao–China, Chinese Taipei, Japan, BSJG–China Korea, Switzerland, Estonia, the Netherlands, Denmark, Finland * Differences in scores are statistically significant only when confidence intervals do not overlap. If the confidence intervals overlap, an additional test of significance was conducted to determine whether the difference was statistically significant. While average performance is useful in assessing the overall performance of students, it can mask significant variation within a jurisdiction. The gap that exists between students with the highest and those with the lowest levels of performance is an important indicator of the equity of educational outcomes. Further information on the performance within jurisdictions can be obtained by examining the relative distribution of scores. For Canada overall, those in the highest decile (90th percentile) scored 238 points higher in reading and 227 points higher in mathematics than those in the lowest decile (10th percentile). This compares to 249 points in reading and 232 points in mathematics across all OECD countries. The amount of variation in performance within a country in reading and mathematics fluctuated widely (see Appendix tables B.2.3 and B.2.4). Canada was one of the few countries with above-average performance and below-average disparity in student performance, as measured by the difference between the 90th and 10th percentiles. 32 Figure 2.1 Estimated average scores and confidence intervals for countries and provinces: Reading 300 350 Above the Canadian average At the Canadian average Below the Canadian average 400 450 500 550 BritishColumbia Columbia British 600 Countries and provinces British Columbia Singapore Alberta Alberta Alberta Alberta Quebec Quebec Ontario Ontario Hong Kong-China CANADA CANADA Finland Ireland Estonia Korea Nova Scotia Nova Scotia Japan Prince Edward Island Prince Edward Island Norway New Zealand Germany Macao-China Poland New Brunswick New Brunswick Slovenia Newfoundland and Labrador Newfoundland and Labrador The Netherlands Australia Sweden Denmark France Belgium Manitoba Manitoba Portugal United Kingdom Chinese Taipei United States Saskatchewan Saskatchewan Spain Russian Federation BSJG-China Switzerland Latvia Czech Republic Croatia Vietnam Vietnam Austria Italy Iceland Luxembourg Israel Lithuania Hungary Greece Chile Slovak Republic Malta Cyprus Uruguay Romania United Arab Emirates Bulgaria Turkey Costa Rica Trinidad and Tobago Montenegro Colombia Mexico Moldova Thailand Jordan Brazil Albania 95% Confidence interval Qatar Georgia Peru Indonesia Tunisia Estimated average score Dominican Republic Republic of Macedonia Algeria Kosovo Lebanon Average S.E. 536 535 533 532 527 527 527 526 521 519 517 517 516 515 513 509 509 509 506 505 505 505 503 503 500 500 499 499 498 498 498 497 497 496 496 495 494 492 488 487 487 487 485 485 482 481 479 472 470 467 459 453 447 443 437 434 434 432 428 427 427 427 425 423 416 409 408 407 405 402 401 398 397 361 358 352 350 347 347 (5.6) (1.6) (5.2) (4.7) (4.4) (2.7) (2.3) (2.5) (2.5) (2.2) (3.5) (4.9) (3.2) (6.1) (2.5) (2.4) (3.0) (1.3) (2.5) (5.2) (1.5) (3.5) (2.4) (1.7) (3.5) (2.5) (2.5) (2.4) (5.0) (2.7) (2.8) (2.5) (3.4) (3.6) (2.4) (3.1) (5.1) (3.0) (1.8) (2.6) (2.7) (3.7) (2.8) (2.7) (2.0) (1.4) (3.8) (2.7) (2.7) (4.3) (2.6) (2.8) (1.8) (1.7) (2.5) (4.1) (2.9) (5.0) (4.0) (2.6) (1.5) (1.6) (2.9) (2.6) (2.5) (3.3) (2.9) (2.8) (4.1) (1.0) (3.0) (2.9) (2.9) (3.1) (3.1) (1.4) (3.0) (1.6) (4.4) Notes: OECD countries appear in italics. The OECD average was 493, with a standard error of 0.5. The results of Argentina, Kazakhstan, and Malaysia are excluded because of insufficient coverage to ensure comparability (see Appendix B.2.1 for these results). Results for the province of Quebec in this table should be treated with caution because of a possible non-response bias (see Appendix A for further details). 33 Figure 2.2 Estimated average scores and confidence intervals for countries and provinces: Mathematics 300 350 400 450 500 550 600 Quebec Above the Canadian average British Columbia At the Canadian average CANADA Alberta Ontario Below the Canadian average Prince Edward Island Nova Scotia New Brunswick Manitoba Newfoundland and Labrador Saskatchewan 95% Confidence interval Estimated average score Countries and provinces Singapore Hong Kong-China Quebec Macao-China Chinese Taipei Japan BSJG-China Korea British Columbia Switzerland Estonia CANADA The Netherlands Alberta Denmark Finland Slovenia Ontario Belgium Germany Poland Ireland Norway Prince Edward Island Nova Scotia Austria New Zealand Vietnam Russian Federation Sweden Australia France New Brunswick United Kingdom Czech Republic Portugal Italy Manitoba Iceland Spain Luxembourg Newfoundland and Labrador Saskatchewan Latvia Malta Lithuania Hungary Slovak Republic Israel United States Croatia Greece Romania Bulgaria Cyprus United Arab Emirates Chile Turkey Moldova Uruguay Montenegro Trinidad and Tobago Thailand Albania Mexico Georgia Qatar Costa Rica Lebanon Colombia Peru Indonesia Jordan Brazil Republic of Macedonia Tunisia Kosovo Algeria Dominican Republic Average S.E. 564 548 544 544 542 532 531 524 522 521 520 516 512 511 511 511 510 509 507 506 504 504 502 499 497 497 495 495 494 494 494 493 493 492 492 492 490 489 488 486 486 486 484 482 479 478 477 475 470 470 464 454 444 441 437 427 423 420 420 418 418 417 415 413 408 404 402 400 396 390 387 386 380 377 371 367 362 360 328 (1.5) (3.0) (4.8) (1.1) (3.0) (3.0) (4.9) (3.7) (5.0) (2.9) (2.0) (2.3) (2.2) (4.7) (2.2) (2.3) (1.3) (4.2) (2.4) (2.9) (2.4) (2.1) (2.2) (6.4) (4.6) (2.9) (2.3) (4.5) (3.1) (3.2) (1.6) (2.1) (5.1) (2.5) (2.4) (2.5) (2.8) (4.2) (2.0) (2.2) (1.3) (3.2) (2.9) (1.9) (1.7) (2.3) (2.5) (2.7) (3.6) (3.2) (2.8) (3.8) (3.8) (4.0) (1.7) (2.4) (2.5) (4.1) (2.5) (2.5) (1.5) (1.4) (3.0) (3.4) (2.2) (2.8) (1.3) (2.5) (3.7) (2.3) (2.7) (3.1) (2.7) (2.9) (1.3) (3.0) (1.6) (3.0) (2.7) Notes: OECD countries appear in italics. The OECD average was 490, with a standard error of 0.4. The results of Argentina, Kazakhstan, and Malaysia are excluded because of insufficient coverage to ensure comparability (see Appendix B.2.2 for these results).Results for the province of Quebec in this table should be treated with caution because of a possible non-response bias (see Appendix A for further details). 34 Most provinces performed at or above the OECD average in reading and mathematics In reading, the performance of students in all provinces, was at or above the OECD average. In mathematics, students in Saskatchewan performed below the OECD average while students in all other provinces performed at or above the OECD average. As Table 2.2 shows, students in Quebec performed above the Canadian average in mathematics and at the Canadian average in reading. Students in Alberta and British Columbia performed at the Canadian average in both minor domains. Students in Newfoundland and Labrador, New Brunswick, Manitoba, and Saskatchewan performed below the Canadian average in both minor domains. Students in Prince Edward Island, Nova Scotia, and Ontario performed below the Canadian average in mathematics and at the Canadian average in reading. Table 2.2 Provincial results in reading and mathematics relative to the Canadian average Better than Canada* Reading Mathematics As well as Canada* Prince Edward Island, Nova Scotia, Quebec, Ontario, Alberta, British Columbia Quebec Alberta, British Columbia * Differences in scores are statistically significant only when confidence intervals do not overlap. If the confidence intervals overlap, an additional test of significance was conducted to determine whether the difference was statistically significant. Results for the province of Quebec in this table should be treated with caution because of a possible non-response bias (see Appendix A for further details). Figures 2.3 and 2.4 show the difference in average scores between those in the lowest decile (10th percentile) and those in the highest (90th percentile) in reading and mathematics. For reading, differences range from 218 in Prince Edward Island to 244 in Ontario, while for mathematics, they ranged from 198 in Prince Edward Island to 227 in Quebec. In all provinces, the difference in performance between high achievers and low achievers was smaller than the OECD average. This indicates that Canada’s education systems continue to achieve high degree of equity. 35 Figure 2.3 PISA 2015 Reading: Difference between high and low achievers, Canada, provinces, and OECD Difference betweeen the 90th and the 10th percentiles* PrinceEdward Edward Island Island Prince 218 Saskatchewan Saskatchewan 222 NovaScotia Scotia Nova 228 BritishColumbia Columbia British 229 Newfoundlandand andLabrador Labrador Newfoundland 229 Quebec Quebec 234 Alberta Alberta 235 Manitoba Manitoba 235 NewBrunswick Brunswick New 236 Canada Canada 238 Ontario Ontario 244 OECD OECD 249 300 300 350 350 400 400 450 450 500 500 550 550 10th *Jurisdictions are ordered from the least to the most difference between the two groups. 600 600 650 650 Percentiles of performance 25th 75th 700 700 90th Mean score with 95% confidence interval Note: Results for the province of Quebec in this table should be treated with caution because of a possible non-response bias (see Appendix A for further details). Figure 2.4 PISA 2015 Mathematics: Difference between high and low achievers, Canada, provinces, and OECD Difference betweeen the 90th and the 10th percentiles* Prince Edward Island 198 Saskatchewan 210 Newfoundland and Labrador 210 Nova Scotia 211 Manitoba 214 British Columbia 219 New Brunswick 223 Alberta 223 Ontario 224 Quebec 227 Canada 227 OECD 232 300 350 400 *Jurisdictions are ordered from the least to the most difference between the two groups. 450 500 550 10th 600 650 Percentiles of performance 25th 75th Mean score with 95% confidence interval Note: Results for the province of Quebec in this table should be treated with caution because of a possible non-response bias (see Appendix A for further details). 36 700 90th Across Canada, differences in reading and mathematics performance are seen between students attending majority-language school systems and those attending minoritylanguage systems in reading and mathematics Seven provinces had sufficiently large samples in the anglophone and francophone school systems (Nova Scotia, New Brunswick, Quebec, Ontario, Manitoba, Alberta, and British Columbia). The performance of the minoritylanguage group (students in francophone school systems in Nova Scotia, New Brunswick, Ontario, Manitoba, Alberta, and British Columbia and students in the anglophone school system in Quebec) is compared to that of the majority-language group. As Table 2.3 indicates, the relative performance of students in the two systems varied across provinces and by domain. Across Canada, the difference in reading performance between students in the anglophone school systems and those in the francophone school systems was not statistically significant. However, across the provinces, students in the majority-language school systems outperformed their peers in the minority-language school systems in four of the seven provinces. The differences between systems varied from 40 points in Manitoba to 57 points in Nova Scotia. Table 2.3 Estimated average reading and mathematics scores, by province and language of the school system Anglophone school system Francophone school system Difference between systems* Average Standard error Average Standard error Score difference Standard error Nova Scotia 519 (5.1) 462 (7.6) 57 (9.2) New Brunswick 509 (6.6) 493 (6.3) 16 (8.7) Quebec 523 (6.0) 533 (5.3) -10 (8.3) Ontario 529 (4.5) 476 (5.0) 54 (6.4) Manitoba 501 (5.3) 461 (8.1) 40 (9.6) Alberta 534 (5.2) 487 (12.6) 46 (14.5) British Columbia Canada 536 527 (5.6) (2.7) 516 526 (14.9) (4.7) 20 1 (14.6) (5.6) Nova Scotia 497 (4.7) 491 (8.3) 7 (8.7) New Brunswick 488 (5.8) 505 (7.3) -17 (8.5) Quebec 505 (6.7) 549 (5.4) -44 (9.0) Ontario 510 (4.4) 496 (6.5) 14 (7.8) Manitoba 489 (4.5) 482 (8.9) 8 (10.7) Alberta 512 (4.7) 503 (12.4) 8 (12.8) British Columbia Canada 522 509 (5.0) (2.6) 531 542 (16.0) (5.0) -9 -34 (16.9) (5.5) Reading Mathematics * Results in bold indicate a statistically significant difference between the two systems. A negative difference means that the result for the francophone school system is higher. The Canadian results include students from all provinces. Results for the province of Quebec in this table should be treated with caution because of a possible non-response bias (see Appendix A for further details). 37 In mathematics, Canadian students in the francophone school systems (542) outperformed their peers in the anglophone school systems (509) by 34 points, mainly as a result of the relatively strong performance of students in the francophone school system in Quebec. As Table 2.3 indicates, in only New Brunswick and Quebec did a statistically significant difference in mathematics performance exist between the two school systems. In both provinces, students from the francophone school system achieved a higher average in mathematics than their peers in the anglophone school system, although in Quebec, the francophone school system is a majoritylanguage school system while in New Brunswick, it is a minority-language one. Canadian girls outperformed boys in reading, while Canadian boys outperformed girls in mathematics As was the case since PISA 2000, girls performed significantly better than boys in PISA 2015 on the reading test in all countries and in all provinces. On average across OECD countries, girls outperformed boys in reading by 27 points in PISA 2015, while in Canada, this difference was 26 points. At the provincial level, the gender gap favouring girls ranged from 18 points in Newfoundland and Labrador to 36 points in Prince Edward Island (Table 2.4, Appendix B.2.7). In mathematics, on average across OECD countries, boys had a statistically significant higher score than girls, but the eight-point difference was small compared to the large gender gap in reading. In Canada, boys outperformed girls in mathematics by nine points. Across the provinces, a gender gap favouring boys was observed in Newfoundland and Labrador, Quebec, Ontario, Alberta, and British Columbia, with no significant gender differences in mathematics observed in the remaining provinces (Table 2.4, Appendix B.2.8). Table 2.4 Summary of gender differences in average reading and mathematics scores for Canada and the provinces Girls performed significantly better* than boys Reading Mathematics Boys performed significantly better* than girls No significant differences between boys and girls Canada, Newfoundland and Labrador, Prince Edward Island, Nova Scotia, New Brunswick, Quebec, Ontario, Manitoba, Saskatchewan, Alberta, British Columbia Canada, Newfoundland and Labrador, Quebec, Ontario, Alberta, British Columbia Prince Edward Island, Nova Scotia, New Brunswick, Manitoba, Saskatchewan * Differences in scores are statistically significant only when confidence intervals do not overlap. If the confidence intervals overlap, an additional test of significance was conducted to determine whether the difference was statistically significant. Results for the province of Quebec in this table should be treated with caution because of a possible non-response bias (see Appendix A for further details). 38 Canadian students’ performance in reading remained relatively stable over time while performance in mathematics stabilized between 2012 and 2015 PISA 2015 is the sixth assessment of reading since 2000, when the first major assessment of reading took place, and the fifth assessment of mathematics since 2003, when the first major assessment of mathematics took place. Because a comprehensive analysis of trends in reading (between 2000 and 2012) and in mathematics (between 2003 and 2012) was included in the PISA 2012 national report,21 this chapter focuses on changes in reading since 2009 and changes in mathematics since 2012 — the most recent cycles when reading and mathematics were major domains. Performance changes over time are always compared to a baseline year, an administration in which the subject was the major domain. While this section looks at changes over time, performance differences should be interpreted with caution. It is possible to compare changes in student performance over time in each PISA domain because a number of common test questions are used in each survey. However, the limited number of such common test items used increases the chances of measurement error. To account for this, an extra error factor, known as the linking error, is introduced into the standard error. The standard errors with linking errors should be used whenever comparing performance across assessments (but not when comparing results across countries/ economies or subpopulations within a particular assessment).22 Only those changes that are indicated as statistically significant should be considered. In Canada, as well as across the OECD countries, reading performance did not change between 2009 and 2015. However, there were changes in performance in some of the 59 countries that participated in both PISA 2009 and PISA 2015. In 19 countries23 reading performance improved on a statistically significant basis, while in 11 countries24 it declined, with the other countries maintaining their scores. In mathematics, after a significant decline between 2003 and 2012, the performance of Canadian students in mathematics remained unchanged between 2012 and 2015. On average across OECD countries, mathematics performance also remained broadly stable over the 2012 to 2015 period, although changes in performance were observed in some of the 61 countries that participated in both cycles. Mathematics performance increased on a statistically significant basis in 10 countries25 and decreased in 12,26 with no statistically significant changes observed in the remaining countries. Performance in reading and mathematics remained stable across the provinces with the following exceptions: reading performance improved in Prince Edward Island between 2009 and 2015 and mathematics performance improved in Prince Edward Island and decreased in Saskatchewan over the 2012-to-2015 period (Table 2.5). Brochu, P., Deussing, M.-P., Houme, K., & Chuy, M. (2013). Measuring up: Canadian results of the OECD PISA Study: The performance of Canada’s youth in mathematics, reading, and science – 2012. First results for Canadians aged 15. Toronto: Council of Ministers of Education, Canada. 22 See OECD, PISA 2015 Results: Excellence and Equity in Education, for information on linking errors. 23 Albania, Austria, Colombia, Croatia, Estonia, Germany, Ireland, Luxembourg, Macao–China, Montenegro, Norway, Peru, Qatar, Russian Federation, Singapore, Slovenia, Spain, Trinidad and Tobago, and Uruguay. 24 Australia, Greece, Hungary, Iceland, Korea, New Zealand, Slovak Republic, Thailand, Tunisia, Turkey, and United Arab Emirates (Dubai region only). 25 Albania, Colombia, Denmark, Montenegro, Norway, Peru, Qatar, Russian Federation, Slovenia, and Sweden. 26 Australia, Brazil, Chinese Taipei, Hong Kong–China, Korea, the Netherlands, Poland, Singapore, Tunisia, Turkey, United States, and Vietnam. 21 39 Table 2.5 Comparison of performance in reading in PISA 2009, 2012, and 2015, Canada and the provinces 2009 2012 2015** Average Standard error Average Standard error Reading Newfoundland and Labrador Prince Edward Island 506 486 (3.7) (2.4) 503 490 (4.5) (3.7) 505 515* (4.9) (7.0) Nova Scotia 516 (2.7) 508 (4.0) 517 (6.0) New Brunswick 499 (2.5) 497 (3.7) 505 (6.3) Quebec 522 (3.1) 520 (4.4) 532 (5.8) Ontario 531 (3.0) 528 (5.1) 527 (5.6) Manitoba 495 (3.6) 495 (4.2) 498 (6.0) Saskatchewan 504 (3.3) 505 (3.8) 496 (4.9) Alberta 533 (4.6) 525 (4.8) 533 (6.2) British Columbia 525 (4.2) 535 (5.2) 536 (6.5) Canada 524 (1.5) 523 (3.2) 527 (4.1) Average Standard error * Significant difference compared with baseline (2009). The linkage error is incorporated into the standard error for 2012 and 2015 to account for the comparison of results over time, compared with baseline (2009). ** Results for the province of Quebec in this table should be treated with caution because of a possible non-response bias (see Appendix A for further details). Table 2.6 Comparison of performance in mathematics in PISA 2012 and 2015, Canada and the provinces 2012 2015** Average Standard error Mathematics Newfoundland and Labrador Prince Edward Island Average Standard error 490 479 (3.7) (2.5) 486 499* (4.8) (7.3) Nova Scotia 497 (4.1) 497 (5.8) New Brunswick 502 (2.6) 493 (6.2) Quebec 536 (3.4) 544 (5.9) Ontario 514 (4.1) 509 (5.5) Manitoba 492 (2.9) 489 (5.5) Saskatchewan 506 (3.0) 484* (4.6) Alberta 517 (4.6) 511 (5.9) British Columbia 522 (4.4) 522 (6.1) Canada 518 (1.8) 516 (4.2) * Significant difference compared with baseline (2012). The linkage error is incorporated into the standard error for 2015 to account for the comparison of results over time, compared with baseline (2012). ** Results for the province of Quebec in this table should be treated with caution because of a possible non-response bias (see Appendix A for further details). 40 Summary Because reading and mathematics were minor domains in PISA 2015, a smaller proportion of students were assessed in them compared to the science assessment. Additionally, they made up a smaller number of items than in the science assessment. This chapter therefore provides an update on overall performance in each of these domains only, and not on their sub-domains as was done in previous years. Canada continues to perform well internationally in reading and mathematics. Students in Canada scored well above the OECD average and were outperformed by students in only one country in reading and six in mathematics among the 72 countries that participated in PISA 2015. Among the provinces, students in Quebec, Ontario, Alberta, and British Columbia performed above the OECD average in both reading and mathematics. Students in Newfoundland and Labrador, Prince Edward Island, Nova Scotia, and New Brunswick performed above the OECD average in reading and at the average in mathematics. Students in Manitoba performed at the OECD average in reading and mathematics, while students in Saskatchewan performed at the average in reading and below the average in mathematics. Students in the anglophone, majority-language school systems in Nova Scotia, Ontario, Manitoba, and Alberta performed significantly better in reading than their peers in the francophone, minority-language school systems. In mathematics, students in the francophone school systems in Quebec and New Brunswick achieved a higher average score than their peers in the anglophone school systems. As was observed in past PISA assessments, girls continue to perform better than boys in reading. Boys performed better than girls in mathematics in Canada overall in half of the provinces; whereas there is no gender gap found in the other provinces. Canada’s overall mean performance in reading remained stable over the 2009 to 2015 period while at the same time its international standing among PISA participants improved. Among the countries that participated in both the 2009 and 2015 assessments, four countries outperformed Canada in 2009 while only one outperformed Canada in 2015. After seeing a decline in the mathematics performance of its students between 2003 and 2012, Canada’s performance in mathematics remained stable between 2012 and 2015 and its relative standing improved among the countries that participated in both assessments, with five countries outperforming Canada in 2015 compared to eight in 2012. Reading performance in all provinces except for Prince Edward Island remained stable since 2009. In Prince Edward Island, a significant improvement in reading performance was observed over the period. Consequently, Prince Edward Island went from performing below the OECD average in 2012 to performing above it in 2015. Only Prince Edward Island and Saskatchewan observed a change in the mathematics performance of their students since 2012. Saskatchewan experienced a significant decline in the mathematics performance of its students and consequently went from performing above the OECD average in 2012 to performing below the OECD average in 2015. On the other hand, Prince Edward Island had a significant increase in performance in mathematics and, as a result, went from performing below the OECD average in 2012 to performing at the OECD average in 2015. 41 Conclusion The Program for International Student Assessment (PISA) is an international study that measures trends in learning outcomes in science, reading, and mathematics for students at age 15. The study has been conducted every three years under the aegis of the Organisation for Economic Cooperation and Development (OECD) since 2000. In 2015, it was administered in 72 countries and economies, including Canada. The major focus of PISA 2015 was science while reading, mathematics, and financial literacy were tested as minor domains. Over 20,000 students from approximately 900 schools took the PISA assessment in the 10 Canadian provinces in the spring of 2015. PISA is valuable for its capacity to provide comparative information on skill levels of students near the end of their compulsory education. Not only does PISA enable comparisons between provinces, countries, and economies on the knowledge and skills of their youth, it also provides an opportunity to monitor their change in performance over time. Overview of results Performance in science, reading, and mathematics According to the results of PISA 2015, Canada remains one of the top-performing countries in science. Close to 90 per cent of Canadian students and 79 per cent of students in OECD countries performed at or above Level 2 in science, which is the baseline level of science proficiency required for people to take advantage of further learning opportunities and to participate fully in modern society. At the provincial level, the percentage of Canadian students at or above the baseline level of performance ranged from 83 per cent in Saskatchewan and Manitoba to over 90 per cent in Quebec, Alberta, and British Columbia. At the lower end of the PISA science scale, 11 per cent of Canadian students performed below the baseline compared with 21 per cent of students across the OECD countries. Twelve per cent of Canadian students performed at the highest proficiency levels (Levels 5 and 6) in PISA 2015 compared to 8 per cent performing at this level for the OECD. The proportion of high-performing students was 10 per cent or more in Nova Scotia, Quebec, Ontario, Alberta, and British Columbia. However, in spite of Canada’s strong performance in science, Singapore, the leading country in PISA 2015, had a much higher proportion of students performing at the highest levels (24 vs. 12%). Overall, Canadian 15-year-old students achieved a mean score of 528 in overall science, 35 points above the OECD average, and were surpassed by students from only three countries. At the provincial level, with the exception of Manitoba and Saskatchewan which scored at the OECD average, all provinces performed above the OECD average. From a Canadian perspective, students in Quebec, Alberta, and British Columbia achieved higher average scores than the Canadian average, placing them among the top-performing participants globally. Canadian results by scientific competency show some differences, with a higher average score in explaining phenomena scientifically and evaluating and designing scientific enquiry (530), and a lower score in interpreting data and evidence scientifically (525). Across OECD countries and economies, students scored 493 in all three competency subscales. Canadian students achieved an average score of 528 in both the content and the procedural and epistemic knowledge subscales. The average score across OECD countries was 493 on both knowledge subscales. 43 At the Canadian level, there was no significant difference in student achievement across the three broad content areas assessed in PISA 2015 and the Canadian results were more than 30 points higher than the OECD averages in all three content subscales. The scores across OECD countries were also very similar in the three content areas. Canada continues to perform well internationally in reading. Canadian students scored well above the OECD average and were outperformed by only one country in reading. At the provincial level, with the exception of Manitoba and Saskatchewan which scored at the OECD average, all provinces scored above the OECD average. Students in Prince Edward Island, Nova Scotia, Quebec, Ontario, Alberta, and British Columbia performed at the Canadian average in reading, whereas students in Newfoundland and Labrador, New Brunswick, Manitoba, and Saskatchewan performed below the Canadian average. Canada also achieved a strong performance in mathematics. Canadian students scored well above the OECD average and were outperformed by only six countries in mathematics among the 72 countries that participated in PISA 2015. At the provincial level, Quebec students performed higher than the Canadian average while Alberta and British Columbia students performed at the Canadian average. Performance by language of the school system In 2015, there was no overall achievement difference in Canada between the anglophone and francophone school systems in science and reading. For those provinces where there was a significant difference in achievement between the two language systems in science and reading, students in majority-language settings (students in anglophone school systems in Nova Scotia, Ontario, Manitoba, and Alberta and students in the francophone school system in Quebec) performed better than their counterparts in the minority-language settings. The same pattern was found for most scientific competencies, knowledge, and content area subscales, with the exception that for Canada overall students in francophone schools achieved higher scores for the Earth and space systems subscale. For mathematics, Canadian students in francophone school systems performed better than their counterparts in the anglophone systems, with students in the francophone school systems in New Brunswick and Quebec outperforming those in the anglophone school systems. No significant differences were observed between the two school systems in the other provinces. Performance by gender No gender achievement gap for science was found in Canada or the provinces. This result is consistent with the most recent Pan-Canadian Assessment Program (PCAP) evaluation of science in 2013.27 As was the case internationally, Canadian girls continue to outperform boys in reading, and this was true in all provinces. In mathematics, boys continue to outperform girls in Canada overall and in Newfoundland and Labrador, Quebec, Ontario, Alberta, and British Columbia; no gender differences were found in other provinces. Performance comparisons over time For science, at the Canadian level and in most provinces, very few statistically significant differences were observed in the proportion of top-performing (Level 5 or above) and low-performing (below Level 2) 15‑yearolds between the baseline year of 2006 and 2015. Science performance has not changed over the period in Canada although three provinces experienced significant declines. The average score decreased by approximately 20 points in Newfoundland and Labrador, Manitoba, and Saskatchewan. K. O’Grady, & K. Houme, PCAP 2013 Report on the Pan-Canadian assessment of science, reading, and mathematics (Toronto: Council of Ministers of Education, Canada, 2014). Available at http://www.cmec.ca/Publications/Lists/Publications/Attachments/337/PCAP-2013-Public-Report-EN.pdf 27 44 Between 2009 — the last time the major focus of PISA was reading — and 2015, reading performance in Canada and across most provinces remained stable. The only exception was Prince Edward Island which saw a significant improvement in its mean reading score. As well, after observing a decline in the mathematics performance of its students between the baseline year of 2003 and 2012, Canada’s performance in mathematics remained stable between 2012 and 2015, with only two provinces experiencing a significant change. More specifically, students in Prince Edward Island saw a significant improvement in their mathematics performance between 2012 and 2015 while students in Saskatchewan saw a significant decline. Equity in education As a measure of equity in educational outcomes, PISA considers the difference between the average score of students at the 90th percentile and those at the 10th percentile. In all three domains assessed by PISA, the gap between high and low achievers was smaller in Canada than in OECD countries (indicating more equity). Provincially, Prince Edward Island and Saskatchewan show a relatively smaller gap in all three domains. Final statement The results of this assessment suggest that in Canada, a majority of students have attained a level of scientific literacy that enables them to use their knowledge and skills to engage with issues and ideas related to science. The PISA 2015 results provide both affirmation and direction for Canadian jurisdictions and classrooms. While students appear to understand what is expected of them in science and appear to practise the key aspects when completing scientific tasks, there is room for improvement because there are numerous students below the baseline level (Level 2) for whom science remains a challenging subject. Results from PISA 2015 provide an opportunity to confirm the success of our world-class education systems from a global perspective. Canada remains in the group of top-performing countries and achieves its standing with relatively equitable outcomes. The trend in decreasing average scores noted in past PISA cycles stabilized in 2015. However, results from PISA as well as other pan-Canadian and international assessments show that several provinces have experienced a decline in the skill levels of their youth over the past decade. The comparative approach taken in this report does not lend itself to developing explanations for these changes. The report provides information for ministries and departments of education as well as for education partners to work together in validating current education policies, learning outcomes, teaching approaches and strategies, as well as resources to ensure that they continue meeting the needs of our society. Further analysis of the information collected through PISA will help readers gain a better understanding of the extent to which important background variables contribute to the differences in performance highlighted here. Reports on such secondary analysis will be available in forthcoming publications of Assessment Matters! (a series of articles available on the CMEC Web site).28 Today’s PISA teenagers will eventually become adults responsible for the success of our economy, so it is important to both celebrate the successes and address the challenges highlighted in this report. It is essential that our education systems contribute significantly in preparing Canadian youth for full participation in our modern society for generations to come. Assessment Matters! is available at http://www.cmec.ca/131/Programs-and-Initiatives/Assessment/Overview/index.html 28 45 Appendix A PISA 2015 sampling procedures, exclusion rates, and response rates The accuracy of PISA survey results depends on the quality of the information on which the sample is based, as well as the sampling procedures. The PISA 2015 sample for Canada was based on a two-stage stratified sample. The first stage consisted of sampling individual schools in which 15-year-old students were enrolled. Schools were sampled systematically, with probabilities proportional to size (the measure of size being a function of the estimated number of eligible 15-year-old students enrolled in the school). While a minimum of 150 schools were required to be selected in each country, in Canada a much larger sample of schools was selected to produce reliable estimates for each province and for each of the anglophone and francophone school systems in Nova Scotia, New Brunswick, Quebec, Ontario, Manitoba, Alberta, and British Columbia. The second stage of the selection process sampled students within the schools. Once schools were selected, a list of all 15-year-old students in each was prepared. From this list, up to 42 students were then selected with equal probability. All 15-year-old students were selected if fewer than 42 were enrolled. In Newfoundland and Labrador, Prince Edward Island, Nova Scotia, New Brunswick, and Quebec, as well as in the francophone school systems in Manitoba and Alberta, more than 42 students were selected in some schools to meet samplesize requirements. Each country participating in PISA attempted to maximize the coverage of PISA’s target population within the sampled schools. Within each sampled school, all eligible students (namely those 15 years of age), regardless of grade, were first listed. Tables A.1a and A.1b show the total number of excluded students by province who were then further described and classified into specific categories in accordance with the international standards. Students could be excluded based on three categories as determined by school staff: 1) students with a functional disability (i.e., the student has a moderate to severe permanent physical disability such that s/he cannot perform in the PISA testing situation); 2) students with an intellectual disability (the student has a mental or emotional disability and is cognitively delayed such that s/he cannot perform in the PISA testing situation); and 3) students with a limited proficiency in the assessment language (if the student is unable to read or speak any of the languages of the assessment in the country and would be unable to overcome the language barrier in the testing situation — typically a student who has received less than one year of instruction in the language of the assessment). The weighted student exclusion rate for Canada overall was 6.9 per cent which is above the maximum exclusion rate of 5 per cent allowed by quality standards in PISA. The weighted student exclusion rate ranged from 3.8 per cent in Quebec to 14.3 per cent in Prince Edward Island. Across all provinces the vast majority of exclusions was a result of an intellectual disability (category 2 above). Compared with PISA 2012, the weighted student exclusion rates increased by more than 2 per cent in Prince Edward Island, Manitoba, Alberta, and British Columbia, and decreased by more than 2 per cent in Newfoundland and Labrador. Steps will be required in future PISA cycles to address the issue of high exclusion rates for schools and students in some provinces. 47 Table A.1a PISA 2015 student exclusion rate Total number of eligible students sampled (participating, not participating, and excluded) Canada and provinces Unweighted* Weighted** Total number of students excluded Unweighted* Weighted** Student exclusion rate Unweighted* Weighted** Newfoundland and Labrador Prince Edward Island 1,662 543 5,579 1,625 85 80 303 233 5.1 14.7 5.4 14.3 Nova Scotia 2,014 9,594 153 754 7.6 7.9 New Brunswick 2,180 8,068 199 679 9.1 8.4 Quebec 5,734 72,433 145 2,743 2.5 3.8 Ontario 6,581 152,406 298 10,298 4.5 6.8 Manitoba 3,134 13,554 231 1,095 7.4 8.1 Saskatchewan 2,705 12,851 137 623 5.1 4.8 Alberta 3,312 42,814 200 3,366 6.0 7.9 British Columbia 2,944 47,475 302 5,247 10.3 11.1 30,809 366,399 1,830 25,340 5.9 6.9 Canada * Based on students selected to participate. ** Weighted based on student enrolment such that the total weighted value represents all 15-year-olds enrolled in the province and not just those selected for PISA. Table A.1b PISA 2015 student exclusion rate by type of exclusion Exclusion rate: students with a physical disability Unweighted* Canada and provinces Weighted** Exclusion rate: students with an intellectual disability Unweighted* Weighted** Exclusion rate: students with limited language skills Unweighted* Weighted** % % % % % % Newfoundland and Labrador Prince Edward Island 0.3 1.8 0.6 1.7 4.5 12.0 4.4 11.7 0.4 0.9 0.5 0.9 Nova Scotia 0.4 0.4 6.3 6.4 0.9 1.0 New Brunswick 1.4 0.9 7.5 7.4 0.2 0.2 Quebec 0.1 0.3 2.1 3.1 0.3 0.4 Ontario 0.3 0.3 3.4 5.2 0.9 1.2 Manitoba 0.4 0.5 5.0 5.0 2.0 2.6 Saskatchewan 0.4 0.3 3.5 3.5 1.2 1.0 Alberta 0.8 1.1 3.6 4.9 1.6 1.9 British Columbia 0.9 1.0 5.7 6.1 3.7 4.0 Canada 0.5 0.5 4.2 4.9 1.2 1.5 * Based on students selected to participate. ** Weighted based on student enrolment such that the total weighted value represents all 15-year-olds enrolled in the province and not just those selected for PISA. 48 To minimize the potential for response bias, data quality standards in PISA require minimum participation rates for schools and students. At the Canada-wide level, a minimum response rate of 85 per cent was required for schools initially selected. PISA 2015 also requires a minimum student participation rate of 80 per cent within all participating schools combined (original sample and replacements) at the national level. Table A.2 shows the response rates for schools and students, before and after replacement, for Canada and the 10 provinces. At the national level 1,010 schools were selected to participate in PISA 2015, and 703 of these initially selected schools participated. As such, the weighted school participation rate was 78.6 per cent for Canada which was slightly lower than the international standard. Canada was required to complete a non-response bias analysis for school-response rate. At the provincial level, school response rates after replacement ranged from 51.7 per cent in Quebec to 99.3 per cent in Prince Edward Island. Canada was required to conduct a non-response bias analysis to determine whether the data were of acceptable quality for inclusion in the PISA data set. Where the school response rates were below the international standard of 85 per cent, a non-response bias analysis was undertaken for Quebec, Ontario, and Alberta where weighted school response rates (after replacement) were 51.5, 81.9, and 80.4 per cent respectively. The following measures related to school characteristics and student achievement were used for this analysis: • In Quebec, these measures included school demographic data for all schools covered by PISA 2015 (type of funding of the school, language of the school, and size of the school), and average student-achievement scores for schools in science and reading. • In Ontario, these measures included school demographic data for all schools covered by PISA 2015 (type of funding of the school, language of the school, and size of the school), and school success rate for a 2015 provincial assessment for schools selected for the PISA sample. • In Alberta, these measures included school success rates for a provincial assessment covering the topics of math, reading, and science for all schools covered by PISA 2015 and demographic data for the full PISA sample. Quebec non-response analysis revealed potential bias. The results showed some differences by non-response in the percentage of English schools, the percentage of public schools, and the percentage of schools in each school size in Quebec. Results from the average student-achievement scores for schools in science showed significant differences between non-responding schools for the adjusted estimates of the mean and the median (absolute differences are 2.15 and 2.81 per cent respectively), and the corresponding population parameters. There was no significant difference in reading. Ontario non-response analysis revealed no potential bias. In Ontario, absolute differences between the distribution of the population and the non-response adjusted sample were found for the results of the language of the school, for school funding type, and for school size. However, these differences are attributed to the sample design and weighting strategy, and not non-response bias. Likewise, the analysis using the success rate for the provincial assessment showed no difference between the non-response adjusted estimates and the population parameters. 49 Alberta non-response analysis revealed no potential bias. In Alberta, differences between the distribution of the population and the non-response adjusted sample for the results of the language of the school and for school funding type were explained by small sample sizes for French and private schools. The differences found for the school size were attributed to the sample design and weighting strategy, and not non-response bias. In terms of the analysis using the success rate for the provincial assessment, very few statistically significant differences were observed between the non-response adjusted estimates and the population parameters estimates. Non-response bias conclusion Based on the non-response bias analysis, the PISA international consortium judged that the Canadian data overall were of suitable quality to be included fully in the PISA data sets without restrictions. However, the results from the province of Quebec are to be treated with caution because of a possible non-response bias, and should be annotated accordingly in all international regional analyses and national reporting. At the student level, Canada’s response rate after replacement was 80.8 per cent. Apart from Alberta and British Columbia, all provinces achieved a student response rate of 80 per cent or higher. Because Canada had undertaken a non-response bias analysis for schools, no additional analyses were required. The consortium deemed the Canadian and provincial data to be fully included in the PISA data sets. 50 Table A.2 Weighted Unweighted Weighted Weighted % student participation rate after replacement (participating and not participating) Unweighted Total number of students participating Weighted % Total number of eligible students sampled (participating and not participating) Number School response rate after replacement Weighted % School response rate before replacement Number Total number of selected schools (participating and not participating) PISA 2015 school and student response rates 55 49 97.0 49 97.0 1,489 4,897 1,197 3,959 80.9 21 18 99.3 18 99.3 448 1,323 392 1,164 88.0 Nova Scotia 59 53 98.6 54 98.7 1,758 8,505 1,414 6,882 80.9 New Brunswick 68 53 96.0 53 96.0 1,832 6,669 1,544 5,488 82.3 Quebec 180 78 40.3 93 51.7 3,543 35,531 2,885 28,941 81.5 Ontario 200 131 78.5 136 81.9 5,034 113,570 4,123 92,974 81.9 Manitoba 112 85 92.4 85 92.4 2,712 11,017 2,285 9,191 83.4 Saskatchewan 106 83 92.6 83 92.6 2,327 10,609 1,894 8,637 81.4 Alberta 114 80 80.4 80 80.4 2,558 30,495 1,973 23,559 77.3 95 73 89.9 75 92.3 2,425 37,770 1,897 29,678 78.6 1,010 703 74.5 726 78.6 24,126 260,387 19,604 210,476 80.8 Canada and provinces Newfoundland and Labrador Prince Edward Island British Columbia Canada Note: School response rates were weighted based on student enrolment. Table A.2 51 Appendix B PISA 2015 data tables Table B.1.1 Percentage of students at each proficiency level for countries, economies, and provinces: SCIENCE Country, economy, or province Vietnam Macao Quebec Alberta British Columbia Estonia Hong Kong-China Singapore Japan Canada Prince Edward Island Finland Ontario Chinese Taipei Nova Scotia Korea Slovenia Ireland Newfoundland and Labrador New Brunswick Denmark BSJG-China Poland Saskatchewan Germany Latvia Portugal United Kingdom New Zealand Manitoba Australia Russian Federation Spain Switzerland The Netherlands Norway Belgium United States Czech Republic Austria Sweden France Italy 52 Below Level 1 0.2 1.2 1.4 1.3 1.3 1.3 1.6 2.1 1.9 2.0 Standard error (0.1) (0.2) (0.4) (0.3) (0.4) (0.2) (0.3) (0.2) (0.3) (0.2) 1.6 Level 1 Level 2 5.7 6.9 7.0 7.3 7.4 7.5 7.8 7.5 7.7 9.1 Standard error (0.7) (0.4) (1.0) (0.9) (1.2) (0.6) (0.6) (0.5) (0.6) (0.4) (0.8) 9.7 2.6 2.3 3.0 2.2 3.3 3.1 3.0 (0.4) (0.4) (0.3) (0.7) (0.4) (0.3) (0.4) 3.0 2.7 3.3 4.4 2.9 3.2 4.2 2.7 3.4 3.8 4.4 3.6 4.8 3.0 4.0 4.5 4.3 4.7 5.4 4.8 4.6 5.0 6.6 6.7 6.0 % Proficiency levels Level 3 25.3 20.6 18.0 18.0 18.6 20.1 19.7 15.1 18.1 20.2 Standard error (1.4) (0.7) (1.3) (1.4) (1.4) (0.7) (0.9) (0.5) (0.8) (0.6) (2.0) 24.0 8.9 10.0 9.4 10.6 11.1 11.9 12.4 (0.6) (0.9) (0.6) (1.3) (0.7) (0.5) (0.8) (0.7) 12.5 (0.8) (0.3) (0.6) (0.4) (0.7) (0.5) (0.3) (0.4) (0.3) (0.3) (1.0) (0.3) (0.4) (0.4) (0.5) (0.5) (0.4) (0.4) (0.5) (0.5) (0.5) (0.6) (0.6) (0.6) 12.9 12.5 11.8 13.3 13.5 12.8 14.5 14.0 13.6 13.0 13.9 12.8 15.2 14.3 13.9 14.3 14.0 14.4 15.5 16.1 15.8 15.0 15.3 17.2 % 36.6 34.2 30.8 29.6 30.2 30.7 36.1 23.4 28.2 30.3 Standard error (1.2) (0.9) (1.5) (1.6) (1.7) (0.9) (0.9) (0.6) (0.9) (0.5) (2.9) 34.8 19.1 20.7 18.1 22.3 21.7 23.3 26.4 (0.7) (1.1) (0.6) (1.3) (0.9) (0.7) (0.9) (1.3) 24.1 (1.6) (0.7) (0.9) (0.7) (1.2) (0.7) (0.7) (0.9) (0.7) (0.8) (1.6) (0.5) (1.0) (0.7) (0.8) (0.7) (0.7) (0.6) (0.8) (0.8) (0.8) (0.9) (0.6) (0.8) 24.3 25.9 20.7 26.6 28.6 22.7 29.8 25.4 22.6 21.6 25.1 21.6 31.2 26.5 22.8 21.8 24.6 21.9 25.5 25.9 23.9 24.0 22.0 27.1 % Level 4 Level 5 Level 6 23.9 28.0 29.9 27.9 27.9 26.9 27.4 27.7 28.8 26.1 Standard error (1.2) (0.7) (1.4) (1.6) (1.6) (0.9) (1.1) (0.7) (0.9) (0.7) 7.1 8.3 11.0 13.1 11.9 11.6 6.9 18.6 12.9 10.4 Standard error (0.8) (0.5) (1.2) (1.3) (1.4) (0.7) (0.6) (0.7) (0.8) (0.5) (3.4) 21.3 (3.2) 8.1 (1.9) 0.6 (0.7) 29.2 30.0 27.0 31.4 29.2 29.1 31.1 (0.8) (1.0) (0.9) (1.8) (0.9) (0.9) (0.9) 26.0 25.0 27.1 23.7 24.0 22.1 20.1 (0.8) (1.5) (0.8) (2.0) (1.0) (0.8) (0.8) 11.9 10.0 12.7 8.7 9.2 9.1 6.3 (0.6) (1.0) (0.8) (1.0) (0.7) (0.6) (0.4) 2.4 2.0 2.7 1.1 1.4 1.5 0.8 (0.3) (0.4) (0.5) (0.4) (0.2) (0.3) (0.2) (1.8) 31.1 (1.8) 21.5 (1.4) 7.0 (1.0) 0.7 (0.4) (1.7) (0.9) (1.1) (0.9) (1.5) (0.8) (0.8) (0.8) (0.7) (0.8) (1.6) (0.5) (0.9) (0.7) (0.8) (0.9) (0.8) (0.6) (0.8) (0.8) (0.8) (0.9) (0.9) (0.9) 30.8 31.1 25.8 29.9 30.4 27.7 31.7 28.8 27.5 26.3 30.8 27.3 30.9 31.3 26.3 26.1 29.1 26.8 26.6 27.7 28.1 26.8 26.5 28.6 (2.2) (1.1) (1.1) (0.9) (1.6) (0.8) (0.8) (0.8) (0.7) (0.8) (1.6) (0.5) (0.9) (0.7) (1.1) (0.9) (0.8) (0.7) (0.9) (0.9) (0.8) (0.9) (0.8) (1.0) 21.2 20.2 23.8 19.9 18.1 22.0 17.4 21.0 21.6 21.8 19.5 22.3 16.0 18.9 22.7 22.4 19.6 22.5 19.1 18.4 19.5 19.0 21.4 17.0 (1.6) (0.8) (1.1) (0.8) (1.2) (0.8) (0.8) (0.8) (0.7) (0.8) (1.7) (0.5) (0.9) (0.7) (1.0) (0.8) (0.8) (0.7) (0.9) (0.7) (0.8) (0.9) (0.8) (0.7) 7.3 6.1 11.5 6.3 5.6 8.8 3.5 6.7 9.1 10.1 6.3 9.2 3.5 4.7 8.6 9.5 6.9 8.0 7.3 6.3 6.8 7.2 7.2 3.8 (1.0) (0.5) (1.1) (0.5) (0.7) (0.6) (0.4) (0.5) (0.6) (0.6) (1.1) (0.4) (0.4) (0.4) (0.6) (0.5) (0.5) (0.4) (0.6) (0.4) (0.5) (0.6) (0.5) (0.4) 0.8 0.9 2.1 1.0 0.6 1.8 0.3 0.7 1.8 2.7 0.8 2.0 0.2 0.3 1.1 1.6 1.1 1.0 1.2 0.9 0.9 1.3 0.8 0.2 (0.3) (0.2) (0.5) (0.2) (0.3) (0.2) (0.1) (0.1) (0.2) (0.4) (0.6) (0.2) (0.1) (0.1) (0.2) (0.2) (0.2) (0.1) (0.2) (0.2) (0.2) (0.2) (0.1) (0.1) % % % % Standard error 1.2 (0.5) 0.9 (0.2) 1.8 (0.5) 2.8 (0.5) 2.7 (0.5) 1.9 (0.3) 0.4 (0.1) 5.6 (0.4) 2.4 (0.4) 2.0 (0.2) Table B.1.1 (cont’d) Percentage of students at each proficiency level for countries, economies, and provinces: SCIENCE Country, economy, or province Croatia Lithuania Iceland Luxembourg Hungary Kazakhstan Slovak Republic Israel Malta Greece Malaysia Chile Bulgaria Romania Argentina Uruguay Albania United Arab Emirates Cyprus Moldova Turkey Trinidad and Tobago Costa Rica Thailand Mexico Colombia Jordan Qatar Georgia Montenegro Indonesia Brazil Peru Lebanon Republic of Macedonia Tunisia Kosovo Algeria Dominican Republic OECD average Below Level 1 Level 1 Level 2 Proficiency levels Level 3 Level 4 % Standard error 5.5 (0.5) 5.9 (0.5) 6.6 (0.5) 6.9 (0.5) 7.6 (0.7) 4.3 (0.6) 11.0 (0.9) 11.5 (0.9) 14.5 (0.6) 10.3 (1.1) 7.8 (0.8) 9.8 (0.6) 15.1 (1.3) 10.2 (0.9) 11.5 (0.9) 12.4 (0.8) 11.9 (0.9) % Standard error 19.2 (1.0) 18.9 (0.8) 18.7 (0.9) 18.9 (0.6) 18.4 (0.9) 23.8 (1.3) 19.7 (0.8) 19.9 (0.9) 18.0 (0.9) 22.4 (1.1) 25.9 (1.2) 25.0 (0.9) 22.8 (1.1) 28.4 (1.4) 28.2 (1.0) 28.4 (0.9) 29.8 (1.2) % Standard error 29.5 (0.9) 29.7 (0.9) 29.0 (1.0) 24.8 (0.7) 25.5 (0.8) 38.2 (1.2) 27.6 (0.8) 24.4 (0.8) 23.4 (0.8) 28.4 (1.1) 36.4 (1.0) 31.0 (1.0) 25.2 (1.1) 35.0 (1.4) 34.2 (1.0) 30.3 (0.8) 34.5 (1.0) % Standard error 27.5 (1.0) 26.3 (0.7) 27.3 (0.9) 25.1 (0.7) 27.3 (0.9) 23.9 (1.3) 24.8 (0.7) 23.3 (1.0) 21.7 (0.9) 25.2 (1.1) 23.6 (1.1) 23.8 (0.9) 22.6 (1.2) 19.9 (1.0) 20.1 (1.1) 20.3 (0.8) 18.9 (1.3) % Standard error 14.4 (0.7) 15.1 (0.7) 14.6 (0.8) 17.3 (0.6) 16.6 (0.8) 8.1 (0.9) 13.3 (0.6) 15.0 (0.8) 14.8 (0.9) 11.6 (0.9) 5.8 (0.6) 9.1 (0.7) 11.4 (0.9) 5.9 (0.7) 5.3 (0.5) 7.4 (0.5) 4.5 (0.6) 15.6 (0.8) 26.1 (0.7) 26.9 (0.6) 19.0 (0.7) 9.5 15.3 14.1 12.9 17.9 10.8 13.0 12.8 16.2 19.4 21.8 20.3 18.9 15.6 24.2 21.8 30.4 (0.6) (0.8) (1.1) (0.7) (0.7) (0.8) (0.8) (1.0) (1.1) (0.5) (1.1) (0.5) (1.2) (0.8) (1.0) (1.6) 26.9 28.2 31.6 27.9 35.6 33.7 35.0 32.8 30.4 28.0 30.5 32.1 40.4 32.4 36.7 32.3 (0.8) (0.8) (1.5) (0.9) (1.0) (1.1) (1.0) (0.9) (0.9) (0.6) (1.1) (0.7) (1.5) (0.6) (1.0) (1.2) 28.6 31.5 31.3 27.1 35.5 32.2 34.7 30.6 30.9 24.6 28.2 29.0 31.7 25.4 27.9 22.0 (0.8) (1.2) (1.3) (0.8) (0.8) (0.9) (0.9) (0.9) (1.0) (0.5) (1.0) (0.6) (1.3) (0.6) (1.0) (1.2) 19.6 19.7 19.1 18.3 15.2 16.0 15.1 15.9 16.1 16.4 15.2 15.1 10.6 13.1 11.5 11.6 (0.7) (0.9) (1.4) (0.7) (0.9) (0.8) (0.9) (0.7) (0.9) (0.5) (0.7) (0.5) (0.8) (0.6) (0.7) (0.9) 29.1 (0.8) 33.8 (0.9) 24.6 (0.7) 10.3 21.7 28.4 28.0 55.4 (1.2) (1.1) (1.3) (1.6) 44.2 39.3 42.8 30.4 (1.1) (1.1) (1.0) (1.3) 26.6 24.4 22.7 11.3 (1.1) (1.0) (1.1) (0.8) 5.5 (0.1) 15.7 (0.1) 24.8 (0.1) Level 5 Level 6 % Standard error 3.6 (0.4) 3.9 (0.5) 3.5 (0.4) 6.0 (0.4) 4.3 (0.4) 1.7 (0.5) 3.3 (0.3) 5.1 (0.5) 6.1 (0.4) 2.0 (0.3) 0.6 (0.2) 1.2 (0.2) 2.7 (0.4) 0.7 (0.2) 0.7 (0.2) 1.2 (0.2) 0.3 (0.1) % Standard error 0.4 (0.1) 0.3 (0.1) 0.3 (0.1) 0.9 (0.2) 0.3 (0.1) 0.1 (0.1) 0.3 (0.1) 0.7 (0.1) 1.6 (0.3) 0.1 (0.1) 0.0 (0.0) 0.0 (0.0) 0.2 (0.1) 0.0 (0.0) 0.0 (0.0) 0.1 (0.0) 0.0 (0.0) (0.5) 2.5 (0.2) 0.2 (0.1) 8.1 5.9 4.8 7.3 2.7 4.6 2.3 4.1 3.1 7.5 4.9 4.4 1.6 4.2 2.0 3.3 (0.4) (0.6) (0.9) (0.5) (0.4) (0.6) (0.3) (0.4) (0.4) (0.3) (0.5) (0.3) (0.3) (0.4) (0.3) (0.4) 1.5 0.7 0.3 1.3 0.1 0.4 0.1 0.3 0.2 1.6 0.8 0.5 0.1 0.6 0.1 0.4 (0.2) (0.1) (0.1) (0.2) (0.1) (0.2) (0.1) (0.1) (0.1) (0.1) (0.2) (0.1) (0.1) (0.1) (0.1) (0.1) 0.1 0.0 0.0 0.1 0.0 0.0 0.0 0.0 0.0 0.1 0.1 0.0 0.0 0.0 0.0 0.0 (0.1) (0.0) (0.0) (0.1) (0.0) (0.0) (0.0) (0.0) (0.0) (0.0) (0.1) (0.0) (0.0) (0.0) (0.0) (0.0) (0.5) 2.0 (0.3) 0.2 (0.1) 0.0 (0.0) 6.8 7.2 5.6 2.6 (0.6) (0.7) (0.6) (0.5) 0.7 0.7 0.9 0.3 (0.3) (0.2) (0.2) (0.1) 0.0 0.0 0.0 0.0 (0.0) (0.0) (0.0) (0.0) 0.0 0.0 0.0 0.0 (0.0) (0.0) (0.0) (0.0) 27.2 (0.1) 19.0 (0.1) 6.7 (0.1) 1.1 (0.0) Note: Countries, economies, and provinces have been sorted in descending order by the total percentage of students who attained Level 2 or higher. BSJG-China represents Beijing, Shanghai, Jiangsu, and Guangdong. The coverage of Argentina, Kazakhstan, and Malaysia is too small to ensure comparability. See OECD, PISA 2015 Results: Excellence and Equity in Education, Volume I (Paris: OECD, 2016) for a note regarding Cyprus. Below Level 1 consists of students who scored at below Level 1 and Level 1b. Level 1 refers to Level 1a. 53 Table B.1.2 Estimated average scores and confidence intervals for countries, economies, and provinces: SCIENCE Country, economy, or province Confidence Confidence Average Standard interval – 95% interval – 95% error lower limit upper limit Singapore Alberta British Columbia 556 541 539 (1.2) (4.0) (4.3) 553 533 530 558 549 547 Japan Quebec 538 537 (3.0) (4.7) 533 528 544 546 Estonia Chinese Taipei Finland Macao-China Canada 534 532 531 529 528 (2.1) (2.7) (2.4) (1.1) (2.1) 530 527 526 526 524 538 538 535 531 532 Vietnam Ontario 525 524 (3.9) (3.9) 517 516 532 532 Hong Kong-China BSJG-China Nova Scotia 523 518 517 (2.5) (4.6) (4.5) 518 509 508 528 527 526 Korea Prince Edward Island 516 515 (3.1) (5.4) 510 504 522 525 New Zealand Slovenia Australia United Kingdom Germany The Netherlands New Brunswick Newfoundland and Labrador 513 513 510 509 509 509 506 (2.4) (1.3) (1.5) (2.6) (2.7) (2.3) (4.5) 509 510 507 504 504 504 498 518 515 513 514 514 513 515 506 (3.2) 500 512 Switzerland Ireland Belgium Denmark Poland Portugal Manitoba 506 503 502 502 501 501 499 (2.9) (2.4) (2.3) (2.4) (2.5) (2.4) (4.7) 500 498 498 497 497 496 490 511 507 506 507 506 506 509 Norway United States Saskatchewan 498 496 496 (2.3) (3.2) (3.1) 494 490 490 503 502 502 Austria France Sweden Czech Republic Spain Latvia Russian Federation Luxembourg Italy Hungary Lithuania Croatia 495 495 493 493 493 490 487 483 481 477 475 475 (2.4) (2.1) (3.6) (2.3) (2.1) (1.6) (2.9) (1.1) (2.5) (2.4) (2.7) (2.5) 490 491 486 488 489 487 481 481 476 472 470 471 500 499 500 497 497 493 492 485 485 481 481 480 54 Country, economy, or province Confidence Confidence Average Standard interval – 95% interval – 95% error lower limit upper limit Iceland 473 (1.7) 470 Israel 467 (3.4) 460 Malta 465 (1.6) 462 Slovak Republic 461 (2.6) 456 Kazakhstan 456 (3.7) 449 Greece 455 (3.9) 447 Chile 447 (2.4) 442 Bulgaria 446 (4.4) 437 Malaysia 443 (3.0) 437 United Arab Emirates 437 (2.4) 432 Uruguay 435 (2.2) 431 Romania 435 (3.2) 429 Cyprus 433 (1.4) 430 Argentina 432 (2.9) 427 Moldova 428 (2.0) 424 Albania 427 (3.3) 421 Turkey 425 (3.9) 418 Trinidad and Tobago 425 (1.4) 422 Thailand 421 (2.8) 416 Costa Rica 420 (2.1) 416 Qatar 418 (1.0) 416 Colombia 416 (2.4) 411 Mexico 416 (2.1) 412 Montenegro 411 (1.0) 409 Georgia 411 (2.4) 406 Jordan 409 (2.7) 403 Indonesia 403 (2.6) 398 Brazil 401 (2.3) 396 Peru 397 (2.4) 392 Lebanon 386 (3.4) 380 Tunisia 386 (2.1) 382 Republic of Macedonia 384 (1.2) 381 Kosovo 378 (1.7) 375 Algeria 376 (2.6) 371 Dominican Republic 332 (2.6) 327 Note : The OECD average was 493, with a standard error of 0.4. Countries, economies, and provinces have been sorted in descending order by average score. BSJG-China represents Beijing, Shanghai, Jiangsu, and Guangdong. The coverage of Argentina, Kazakhstan, and Malaysia is too small to ensure comparability. See OECD, PISA 2015 Results for a note regarding Cyprus. 477 473 468 466 464 463 452 454 449 441 440 441 435 438 432 434 433 427 427 424 420 420 420 413 416 414 408 405 401 393 391 386 382 381 337 Table B.1.3 Estimated average scores and confidence intervals for Canada and the provinces: SCIENCE BY COMPETENCY SUBSCALES Average Standard error Confidence interval – 95% lower limit Confidence interval – 95% upper limit Canada 530 (2.1) 526 534 Newfoundland and Labrador 509 (3.5) 502 516 Prince Edward Island 516 (5.6) 505 527 Nova Scotia 519 (4.9) 509 528 New Brunswick 509 (4.5) 500 518 Quebec 537 (5.2) 527 547 Ontario 525 (3.8) 518 533 Manitoba 504 (5.0) 494 514 Saskatchewan 501 (3.3) 494 508 Alberta 547 (4.6) 538 556 British Columbia 542 (4.5) 533 550 Canada and provinces Explain phenomena scientifically Note: The OECD average was 493, with a standard error of 0.5. Evaluate and design scientific enquiry Canada 530 (2.7) 524 535 Newfoundland and Labrador 506 (4.0) 498 514 Prince Edward Island 515 (7.0) 502 529 Nova Scotia 516 (6.1) 504 528 New Brunswick 508 (5.6) 497 519 Quebec 542 (5.5) 532 553 Ontario 527 (5.0) 517 537 Manitoba 498 (5.5) 487 509 Saskatchewan 495 (3.9) 488 503 Alberta 540 (4.9) 530 549 British Columbia 537 (5.8) Note: The OECD average was 493, with a standard error of 0.5. 526 549 Canada 525 (2.7) 520 530 Newfoundland and Labrador 501 (3.5) 494 508 Prince Edward Island 512 (6.1) 500 524 Nova Scotia 514 (5.5) 503 525 New Brunswick 503 (5.3) 493 513 Quebec 536 (5.3) 525 546 Ontario 521 (4.8) 512 530 Manitoba 498 (4.7) 488 507 Saskatchewan 491 (3.4) 485 498 Alberta 537 (4.7) 527 546 British Columbia 536 (5.8) Note: The OECD average was 493, with a standard error of 0.5. 525 547 Interpret data and evidence scientifically 55 Table B.1.4 Estimated average scores and confidence intervals for Canada and the provinces: SCIENCE BY KNOWLEDGE SUBSCALES Average Standard error Confidence interval – 95% lower limit Confidence interval – 95% upper limit Canada 528 (2.2) 524 533 Newfoundland and Labrador 507 (3.4) 501 514 Prince Edward Island 517 (6.7) 504 530 Nova Scotia 517 (4.5) 509 526 New Brunswick 508 (5.2) 498 518 Quebec 537 (5.1) 527 546 Ontario 523 (4.0) 516 531 Manitoba 502 (4.7) 493 511 Saskatchewan 499 (3.4) 492 505 Alberta 545 (4.3) 537 554 British Columbia 540 (4.4) 532 549 Canada and provinces Content Note: The OECD average was 493, with a standard error of 0.5. Procedural and epistemic 56 Canada 528 (2.4) 523 532 Newfoundland and Labrador 504 (3.4) 498 511 Prince Edward Island 514 (5.6) 503 525 Nova Scotia 515 (4.9) 505 524 New Brunswick 505 (5.0) 495 515 Quebec 538 (5.1) 528 548 Ontario 525 (4.4) 517 534 Manitoba 498 (4.7) 489 507 Saskatchewan 493 (3.3) 487 500 Alberta 538 (4.5) 529 547 British Columbia 537 (4.7) Note: The OECD average was 493, with a standard error of 0.4. 528 547 Table B.1.5 Estimated average scores and confidence intervals for Canada and the provinces: SCIENCE BY CONTENT SUBSCALES Average Standard error Confidence interval – 95% lower limit Confidence interval – 95% upper limit Canada 527 (2.4) 523 532 Newfoundland and Labrador 506 (4.6) 497 516 Prince Edward Island 518 (6.3) 505 530 Nova Scotia 517 (5.3) 507 527 New Brunswick 505 (5.0) 495 515 Quebec 537 (5.2) 526 547 Ontario 524 (4.3) 515 532 Manitoba 502 (5.0) 493 512 Saskatchewan 498 (4.2) 489 506 Alberta 543 (4.9) 533 553 British Columbia 534 (5.2) 524 545 Canada and provinces Physical systems Note: The OECD average was 493, with a standard error of 0.5. Living systems Canada 528 (2.4) 523 532 Newfoundland and Labrador 505 (3.6) 498 512 Prince Edward Island 516 (6.0) 504 527 Nova Scotia 518 (4.8) 509 528 New Brunswick 507 (5.2) 497 517 Quebec 535 (5.0) 525 545 Ontario 525 (4.5) 516 533 Manitoba 497 (4.8) 488 507 Saskatchewan 493 (3.5) 486 500 Alberta 539 (4.7) 530 548 British Columbia 543 (4.9) Note: The OECD average was 492, with a standard error of 0.5. 533 552 Canada 529 (2.5) 524 534 Newfoundland and Labrador 503 (4.0) 495 511 Prince Edward Island 516 (5.7) 505 528 Nova Scotia 515 (5.1) 505 525 New Brunswick 508 (5.6) 497 519 Quebec 542 (5.4) 532 553 Ontario 525 (4.3) 516 533 Manitoba 500 (5.0) 491 510 Saskatchewan 498 (4.1) 490 506 Alberta 542 (5.2) 532 553 British Columbia 538 (6.1) Note: The OECD average was 494, with a standard error of 0.5. 526 550 Earth and space systems 57 Table B.1.6 Variation in student performance for countries, economies, and provinces: SCIENCE Percentiles Country, economy, or province 5th 10th 25th 75th 90th Standard Score error Standard Score error Standard Score error Standard Score error Standard Score error Difference in score points between 95th the 10th and 90th Standard percentiles Score error Tunisia 287 (3.1) 306 (2.6) 341 (2.2) 428 (2.5) 472 (3.8) 500 (5.3) 166 Algeria 268 (3.4) 291 (3.3) 329 (2.5) 419 (3.2) 465 (4.5) 496 (6.1) 174 Indonesia 296 (4.1) 319 (3.2) 356 (2.9) 447 (3.3) 493 (3.9) 522 (4.9) 175 Costa Rica 310 (2.6) 332 (2.3) 370 (2.3) 466 (2.8) 514 (3.3) 541 (3.7) 182 Mexico 301 (3.2) 325 (2.5) 366 (2.2) 464 (2.8) 510 (3.1) 535 (3.4) 185 Dominican Republic 224 (3.0) 244 (2.7) 281 (2.5) 376 (3.3) 429 (4.9) 461 (6.3) 185 Kosovo 266 (3.3) 289 (2.2) 328 (2.2) 426 (2.2) 474 (3.7) 501 (4.3) 185 Kazakhstan 340 (4.2) 363 (3.3) 403 (3.2) 505 (4.6) 558 (6.9) 590 (8.7) 195 Vietnam 404 (4.7) 428 (4.1) 470 (4.3) 576 (4.5) 624 (6.6) 655 (8.3) 196 Malaysia 320 (3.7) 345 (3.5) 389 (3.4) 496 (3.4) 541 (3.9) 568 (5.0) 196 Peru 278 (3.2) 301 (2.6) 342 (2.4) 448 (3.3) 500 (3.9) 529 (4.7) 198 Albania 301 (3.8) 328 (3.2) 373 (3.2) 481 (4.8) 530 (5.0) 558 (4.7) 202 Thailand 301 (2.7) 324 (2.9) 365 (2.6) 473 (3.6) 528 (4.9) 559 (6.0) 203 Romania 309 (4.2) 334 (3.8) 379 (3.6) 488 (4.1) 539 (5.1) 570 (5.4) 205 Turkey 301 (3.8) 325 (3.5) 368 (3.7) 482 (5.5) 532 (6.1) 560 (5.7) 207 Argentina 303 (4.1) 329 (3.5) 376 (3.4) 487 (3.4) 536 (3.7) 567 (4.1) 207 Colombia 291 (3.9) 315 (3.1) 357 (2.8) 471 (2.9) 524 (3.4) 554 (3.5) 208 Hong Kong-China 379 (5.5) 413 (4.5) 473 (3.5) 579 (2.6) 622 (2.7) 646 (3.2) 209 Macao-China 389 (3.6) 420 (2.3) 474 (1.7) 586 (1.8) 630 (2.0) 656 (3.2) 210 Latvia 355 (3.3) 382 (3.0) 432 (2.4) 548 (2.0) 596 (2.2) 623 (3.3) 214 Russian Federation 352 (4.1) 379 (3.8) 428 (3.4) 544 (3.3) 595 (3.5) 623 (3.7) 215 Jordan 268 (5.2) 299 (3.8) 351 (3.4) 468 (3.0) 517 (3.4) 544 (3.5) 217 Republic of Macedonia 248 (3.2) 277 (3.0) 325 (1.9) 440 (2.1) 496 (2.7) 528 (4.1) 219 Montenegro 277 (2.8) 304 (2.1) 352 (1.5) 468 (1.9) 526 (2.9) 558 (3.1) 221 Moldova 290 (4.0) 318 (3.0) 367 (2.6) 488 (2.9) 541 (3.1) 570 (3.8) 223 Prince Edward Island 370 (15.7) 404 (10.3) 459 (7.6) 571 (8.7) 627 (13.1) 654 (10.2) 223 Chile 308 (3.1) 336 (2.7) 385 (3.0) 509 (3.2) 560 (3.3) 589 (3.4) 225 Uruguay 301 (2.8) 326 (2.6) 372 (2.4) 496 (3.0) 552 (3.6) 583 (4.2) 226 Quebec 383 (6.9) 419 (7.0) 479 (6.5) 598 (5.6) 645 (5.8) 673 (6.7) 226 Saskatchewan 352 (6.9) 383 (5.2) 435 (4.7) 557 (4.2) 611 (4.4) 643 (6.0) 229 Ireland 356 (5.0) 387 (3.9) 441 (3.2) 565 (2.5) 618 (2.5) 648 (3.2) 231 Spain 344 (4.0) 374 (3.5) 432 (2.9) 556 (2.4) 605 (2.4) 633 (2.9) 231 Brazil 265 (2.4) 291 (2.1) 337 (1.9) 460 (3.3) 522 (4.1) 558 (4.6) 231 Estonia 384 (4.3) 416 (3.3) 473 (2.7) 597 (2.7) 648 (2.9) 677 (3.7) 233 Croatia 332 (3.5) 360 (3.3) 411 (3.4) 538 (2.8) 593 (3.3) 624 (3.9) 233 Georgia 267 (3.8) 297 (3.7) 348 (3.0) 471 (3.1) 531 (3.9) 566 (4.5) 233 Denmark 351 (3.8) 383 (3.6) 440 (3.1) 565 (2.8) 617 (3.2) 648 (4.0) 234 Lebanon 249 (4.6) 276 (3.9) 322 (3.6) 446 (5.1) 511 (4.9) 545 (5.2) 235 Poland 354 (4.3) 384 (3.4) 437 (2.9) 565 (3.1) 619 (3.5) 650 (4.0) 235 Nova Scotia 366 (9.6) 397 (7.2) 455 (6.5) 580 (5.0) 632 (6.7) 662 (6.4) 235 British Columbia 381 (6.9) 417 (7.6) 478 (5.3) 601 (5.1) 654 (5.8) 686 (6.9) 237 New Brunswick 355 (9.5) 387 (9.4) 443 (7.2) 571 (5.4) 623 (5.8) 652 (6.7) 237 Alberta 384 (6.2) 419 (5.7) 479 (5.7) 605 (4.5) 657 (5.1) 685 (4.9) 238 58 Table B.1.6 (cont’d) Variation in student performance for countries, economies, and provinces: SCIENCE Percentiles Country, economy, or province 5th 10th 25th 75th 90th Standard Score error Standard Score error Standard Score error Standard Score error Standard Score error Difference in score points between 95th the 10th and 90th Standard percentiles Score error Iceland 324 (3.5) 354 (3.1) 408 (2.9) 538 (2.3) 593 (3.3) 622 (3.9) 238 Manitoba 347 (8.9) 379 (6.5) 435 (6.2) 564 (6.0) 619 (7.1) 647 (7.7) 239 Lithuania 329 (3.2) 357 (3.8) 410 (2.9) 540 (3.3) 597 (3.7) 626 (4.3) 240 Italy Newfoundland and Labrador Canada 328 (4.1) 359 (3.8) 415 (3.2) 547 (2.8) 599 (2.8) 626 (3.3) 240 352 (7.4) 382 (6.8) 444 (5.3) 571 (4.6) 622 (4.8) 651 (6.9) 240 369 (3.3) 404 (2.9) 465 (2.5) 593 (2.2) 644 (2.6) 674 (2.7) 240 Portugal 349 (3.8) 379 (3.2) 435 (3.4) 568 (2.7) 620 (3.1) 649 (3.1) 241 Greece 305 (5.7) 333 (5.6) 388 (5.2) 522 (3.8) 575 (4.1) 604 (4.5) 241 Cyprus 286 (2.9) 314 (2.4) 365 (2.1) 497 (2.2) 557 (2.8) 590 (4.1) 243 Japan 375 (5.3) 412 (4.4) 475 (3.9) 605 (3.2) 655 (4.0) 683 (4.7) 243 Trinidad and Tobago 279 (4.0) 306 (3.5) 356 (1.9) 491 (2.1) 551 (3.3) 585 (3.7) 244 Ontario 364 (5.8) 398 (5.1) 460 (4.8) 591 (4.3) 643 (5.1) 674 (5.3) 246 Korea 352 (4.7) 388 (4.5) 451 (3.8) 584 (3.3) 636 (3.7) 665 (3.9) 248 Finland 364 (4.6) 402 (4.2) 466 (3.5) 599 (2.5) 651 (2.7) 681 (3.5) 250 Slovenia 354 (3.1) 386 (2.6) 445 (2.1) 581 (2.1) 636 (3.0) 667 (3.6) 250 Czech Republic 338 (4.1) 367 (3.7) 424 (3.4) 561 (2.5) 618 (3.1) 650 (3.8) 251 Norway 338 (3.8) 370 (3.3) 432 (3.0) 566 (2.9) 622 (3.3) 655 (3.9) 251 Hungary 319 (4.0) 347 (4.1) 406 (3.5) 547 (3.0) 601 (3.5) 630 (3.7) 254 Austria 335 (3.8) 365 (3.4) 424 (3.6) 565 (2.8) 621 (3.0) 652 (3.6) 256 United States 336 (4.1) 368 (3.9) 425 (3.7) 567 (3.9) 626 (3.9) 658 (4.9) 258 Qatar 268 (1.9) 295 (1.8) 344 (1.3) 486 (2.1) 554 (1.9) 589 (2.4) 259 Slovak Republic 296 (5.3) 329 (4.6) 391 (3.6) 532 (2.8) 588 (3.2) 621 (3.7) 259 United Arab Emirates 284 (3.3) 312 (2.8) 364 (2.8) 505 (3.2) 571 (3.2) 608 (3.0) 259 Switzerland 339 (4.7) 373 (4.1) 433 (4.3) 580 (3.3) 632 (2.9) 662 (3.3) 259 Chinese Taipei 358 (4.6) 395 (4.6) 465 (3.5) 603 (3.5) 655 (4.2) 685 (4.9) 260 Germany 342 (4.4) 376 (4.3) 439 (3.6) 580 (2.8) 636 (2.9) 669 (3.8) 260 United Kingdom 345 (2.9) 377 (3.2) 438 (2.9) 581 (3.1) 638 (3.2) 670 (3.5) 261 Luxembourg 323 (2.9) 351 (2.6) 407 (2.2) 556 (1.7) 615 (2.3) 649 (3.1) 264 Belgium 332 (3.4) 364 (3.8) 429 (3.5) 577 (2.2) 629 (2.1) 657 (2.2) 265 Bulgaria 283 (4.8) 313 (4.8) 370 (5.3) 521 (5.1) 578 (5.2) 611 (5.6) 266 The Netherlands 341 (4.0) 372 (4.3) 434 (3.9) 583 (2.5) 638 (2.9) 668 (3.6) 266 Australia 336 (2.6) 372 (2.5) 438 (2.2) 583 (1.9) 639 (2.2) 672 (2.8) 267 France 322 (4.1) 355 (3.7) 421 (3.4) 571 (2.4) 623 (2.8) 652 (3.3) 268 Sweden 322 (4.7) 357 (4.6) 421 (4.2) 567 (4.2) 625 (4.0) 658 (4.4) 269 BSJG-China 341 (6.5) 377 (6.0) 445 (5.6) 595 (5.3) 649 (5.6) 677 (6.5) 271 Singapore 373 (3.7) 412 (2.8) 485 (2.2) 631 (1.8) 683 (2.2) 712 (3.1) 271 New Zealand 341 (3.5) 374 (3.8) 439 (3.8) 588 (2.8) 647 (3.5) 682 (3.8) 273 Israel 295 (4.9) 327 (4.6) 389 (4.4) 544 (4.1) 606 (3.7) 640 (3.5) 279 Malta 273 (4.2) 310 (4.3) 382 (3.4) 548 (2.8) 618 (3.4) 656 (4.4) 308 OECD average 336 (0.7) 368 (0.6) 426 (0.6) 561 (0.5) 615 (0.5) 645 (0.6) 247 Note: Countries, economies, and provinces have been sorted in ascending order by the difference in score points between the 10th and 90th percentiles. BSJG-China represents Beijing, Shanghai, Jiangsu, and Guangdong. The coverage of Argentina, Kazakhstan, and Malaysia is too small to ensure comparability. See OECD, PISA 2015 Results for a note regarding Cyprus. 59 Table B.1.7 Estimated average scores by language of the school system for Canada and the provinces: SCIENCE Anglophone school system Canada and provinces Francophone school system Difference between systems Average Standard error Average Standard error Difference Standard error Canada 526 (2.2) 533 (4.7) -7 (5.0) Nova Scotia 518 (4.6) 477 (7.3) 42* (8.7) New Brunswick 508 (5.7) 502 (4.9) 6 (7.1) Quebec 514 (3.5) 540 (5.3) -26* (6.2) Ontario 526 (4.1) 486 (4.2) 39* (5.4) Manitoba 501 (5.0) 473 (6.9) 28* (8.3) Alberta 541 (4.1) 504 (8.9) 37* (10.6) British Columbia 539 (4.3) 532 (15.8) 7 (15.9) * Statistically significant differences. 60 Table B.1.8 Estimated average scores by language of the school system for Canada and the provinces: SCIENCE BY COMPETENCY SUBSCALES Anglophone school system Evaluate and design scientific enquiry Difference between systems Average Standard error Average Standard error Canada 529 (2.2) 533 (5.0) -5 (5.2) Nova Scotia 520 (5.0) 480 (8.0) 40* (9.2) New Brunswick 511 (5.6) 504 (6.8) 7 (8.7) Quebec 512 (5.6) 540 (5.6) -28* (7.2) Ontario 527 (3.9) 489 (6.4) 38* (7.2) Manitoba 506 (5.2) 478 (7.0) 28* (8.3) Alberta 547 (4.7) 507 (10.2) 40* (11.7) British Columbia 542 (4.5) 540 (16.1) 1 (16.3) Canada 528 (3.1) 538 (5.5) -10 (6.3) Nova Scotia 517 (6.4) 479 (9.9) 37* (12.8) New Brunswick 507 (6.7) 510 (7.4) -3 (9.1) Quebec 521 (6.9) 545 (6.1) -24* (9.1) Ontario 529 (5.3) 490 (8.7) 39* (10.6) Manitoba 499 (5.8) 474 (9.9) 25* (11.6) Alberta 540 (5.0) 502 (11.8) 38* (13.3) British Columbia 537 (5.8) 530 (19.3) 8 (19.4) Canada 523 (2.9) 531 (5.1) -7 (5.5) Nova Scotia 515 (5.8) 473 (8.5) 43* (11.3) New Brunswick 506 (6.6) 493 (5.9) 13 (8.4) Quebec 516 (6.0) 538 (5.7) -22* (7.3) Ontario 523 (4.9) 481 (6.1) 42* (6.8) Manitoba 499 (5.1) 468 (8.9) 32* (11.2) Alberta 537 (4.7) 499 (11.6) 38* (13.4) British Columbia 536 (5.8) 521 (15.2) 15 (15.4) Canada and provinces Explain phenomena scientifically Francophone school system Difference Standard error Interpret data and evidence scientifically * Statistically significant differences. 61 Table B.1.9 Estimated average scores by language of the school system for Canada and the provinces: SCIENCE BY KNOWLEDGE SUBSCALES Anglophone school system Procedural and epistemic * Statistically significant differences. 62 Difference between systems Average Standard error Average Standard error Canada 527 (2.3) 534 (5.1) -7 (5.5) Nova Scotia 518 (4.7) 483 (8.1) 36* (9.8) New Brunswick 509 (6.3) 503 (6.7) 6 (8.6) Quebec 510 (4.9) 540 (5.7) -30* (7.5) Ontario 525 (4.1) 491 (5.0) 34* (6.5) Manitoba 504 (4.9) 481 (6.6) 23* (8.2) Alberta 546 (4.3) 508 (11.3) 38* (12.3) British Columbia 540 (4.4) 544 (17.9) -3 (17.8) Canada 526 (2.6) 533 (5.1) -7 (5.5) Nova Scotia 516 (5.1) 477 (8.0) 39* (9.4) New Brunswick 507 (6.4) 499 (5.8) 7 (8.5) Quebec 517 (5.1) 540 (5.7) -23* (7.3) Ontario 527 (4.5) 484 (5.0) 43* (6.3) Manitoba 500 (5.0) 469 (6.8) 31* (8.5) Alberta 538 (4.5) 499 (10.7) 39* (12.5) British Columbia 537 (4.7) 526 (16.3) 12 (16.3) Canada and provinces Content Francophone school system Difference Standard error Table B.1.10 Estimated average scores by language of the school system for Canada and the provinces: SCIENCE BY CONTENT SUBSCALES Anglophone school system Living systems Earth and space systems Difference between systems Average Standard error Average Standard error Canada 526 (2.6) 533 (5.1) -7 (5.6) Nova Scotia 518 (5.4) 479 (8.7) 39* (10.2) New Brunswick 505 (6.7) 503 (5.2) 2 (8.8) Quebec 516 (5.8) 539 (5.8) -24* (8.0) Ontario 525 (4.4) 489 (6.2) 36* (7.2) Manitoba 504 (5.3) 479 (7.1) 25* (9.0) Alberta 543 (5.0) 506 (10.1) 38* (12.1) British Columbia 534 (5.2) 536 (17.1) -2 (17.0) Canada 527 (2.6) 531 (5.0) -4 (5.4) Nova Scotia 520 (5.0) 471 (7.8) 49* (9.3) New Brunswick 511 (6.5) 496 (7.0) 14 (9.2) Quebec 514 (5.4) 538 (5.7) -24* (8.3) Ontario 526 (4.6) 482 (5.3) 44* (6.6) Manitoba 499 (5.1) 464 (8.3) 35* (9.5) Alberta 539 (4.7) 496 (9.1) 43* (11.3) British Columbia 543 (4.9) 529 (17.2) 14 (17.6) Canada 527 (2.7) 539 (5.2) -12* (5.8) Nova Scotia 516 (5.3) 483 (7.7) 33* (9.5) New Brunswick 508 (6.6) 506 (7.3) 2 (8.9) Quebec 515 (7.6) 546 (5.9) -30* (9.4) Ontario 526 (4.5) 490 (5.0) 36* (6.4) Manitoba 502 (5.3) 480 (9.3) 22* (11.0) Alberta 543 (5.2) 506 (10.6) 36* (12.6) British Columbia 538 (6.1) 537 (17.8) 1 (18.2) Canada and provinces Physical systems Francophone school system Difference * Statistically significant differences. 63 Standard error Table B.1.11 Estimated average scores by gender for Canada and the provinces: SCIENCE Females Canada and provinces Males Difference (female-male) Average Standard error Average Standard error Difference Standard error Canada 527 (2.3) 528 (2.5) -1 (2.4) Newfoundland and Labrador 502 (3.8) 510 (5.2) -9 (6.5) Prince Edward Island 521 (6.0) 509 (7.5) 12 (8.5) Nova Scotia 518 (4.8) 516 (5.9) 2 (6.0) New Brunswick 507 (4.5) 506 (6.2) 0 (6.3) Quebec 533 (5.2) 541 (5.6) -8 (5.1) Ontario 525 (4.2) 523 (4.5) 2 (3.7) Manitoba 500 (6.1) 499 (5.0) 0 (5.8) Saskatchewan 493 (4.0) 498 (3.7) -5 (4.6) Alberta 539 (4.8) 542 (4.4) -3 (4.3) British Columbia 538 (4.2) 540 (6.0) -2 (5.7) Table B.1.12 Proportion of males and females who performed below Level 2 and at Levels 5 and 6, PISA 2015, Canada and the provinces: SCIENCE Below Level 2 Female Canada and provinces Levels 5 and 6 Difference (F-M) Male % Standard error % Standard error Canada 10.1 (0.6) 12.0 (0.7) Newfoundland and Labrador 14.9 (1.8) 16.0 8.1 (2.3) Nova Scotia 11.7 New Brunswick Female Difference (F-M) Male Standard error % Standard error % Standard error -1.9* (0.7) 11.4 (0.7) 13.4 (0.8) -2.0* (1.0) (1.9) -1.0 (2.5) 5.4 (1.0) 10.3 (1.6) -4.9* (1.8) 14.3 (3.0) -6.2 (3.4) 7.5 (2.4) 9.9 (2.8) -2.4 (3.4) (2.0) 14.0 (1.8) -2.3 (2.2) 9.7 (1.6) 9.8 (1.8) -0.1 (2.3) 14.1 (1.8) 17.0 (2.7) -2.9 (2.6) 7.1 (1.5) 9.0 (1.6) -1.9 (2.1) Quebec 7.8 (1.2) 9.2 (1.3) -1.4 (1.2) 10.4 (1.6) 15.4 (2.1) -4.9* (2.3) Ontario 11.1 (1.2) 13.4 (1.2) -2.2 (1.3) 11.6 (1.5) 12.5 (1.3) -0.8 (1.6) Manitoba 17.7 (2.4) 17.2 (2.0) 0.5 (2.8) 7.3 (1.6) 6.8 (1.1) 0.5 (1.7) Saskatchewan 16.5 (1.9) 16.9 (1.7) -0.5 (2.4) 5.0 (0.9) 7.2 (1.1) -2.1 (1.4) Alberta 7.7 (1.3) 9.4 (1.2) -1.7 (1.6) 14.9 (1.7) 16.9 (1.7) -2.0 (2.0) British Columbia 7.8 (1.3) 9.6 (1.5) -1.8 (1.5) 13.5 (1.6) 15.8 (1.9) -2.3 (1.9) Prince Edward Island * Statistically significant differences. 64 difference difference Standard error Table B.1.13 Estimated average scores by gender for Canada and the provinces: SCIENCE BY COMPETENCY SUBSCALES Females Evaluate and design scientific enquiry Difference (female-male) Average Standard error Average Standard error Difference Standard error Canada 525 (2.4) 535 (2.6) -10* (2.6) Newfoundland and Labrador 500 (4.2) 518 (5.4) -18* (6.8) Prince Edward Island 518 (5.9) 515 (7.9) 3 (8.4) Nova Scotia 515 (5.2) 522 (6.5) -7 (6.6) New Brunswick 504 (4.7) 513 (6.3) -9 (6.4) Quebec 529 (5.3) 545 (6.1) -16* (4.8) Ontario 522 (4.4) 529 (4.3) -6 (4.1) Manitoba 499 (6.1) 509 (5.4) -9 (5.8) Saskatchewan 494 (4.3) 507 (4.1) -13* (5.1) Alberta 542 (5.3) 552 (5.0) -10* (4.5) British Columbia 536 (5.0) 548 (5.7) -12* (6.1) Canada 535 (3.1) 525 (3.0) 10* (2.7) Newfoundland and Labrador 508 (5.0) 505 (6.4) 3 (8.2) Prince Edward Island 527 (7.3) 504 (9.7) 24* (10.2) Nova Scotia 522 (6.9) 509 (7.6) 13 (7.7) New Brunswick 514 (5.5) 502 (7.5) 12 (7.2) Quebec 542 (6.3) 542 (5.8) 0 (5.3) Ontario 535 (5.4) 520 (5.4) 15* (4.1) Manitoba 503 (7.0) 493 (5.9) 10 (6.6) Saskatchewan 499 (5.2) 492 (4.3) 7 (5.4) Alberta 544 (5.6) 536 (5.6) 9 (5.3) British Columbia 541 (5.9) 533 (7.4) 8 (6.6) Canada 525 (2.8) 525 (3.2) 1 (2.8) Newfoundland and Labrador 498 (4.5) 504 (5.4) -6 (7.0) Prince Edward Island 519 (6.6) 504 (8.4) 15 (9.0) Nova Scotia 516 (6.2) 512 (6.6) 4 (6.5) New Brunswick 504 (4.9) 502 (7.2) 1 (6.6) Quebec 533 (5.8) 538 (6.1) -5 (5.4) Ontario 523 (4.9) 519 (5.4) 3 (4.1) Manitoba 499 (6.3) 496 (5.0) 3 (6.4) Saskatchewan 491 (4.6) 492 (4.3) -1 (5.8) Alberta 536 (5.6) 537 (4.9) -1 (4.8) British Columbia 535 (5.4) 537 (7.7) -3 (6.4) Canada and provinces Explain phenomena scientifically Males Interpret data and evidence scientifically * Statistically significant differences. 65 Table B.1.14 Estimated average scores by gender for Canada and the provinces: SCIENCE BY KNOWLEDGE SUBSCALES Females Procedural and epistemic * Statistically significant differences. 66 Difference (female-male) Average Standard error Average Standard error Difference Standard error Canada 524 (2.3) 533 (2.6) -9* (2.5) Newfoundland and Labrador 499 (3.8) 516 (5.2) -17* (6.1) Prince Edward Island 518 (7.2) 516 (8.9) 2 (9.2) Nova Scotia 514 (4.9) 520 (6.2) -6 (6.7) New Brunswick 504 (4.9) 512 (6.9) -8 (6.2) Quebec 528 (5.4) 546 (5.9) -18* (4.9) Ontario 521 (4.3) 526 (4.5) -6 (3.9) Manitoba 499 (6.0) 506 (5.0) -7 (5.9) Saskatchewan 493 (4.2) 504 (3.9) -12* (4.5) Alberta 541 (4.9) 550 (4.9) -9 (4.6) British Columbia 535 (4.4) 546 (6.0) -10 (5.9) Canada 530 (2.6) 525 (2.8) 6* (2.4) Newfoundland and Labrador 504 (4.6) 505 (5.3) -2 (7.2) Prince Edward Island 523 (6.6) 506 (8.0) 17 (9.4) Nova Scotia 519 (5.6) 510 (6.1) 9 (6.4) New Brunswick 508 (4.7) 501 (7.0) 7 (6.5) Quebec 537 (5.5) 538 (5.9) -1 (5.0) Ontario 530 (4.6) 521 (5.1) 9* (4.1) Manitoba 502 (6.1) 494 (5.0) 7 (5.9) Saskatchewan 495 (4.3) 491 (4.0) 3 (4.9) Alberta 539 (5.3) 536 (4.7) 3 (4.5) British Columbia 539 (4.4) 535 (6.6) 4 (6.0) Canada and provinces Content Males Table B.1.15 Estimated average scores by gender for Canada and the provinces: SCIENCE BY CONTENT SUBSCALES Females Living systems Difference (female-male) Average Standard error Average Standard error Canada 525 (2.4) 530 (3.3) -5 (3.3) Newfoundland and Labrador 501 (4.7) 512 (6.9) -12 (7.2) Prince Edward Island 522 (7.2) 513 (8.6) 8 (9.8) Nova Scotia 516 (5.5) 517 (6.8) -1 (6.6) New Brunswick 504 (4.9) 506 (7.0) -2 (6.8) Quebec 530 (5.6) 543 (6.1) -13* (5.3) Ontario 523 (4.6) 524 (5.2) -2 (5.0) Manitoba 500 (6.5) 504 (5.4) -5 (6.7) Saskatchewan 494 (4.9) 501 (4.8) -8 (4.8) Alberta 541 (5.9) 545 (5.7) -4 (6.1) British Columbia 533 (4.9) 537 (7.1) -4 (6.3) Canada 528 (2.6) 527 (2.9) 1 (2.6) Newfoundland and Labrador 503 (4.5) 508 (5.5) -5 (7.1) Prince Edward Island 522 (6.4) 509 (8.5) 13 (9.1) Nova Scotia 521 (5.6) 516 (5.8) 5 (6.2) New Brunswick 508 (5.3) 506 (6.8) 2 (6.5) Quebec 533 (5.3) 537 (5.9) -4 (5.2) Ontario 527 (4.6) 522 (5.1) 4 (4.0) Manitoba 499 (6.4) 495 (4.9) 3 (5.9) Saskatchewan 492 (4.5) 494 (4.5) -2 (5.8) Alberta 538 (6.0) 539 (4.6) -1 (5.0) British Columbia 542 (4.9) 543 (6.4) -1 (5.9) Canada 528 (2.7) 530 (3.0) -2 (2.9) Newfoundland and Labrador 498 (4.8) 508 (5.6) -9 (6.8) Prince Edward Island 521 (6.2) 512 (8.1) 9 (9.0) Nova Scotia 514 (5.6) 515 (6.4) -1 (6.3) New Brunswick 507 (5.6) 508 (7.3) -1 (6.9) Quebec 538 (5.6) 546 (6.7) -8 (6.0) Ontario 525 (4.5) 524 (5.2) 1 (4.4) Manitoba 501 (6.2) 500 (5.7) 1 (6.4) Saskatchewan 496 (5.5) 500 (4.4) -5 (5.6) Alberta 540 (6.3) 545 (5.2) -5 (5.0) British Columbia 537 (6.1) 540 (7.4) -3 (5.9) Canada and provinces Physical systems Males Difference Standard error Earth and space systems * Statistically significant differences. 67 Table B.1.16 Comparisons of performance, PISA 2006, 2009, 2012, and 2015, Canada and the provinces: SCIENCE 2009 2006 Canada and provinces Standard error Average 2012 Average Standard error Average 2015 Standard error Standard error Average Canada 534 (2.0) 529 (3.0) 525* (4.0) 528 (4.9) Newfoundland and Labrador 526 (2.5) 518 (4.0) 514* (5.0) 506* (5.5) Prince Edward Island 509 (2.7) 495* (3.5) 490* (4.4) 515 (7.0) Nova Scotia 520 (2.5) 523 (3.7) 516 (4.6) 517 (6.3) New Brunswick 506 (2.3) 501 (3.5) 507 (4.4) 506 (6.3) Quebec 531 (4.2) 524 (4.1) 516* (4.8) 537 (6.5) Ontario 537 (4.2) 531 (4.2) 527 (5.6) 524 (6.0) Manitoba 523 (3.2) 506* (4.7) 503* (4.8) 499* (6.5) Saskatchewan 517 (3.6) 513 (4.5) 516 (4.6) 496* (5.5) Alberta 550 (3.8) 545 (5.0) 539 (5.8) 541 (6.0) British Columbia 539 (4.7) 535 (4.8) 544 (5.3) 539 (6.2) * Statistically significant differences compared to PISA 2006. Note: The linkage error is incorporated into the standard error for 2009, 2012, and 2015. Also, for some provinces, the standard errors from 2006 to 2009 and to 2012 differ from those in the previous PISA reports on trend results. These differences result from the change of the method used by the OECD to compute the linkage error. Table B.1.17 Proportion of students who performed below Level 2 and at Levels 5 and 6, in PISA 2006 and 2015, Canada and the provinces: SCIENCE Below Level 2 2006 Canada and provinces Levels 5 and 6 Difference 2006-2015 2015 2006 Difference 2006-2015 2015 % Standard error % Standard error difference Standard error % Standard error % Standard error Canada 10.0 (0.6) 11.1 (0.5) 1.1 (1.0) 14.4 (0.5) 12.4 (0.6) -2.1* (1.8) Newfoundland and Labrador 11.9 (0.9) 15.5 (1.3) 3.5* (1.7) 13.5 (1.1) 7.8 (1.0) -5.8* (2.2) Prince Edward Island 16.0 (1.2) 11.3 (2.1) -4.7 (2.5) 9.8 (1.0) 8.7 (2.0) -1.0 (2.7) Nova Scotia 11.8 (1.2) 12.8 (1.5) 1.0 (2.0) 10.2 (0.9) 9.8 (1.2) -0.4 (2.2) New Brunswick 15.3 (1.0) 15.6 (1.9) 0.3 (2.3) 7.9 (0.8) 8.1 (1.1) 0.1 (2.1) Quebec 11.3 (1.2) 8.5 (1.1) -2.8 (1.8) 14.3 (1.1) 12.8 (1.5) -1.5 (2.4) Ontario 9.5 (1.2) 12.3 (1.0) 2.8 (1.7) 14.2 (1.1) 12.1 (1.1) -2.1 (2.2) Manitoba 12.5 (1.2) 17.4 (1.7) 5.0* (2.2) 12.4 (1.1) 7.1 (1.1) -5.3* (2.2) Saskatchewan 13.6 (1.3) 16.7 (1.4) 3.1 (2.0) 10.8 (1.1) 6.2 (0.7) -4.6* (2.1) Alberta 6.2 (0.9) 8.6 (1.0) 2.4 (1.4) 18.3 (1.2) 15.9 (1.4) -2.5 (2.4) British Columbia 9.1 (1.2) 8.7 (1.2) -0.4 (1.8) 15.9 (1.5) 14.7 (1.5) -1.2 (2.6) * Statistically significant differences. 68 difference Standard error Table B.1.18 Gender differences in student performance, PISA 2006 and 2015, Canada and the provinces: SCIENCE 2006 Canada and provinces Gender difference (F-M) 2015 Standard error Gender difference (F-M) Standard error Canada -4 (2.2) -1 (2.4) Newfoundland and Labrador 12* (4.9) -9 (6.5) Prince Edward Island 3 (5.0) 12 (8.5) Nova Scotia -1 (4.9) 2 (6.0) New Brunswick -1 (4.1) 0 (6.3) Quebec -8 (4.2) -8 (5.1) Ontario -4 (4.1) 2 (3.7) Manitoba -4 (5.3) 0 (5.8) Saskatchewan 5 (5.7) -5 (4.6) Alberta -4 (4.7) -3 (4.3) British Columbia -5 (5.4) -2 (5.7) * Statistically significant differences. 69 Table B.2.1 Estimated average scores and confidence intervals for countries, economies, and provinces: READING Country, economy, or province Average Confidence Confidence Standard interval – 95% interval – 95% error lower limit upper limit British Columbia 536 (5.6) 525 547 Singapore Alberta Quebec Ontario Hong Kong-China Canada 535 533 532 527 527 527 (1.6) (5.2) (4.7) (4.4) (2.7) (2.3) 532 523 523 519 521 522 538 544 541 536 532 531 Finland Ireland Estonia Korea Nova Scotia 526 521 519 517 517 (2.5) (2.5) (2.2) (3.5) (4.9) 521 516 515 511 508 531 526 523 524 527 Japan Prince Edward Island 516 515 (3.2) (6.1) 510 503 522 527 Norway New Zealand Germany Macao-China Poland New Brunswick 513 509 509 509 506 505 (2.5) (2.4) (3.0) (1.3) (2.5) (5.2) 508 505 503 506 501 495 518 514 515 511 511 516 Slovenia Newfoundland and Labrador The Netherlands Australia 505 (1.5) 502 508 505 (3.5) 498 512 503 503 (2.4) (1.7) 498 500 508 506 Sweden 500 (3.5) 493 507 Denmark France Belgium Manitoba Portugal United Kingdom Chinese Taipei United States Saskatchewan Spain Russian Federation BSJG-China Switzerland Latvia Czech Republic Croatia Vietnam Austria Italy Iceland Luxembourg Israel 500 499 499 498 498 498 497 497 496 496 495 494 492 488 487 487 487 485 485 482 481 479 (2.5) (2.5) (2.4) (5.0) (2.7) (2.8) (2.5) (3.4) (3.6) (2.4) (3.1) (5.1) (3.0) (1.8) (2.6) (2.7) (3.7) (2.8) (2.7) (2.0) (1.4) (3.8) 495 494 494 489 493 493 492 490 489 491 489 484 486 484 482 482 479 479 480 478 479 472 505 504 503 508 503 503 502 504 503 500 501 504 498 491 492 492 494 490 490 485 484 486 70 Country, economy, or province Lithuania Hungary Greece Chile Slovak Republic Malta Cyprus Uruguay Romania United Arab Emirates Bulgaria Malaysia Turkey Costa Rica Trinidad and Tobago Kazakhstan Montenegro Argentina Colombia Mexico Moldova Thailand Jordan Brazil Albania Qatar Georgia Peru Indonesia Tunisia Dominican Republic Republic of Macedonia Algeria Kosovo Lebanon Average 472 470 467 459 453 447 443 437 434 434 432 431 428 427 427 427 427 425 425 423 416 409 408 407 405 402 401 398 397 361 358 352 350 347 347 Confidence Confidence Standard interval – 95% interval – 95% error lower limit upper limit (2.7) (2.7) (4.3) (2.6) (2.8) (1.8) (1.7) (2.5) (4.1) (2.9) (5.0) (3.5) (4.0) (2.6) (1.5) (3.4) (1.6) (3.2) (2.9) (2.6) (2.5) (3.3) (2.9) (2.8) (4.1) (1.0) (3.0) (2.9) (2.9) (3.1) (3.1) (1.4) (3.0) (1.6) (4.4) 467 464 459 454 447 443 440 432 426 428 422 424 421 422 424 420 424 419 419 418 411 403 402 402 397 400 395 392 392 355 352 349 344 344 338 478 475 476 464 458 450 446 442 442 439 442 437 436 433 430 434 430 432 431 428 421 416 414 413 413 404 407 403 403 367 364 355 356 350 355 Note: The OECD average was 493, with a standard error of 0.5. Countries, economies, and provinces have been sorted in descending order by average score. BSJG-China represents Beijing, Shanghai, Jiangsu, and Guangdong. The coverage of Argentina, Kazakhstan, and Malaysia is too small to ensure comparability. See OECD, PISA 2015 Results for a note regarding Cyprus. Table B.2.2 Estimated average scores and confidence intervals for provinces, countries and economies: MATHEMATICS Country, economy, or province Average Confidence Confidence Standard interval – 95% interval – 95% error lower limit upper limit Singapore Hong Kong-China Quebec Macao-China Chinese Taipei Japan BSJG-China 564 548 544 544 542 532 531 (1.5) (3.0) (4.8) (1.1) (3.0) (3.0) (4.9) 561 542 535 542 536 527 522 567 554 553 546 548 538 541 Korea British Columbia Switzerland Estonia 524 522 521 520 (3.7) (5.0) (2.9) (2.0) 517 512 516 516 531 531 527 524 Canada The Netherlands Alberta Denmark Finland Slovenia Ontario Belgium Germany Poland Ireland Norway Prince Edward Island Nova Scotia Austria New Zealand Vietnam Russian Federation Sweden Australia France New Brunswick United Kingdom Czech Republic Portugal Italy Manitoba Iceland Spain Luxembourg Newfoundland and Labrador Saskatchewan Latvia Malta Lithuania Hungary 516 512 511 511 511 510 509 507 506 504 504 502 499 497 497 495 495 494 494 494 493 493 492 492 492 490 489 488 486 486 (2.3) (2.2) (4.7) (2.2) (2.3) (1.3) (4.2) (2.4) (2.9) (2.4) (2.1) (2.2) (6.4) (4.6) (2.9) (2.3) (4.5) (3.1) (3.2) (1.6) (2.1) (5.1) (2.5) (2.4) (2.5) (2.8) (4.2) (2.0) (2.2) (1.3) 511 508 502 507 507 507 501 502 500 500 500 497 486 488 491 491 486 488 488 491 489 483 488 488 487 484 481 484 482 483 520 517 521 515 516 512 518 512 512 509 508 506 511 506 502 500 503 500 500 497 497 502 497 497 497 495 497 492 490 488 486 (3.2) 479 492 484 482 479 478 477 (2.9) (1.9) (1.7) (2.3) (2.5) 479 479 475 474 472 490 486 482 483 482 Country, economy, or province Slovak Republic Israel United States Croatia Kazakhstan Greece Malaysia Romania Bulgaria Cyprus United Arab Emirates Chile Turkey Moldova Uruguay Montenegro Trinidad and Tobago Thailand Albania Argentina Mexico Georgia Qatar Costa Rica Lebanon Colombia Peru Indonesia Jordan Brazil Republic of Macedonia Tunisia Kosovo Algeria Dominican Republic Average 475 470 470 464 460 454 446 444 441 437 427 423 420 420 418 418 417 415 413 409 408 404 402 400 396 390 387 386 380 377 371 367 362 360 328 Confidence Confidence Standard interval – 95% interval – 95% error lower limit upper limit (2.7) (3.6) (3.2) (2.8) (4.3) (3.8) (3.3) (3.8) (4.0) (1.7) (2.4) (2.5) (4.1) (2.5) (2.5) (1.5) (1.4) (3.0) (3.4) (3.1) (2.2) (2.8) (1.3) (2.5) (3.7) (2.3) (2.7) (3.1) (2.7) (2.9) (1.3) (3.0) (1.6) (3.0) (2.7) 470 463 463 459 451 446 440 437 433 434 423 418 412 415 413 415 414 410 406 403 404 398 400 395 389 385 381 380 375 371 369 361 358 354 322 480 477 476 469 468 461 452 451 449 441 432 428 429 424 423 421 420 421 420 415 412 409 405 405 403 394 392 392 385 383 374 373 365 365 333 Note: The OECD average was 490, with a standard error of 0.4. Countries, economies and provinces have been sorted in descending order by average score. BSJG-China represents Beijing, Shanghai, Jiangsu, and Guangdong. The coverage of Argentina, Kazakhstan, and Malaysia is too small to ensure comparability. See OECD, PISA 2015 Results for a note regarding Cyprus. 71 Table B.2.3 Variation in student performance for countries, economies, and provinces: READING Score Standard error Score Standard error Score Standard error Score Standard error Score Standard error Score Standard error Difference in score points between the 10th and 90th percentiles Algeria 232 (4.1) 258 (4.1) 301 (2.6) 397 (3.8) 443 (4.8) 472 (5.4) 185 Vietnam 367 (5.2) 393 (4.9) 438 (4.3) 537 (4.2) 580 (5.3) 605 (6.2) 187 Indonesia 272 (5.9) 300 (5.1) 346 (3.7) 448 (3.0) 495 (3.3) 522 (4.0) 195 Mexico 292 (3.8) 321 (3.6) 370 (3.0) 478 (3.2) 523 (3.9) 549 (4.2) 202 Kosovo 215 (4.3) 243 (2.8) 294 (2.5) 403 (2.3) 447 (2.6) 471 (3.0) 204 Costa Rica 298 (4.0) 326 (3.5) 374 (3.0) 480 (3.2) 530 (3.8) 560 (4.8) 204 Thailand 281 (4.0) 308 (3.3) 354 (3.7) 463 (4.2) 514 (4.9) 543 (5.9) 206 Kazakhstan 299 (4.4) 325 (4.1) 372 (3.4) 481 (4.7) 533 (5.3) 563 (6.6) 207 Malaysia 290 (5.7) 322 (5.0) 377 (4.1) 488 (3.7) 531 (3.9) 556 (5.3) 209 Tunisia 228 (6.0) 257 (4.7) 305 (3.6) 416 (3.2) 467 (3.6) 496 (5.1) 209 Macao-China 365 (3.7) 399 (2.6) 456 (2.0) 566 (2.0) 610 (2.8) 635 (3.4) 212 Turkey Prince Edward Island 291 367 (4.8) (20.6) 322 404 (4.9) (10.4) 372 461 (4.4) (8.7) 487 575 (5.2) (8.4) 535 622 (5.9) (9.8) 561 648 (6.1) (14.1) 213 218 Hong Kong-China 372 (5.6) 412 (4.5) 473 (3.7) 587 (2.5) 632 (3.1) 656 (3.5) 220 Dominican Republic 226 (4.5) 250 (3.8) 297 (3.5) 416 (4.1) 471 (5.1) 503 (5.8) 220 Latvia Saskatchewan 341 355 (3.8) (7.3) 374 384 (3.4) (5.9) 431 437 (3.0) (5.4) 548 556 (2.0) (4.7) 595 605 (2.5) (4.5) 621 633 (3.6) (5.8) 221 222 Ireland 373 (4.6) 406 (4.1) 463 (3.1) 582 (2.7) 629 (2.8) 657 (4.1) 222 Spain 343 (4.5) 379 (3.9) 438 (3.3) 558 (2.7) 603 (2.9) 629 (3.5) 224 Denmark 347 (4.1) 383 (4.3) 443 (3.2) 561 (2.6) 608 (3.4) 635 (3.6) 225 Estonia 369 (4.2) 404 (4.0) 460 (2.8) 581 (2.6) 630 (2.9) 659 (3.2) 226 Russian Federation Nova Scotia 350 366 (4.4) (9.4) 381 401 (3.9) (8.4) 434 458 (3.9) (5.9) 556 579 (3.5) (5.3) 608 628 (3.5) (6.5) 637 657 (3.7) (8.6) 227 228 British Columbia Newfoundland and Labrador 381 (8.7) 419 (7.6) 477 (6.1) 597 (7.0) 648 (7.1) 678 (6.9) 229 353 (9.7) 387 (7.5) 448 (4.9) 567 (5.4) 616 (6.8) 641 (7.3) 229 Chile 310 (4.9) 342 (3.7) 398 (3.3) 521 (3.2) 572 (3.5) 599 (3.7) 229 Argentina 277 (5.5) 309 (4.3) 364 (4.2) 487 (3.6) 538 (3.9) 569 (4.7) 230 Poland 349 (5.1) 386 (3.7) 446 (3.5) 570 (2.8) 617 (3.5) 644 (4.6) 231 Peru Quebec 253 368 (3.3) (9.0) 281 410 (3.2) (7.7) 333 474 (3.2) (5.9) 462 596 (3.9) (5.1) 514 644 (4.5) (5.7) 543 672 (5.1) (7.0) 233 234 Colombia 278 (4.9) 308 (4.4) 361 (4.0) 489 (3.3) 542 (3.1) 572 (3.0) 235 Alberta 377 (8.6) 412 (7.5) 474 (6.3) 597 (6.1) 647 (5.8) 675 (7.5) 235 Manitoba 345 (8.8) 378 (7.5) 436 (6.8) 563 (5.3) 613 (6.4) 642 (7.4) 235 New Brunswick 350 (11.0) 383 (12.0) 444 (7.8) 570 (5.4) 619 (6.3) 645 (7.0) 236 Croatia 334 (4.6) 367 (4.2) 424 (3.8) 553 (3.1) 603 (3.3) 632 (3.6) 237 Japan Canada 352 366 (7.0) (4.3) 391 404 (5.8) (3.6) 457 466 (4.2) (2.8) 581 591 (3.4) (2.4) 629 642 (3.7) (2.7) 656 671 (3.8) (2.8) 238 238 Slovenia 346 (4.1) 382 (2.7) 444 (2.3) 570 (2.1) 621 (3.4) 648 (3.9) 239 Finland 359 (5.4) 401 (4.7) 469 (3.7) 592 (2.7) 640 (2.6) 668 (3.8) 239 Portugal 339 (4.7) 374 (3.7) 436 (4.2) 564 (2.8) 614 (3.1) 641 (3.3) 240 Chinese Taipei 331 (4.5) 371 (4.2) 437 (3.4) 563 (3.0) 611 (3.8) 638 (4.8) 240 Jordan 241 (6.3) 281 (5.4) 348 (3.7) 475 (3.1) 522 (2.9) 549 (3.1) 242 Percentiles 5th Country, economy, or province 72 10th 25th 75th 90th 95th Table B.2.3 (cont’d) Variation in student performance for countries, economies, and provinces: READING Percentiles 5th Country, economy, or province 10th 25th 75th 90th 95th Difference in score points between the 10th and 90th percentiles Score 364 Standard error (7.1) Score 401 Standard error (6.2) Score 465 Standard error (5.5) Score 593 Standard error (4.7) Score 645 Standard error (4.5) Score 675 Standard error (5.8) Italy 323 (4.8) 359 (4.2) 421 (3.7) 552 (3.1) 602 (2.9) 631 (3.5) 244 Montenegro 271 (3.5) 304 (2.5) 361 (2.5) 493 (2.4) 549 (2.8) 581 (3.0) 245 Romania 276 (6.3) 310 (5.4) 370 (5.0) 499 (4.7) 555 (5.4) 588 (6.1) 245 Lithuania 312 (4.6) 347 (3.5) 407 (3.0) 541 (3.6) 593 (4.4) 622 (3.7) 246 United Kingdom 336 (4.4) 372 (4.0) 432 (3.2) 565 (3.0) 621 (3.6) 653 (4.1) 249 Albania 244 (5.1) 279 (5.2) 340 (4.7) 472 (4.7) 528 (5.2) 561 (5.6) 250 Korea 345 (7.3) 386 (5.6) 455 (4.4) 586 (3.9) 637 (4.3) 666 (4.1) 251 Moldova 253 (4.2) 289 (3.7) 349 (3.1) 485 (3.3) 541 (4.1) 574 (5.0) 252 Uruguay 280 (3.7) 311 (3.1) 368 (3.3) 504 (3.1) 563 (4.6) 597 (5.5) 252 Switzerland 322 (5.6) 360 (5.0) 426 (4.0) 563 (3.6) 614 (3.6) 643 (3.7) 254 Hungary 306 (5.3) 338 (4.2) 399 (3.9) 541 (3.1) 593 (3.2) 620 (3.4) 255 Norway 342 (5.2) 381 (4.0) 449 (3.3) 583 (2.9) 636 (3.0) 666 (3.7) 255 Greece 296 (7.6) 334 (8.2) 400 (6.1) 539 (3.6) 590 (3.7) 618 (3.8) 256 Iceland 310 (4.9) 350 (4.3) 417 (3.2) 552 (2.6) 607 (4.0) 638 (5.0) 256 Singapore 362 (4.4) 400 (3.7) 470 (2.6) 607 (2.0) 657 (2.6) 686 (3.3) 257 Republic of Macedonia 187 (3.7) 222 (3.3) 284 (2.4) 421 (2.2) 480 (3.3) 513 (4.3) 258 Germany 334 (5.2) 375 (5.3) 442 (3.8) 581 (3.1) 634 (3.4) 664 (3.2) 258 United States 326 (6.0) 364 (5.4) 430 (4.7) 568 (3.9) 624 (3.8) 655 (3.7) 259 Brazil 247 (3.4) 279 (2.8) 336 (3.0) 477 (3.2) 539 (3.9) 576 (4.6) 260 Sweden 321 (6.0) 364 (4.6) 433 (4.4) 573 (3.8) 625 (3.6) 655 (4.4) 262 The Netherlands 330 (5.3) 368 (4.6) 434 (4.0) 577 (2.8) 630 (3.1) 658 (3.5) 262 Czech Republic 315 (5.7) 352 (4.8) 418 (4.0) 559 (2.8) 614 (3.5) 645 (3.6) 262 Belgium 323 (3.8) 360 (3.9) 429 (3.8) 573 (2.2) 623 (2.5) 650 (2.9) 263 Austria 308 (5.1) 347 (5.1) 417 (4.0) 559 (3.1) 611 (3.0) 641 (3.5) 265 Australia 324 (3.0) 365 (2.7) 435 (2.4) 576 (2.0) 631 (2.2) 662 (2.6) 265 Georgia 226 (5.7) 266 (4.2) 332 (3.9) 474 (3.3) 533 (4.5) 568 (4.9) 268 Cyprus 268 (3.7) 305 (2.7) 372 (2.8) 516 (2.6) 573 (3.4) 606 (4.2) 269 Trinidad and Tobago 256 (4.4) 291 (3.2) 353 (2.8) 502 (2.3) 561 (3.5) 596 (4.6) 270 Slovak Republic 269 (6.5) 312 (4.6) 382 (4.1) 528 (3.1) 583 (3.2) 613 (4.1) 271 New Zealand 327 (4.8) 368 (4.5) 439 (3.6) 584 (3.3) 643 (4.3) 674 (4.4) 274 United Arab Emirates 258 (3.9) 295 (3.9) 359 (3.5) 509 (3.4) 572 (3.1) 605 (3.2) 277 Luxembourg 299 (3.3) 336 (2.9) 405 (2.1) 561 (2.1) 616 (2.5) 647 (3.8) 279 BSJG-China 304 (8.7) 346 (7.2) 420 (6.1) 573 (5.7) 630 (6.3) 661 (7.3) 283 Qatar 221 (2.2) 256 (1.8) 321 (1.8) 483 (2.2) 547 (2.2) 581 (2.7) 291 France 299 (6.6) 344 (5.7) 423 (3.7) 583 (3.1) 637 (3.0) 666 (3.6) 293 Israel 284 (7.1) 326 (5.8) 401 (5.1) 562 (4.3) 621 (4.3) 655 (5.1) 295 Bulgaria 241 (6.2) 277 (6.6) 347 (7.0) 517 (5.5) 578 (5.0) 611 (5.4) 300 Lebanon 167 (5.5) 203 (5.8) 265 (4.9) 426 (6.2) 503 (7.0) 546 (7.6) 301 Malta OECD average 236 326 (5.6) (0.9) 284 364 (4.9) (0.8) 366 428 (3.7) (0.6) 533 561 (2.7) (0.5) 595 613 (3.1) (0.6) 631 642 (3.8) (0.7) 311 249 Ontario 244 Note: Countries, economies, and provinces have been sorted in ascending order by the difference in score points between the 10th and 90th percentiles. BSJG-China represents Beijing, Shanghai, Jiangsu, and Guangdong. The coverage of Argentina, Kazakhstan, and Malaysia is too small to ensure comparability. See OECD, PISA 2015 Results for a note regarding Cyprus. 73 Table B.2.4 Variation in student performance for countries, economies, and provinces: MATHEMATICS Percentiles Difference in score points between the 10th and 90th percentiles 5th 10th 25th 75th 90th 95th Country, economy, or province Standard Score error Standard Score error Standard Score error Standard Score error Standard Score error Standard Score error Costa Rica 292 (2.7) 315 (2.9) 353 (2.5) 445 (3.0) 489 (4.2) 517 (5.0) 174 Dominican Republic 220 (4.3) 243 (3.9) 281 (3.2) 373 (3.6) 418 (4.7) 446 (7.0) 175 Algeria 247 (4.2) 271 (3.8) 312 (3.0) 405 (3.6) 452 (4.4) 481 (5.2) 181 Mexico 284 (4.1) 312 (2.6) 357 (2.5) 459 (2.9) 505 (3.5) 533 (3.6) 193 Kosovo Prince Edward Island 238 375 (3.5) (11.5) 265 401 (2.9) (10.7) 310 446 (2.3) (8.6) 413 550 (2.6) (9.4) 460 599 (4.2) (11.2) 487 624 (4.3) (12.7) 195 198 Colombia 269 (3.7) 293 (3.1) 335 (2.9) 441 (2.7) 492 (3.3) 522 (3.8) 199 Latvia 353 (4.4) 382 (3.0) 430 (2.7) 536 (2.1) 582 (2.9) 608 (3.1) 200 Indonesia 264 (4.1) 289 (4.1) 331 (3.5) 436 (3.9) 492 (5.4) 528 (6.2) 203 Macao-China 408 (4.4) 439 (2.4) 491 (1.7) 599 (1.9) 643 (2.5) 669 (4.0) 204 Ireland 371 (4.4) 400 (3.8) 450 (2.7) 559 (2.2) 606 (2.6) 633 (2.7) 206 Malaysia 315 (4.4) 343 (3.9) 391 (3.4) 501 (3.9) 549 (4.5) 577 (5.3) 207 Argentina 280 (4.3) 306 (3.4) 354 (3.5) 463 (3.7) 514 (4.1) 545 (4.7) 207 Thailand 286 (4.1) 313 (3.7) 360 (3.1) 468 (4.0) 521 (5.2) 555 (6.3) 208 Estonia 386 (3.7) 415 (3.1) 464 (2.6) 576 (2.6) 623 (2.7) 650 (3.4) 209 Denmark Saskatchewan Newfoundland and Labrador 376 350 (3.3) (7.9) 405 379 (3.2) (6.8) 457 428 (2.9) (4.4) 567 542 (2.5) (4.5) 614 589 (2.9) (5.2) 639 618 (3.5) (5.4) 209 210 348 (7.6) 379 (6.9) 432 (4.5) 542 (4.6) 589 (6.2) 617 (6.9) 210 Finland 372 (5.1) 404 (3.8) 456 (3.1) 568 (2.4) 614 (2.9) 642 (3.5) 210 Kazakhstan Nova Scotia 329 360 (5.8) (8.3) 357 390 (4.9) (7.2) 403 440 (4.7) (5.3) 513 554 (5.1) (5.1) 567 602 (6.3) (7.2) 600 632 (7.4) (6.6) 211 211 Peru 254 (3.5) 283 (2.6) 329 (2.7) 442 (4.0) 495 (4.3) 526 (4.5) 212 Turkey 291 (4.8) 317 (3.9) 363 (3.8) 477 (6.0) 529 (6.3) 559 (7.5) 212 Tunisia 235 (4.7) 263 (4.6) 310 (3.3) 421 (3.6) 476 (5.0) 510 (7.2) 213 Russian Federation Manitoba 357 354 (5.5) (8.6) 387 382 (4.6) (7.4) 437 433 (3.4) (5.2) 552 545 (3.4) (5.4) 601 597 (3.8) (6.7) 629 624 (4.2) (8.5) 214 214 Vietnam 361 (5.9) 388 (5.4) 436 (4.7) 551 (4.9) 604 (6.9) 636 (8.3) 215 Jordan British Columbia 238 380 (6.1) (8.5) 271 412 (4.0) (6.0) 324 465 (3.2) (6.2) 439 580 (3.2) (5.7) 489 631 (3.2) (6.5) 519 663 (3.9) (7.8) 219 219 Norway 359 (4.0) 391 (3.4) 444 (2.5) 561 (2.7) 610 (3.0) 638 (3.0) 219 Spain 342 (3.8) 374 (3.4) 428 (2.8) 546 (2.5) 593 (3.3) 621 (3.7) 220 Chile 284 (4.0) 313 (3.5) 363 (2.9) 483 (3.5) 534 (3.6) 563 (3.7) 221 Albania New Brunswick Alberta 272 351 365 (5.7) (10.7) (8.0) 303 380 398 (4.3) (9.3) (6.5) 354 432 453 (4.0) (6.4) (5.8) 472 553 571 (4.2) (5.4) (5.3) 525 602 621 (4.4) (6.8) (5.6) 556 628 650 (5.0) (7.3) (6.2) 221 223 223 Romania 305 (5.1) 334 (4.6) 384 (4.3) 502 (4.6) 557 (5.4) 590 (5.9) 223 Montenegro Ontario 279 365 (3.5) (5.6) 308 395 (2.8) (5.2) 358 450 (2.2) (5.2) 477 570 (2.4) (4.7) 531 619 (2.3) (5.4) 563 649 (3.3) (6.3) 223 224 Uruguay 281 (3.5) 309 (2.7) 357 (3.3) 477 (3.4) 532 (3.6) 565 (5.2) 224 Lithuania 337 (3.8) 365 (3.8) 419 (3.0) 539 (2.9) 590 (3.5) 620 (4.0) 225 Poland Quebec 363 392 (4.5) (7.0) 391 426 (4.1) (6.0) 443 486 (3.0) (5.8) 565 606 (3.0) (6.0) 617 652 (3.6) (6.5) 649 681 (4.8) (6.9) 226 227 74 Table B.2.4 (cont’d) Variation in student performance for countries, economies, and provinces: MATHEMATICS Percentiles Difference in score points between the 10th and 90th percentiles 5th 10th 25th 75th 90th 95th Japan Standard Score error 381 (5.6) Standard Score error 416 (4.4) Standard Score error 474 (3.5) Standard Score error 594 (3.5) Standard Score error 643 (4.2) Standard Score error 672 (5.4) Canada 368 (3.7) 400 (3.2) 456 (2.9) 577 (2.6) 627 (3.2) 657 (3.6) 227 Slovenia 363 (3.5) 394 (2.5) 449 (2.1) 572 (1.9) 622 (3.0) 651 (4.1) 228 Brazil 240 (3.0) 267 (3.3) 315 (3.1) 434 (3.7) 496 (4.7) 533 (5.5) 229 Croatia 322 (4.6) 351 (4.2) 402 (3.7) 525 (3.3) 580 (3.6) 612 (4.5) 229 United States 323 (4.7) 355 (3.9) 408 (3.9) 532 (3.5) 585 (4.2) 613 (5.0) 230 Germany 356 (4.9) 389 (4.1) 445 (3.5) 568 (3.4) 620 (3.4) 650 (3.9) 230 Hong Kong-China 389 (5.8) 426 (5.0) 490 (4.3) 611 (2.8) 659 (3.5) 687 (4.6) 232 Sweden 342 (5.0) 376 (4.4) 433 (3.8) 557 (4.0) 609 (3.9) 638 (4.7) 233 Moldova 271 (4.8) 303 (3.7) 358 (3.4) 482 (3.3) 536 (4.1) 568 (4.2) 233 Greece 306 (5.7) 336 (5.3) 391 (5.0) 517 (4.0) 570 (3.7) 598 (4.2) 234 Czech Republic 340 (4.8) 373 (4.2) 431 (3.4) 555 (2.9) 608 (3.6) 639 (4.4) 235 The Netherlands 356 (3.9) 390 (3.9) 449 (3.3) 579 (2.4) 627 (3.1) 655 (3.6) 237 New Zealand 342 (3.8) 375 (3.8) 431 (3.2) 560 (2.8) 613 (3.1) 646 (4.4) 238 United Kingdom 337 (4.3) 371 (3.7) 430 (3.2) 556 (3.1) 610 (3.1) 641 (4.0) 239 Georgia 250 (4.9) 285 (4.3) 341 (3.6) 467 (3.4) 525 (4.7) 559 (6.3) 240 Italy 334 (4.7) 368 (3.8) 426 (3.3) 555 (3.6) 610 (3.8) 640 (4.4) 241 Iceland 333 (3.9) 367 (3.6) 424 (3.0) 553 (2.7) 608 (4.0) 640 (4.3) 241 Cyprus 286 (3.4) 317 (3.5) 373 (2.2) 501 (2.2) 558 (3.0) 590 (3.9) 241 Australia 339 (2.8) 371 (2.5) 430 (2.0) 559 (2.1) 613 (2.8) 645 (3.3) 242 Luxembourg 334 (2.8) 363 (2.2) 417 (2.1) 553 (2.0) 607 (2.5) 638 (3.7) 244 Republic of Macedonia 217 (4.5) 251 (3.0) 306 (2.0) 434 (2.4) 496 (3.4) 533 (4.4) 245 Hungary 321 (4.0) 351 (4.1) 411 (3.7) 543 (3.2) 598 (3.5) 627 (4.0) 246 Singapore 399 (2.8) 436 (2.6) 500 (2.4) 632 (1.6) 682 (2.4) 711 (3.4) 247 Austria 337 (5.7) 370 (4.5) 431 (3.9) 564 (3.4) 618 (3.7) 648 (4.2) 247 Slovak Republic 312 (5.4) 349 (4.2) 412 (3.9) 543 (2.8) 596 (3.3) 625 (3.9) 247 Switzerland 358 (5.1) 394 (4.4) 455 (3.9) 590 (3.4) 641 (3.4) 671 (3.9) 247 Portugal 332 (4.4) 365 (3.8) 424 (3.1) 561 (2.8) 614 (3.6) 644 (4.1) 249 France 331 (4.5) 364 (3.9) 425 (3.3) 564 (2.6) 613 (2.7) 639 (3.3) 249 United Arab Emirates 275 (3.8) 306 (3.3) 360 (2.9) 493 (3.2) 557 (3.5) 593 (3.6) 251 Trinidad and Tobago 265 (3.6) 294 (3.0) 348 (2.4) 484 (2.1) 545 (3.3) 578 (3.5) 251 Bulgaria 284 (5.6) 315 (5.2) 371 (4.7) 509 (4.9) 568 (5.6) 601 (5.8) 253 Belgium 341 (4.4) 374 (3.9) 438 (3.5) 579 (2.5) 630 (2.5) 657 (2.7) 255 Korea 353 (5.9) 391 (5.5) 458 (4.5) 594 (4.2) 649 (4.3) 681 (4.8) 258 Qatar 248 (2.6) 278 (2.0) 331 (1.8) 470 (1.6) 536 (2.0) 573 (2.8) 258 Lebanon 236 (5.5) 268 (5.2) 324 (4.7) 464 (4.6) 531 (5.5) 568 (6.2) 263 Chinese Taipei 364 (4.4) 404 (4.2) 474 (3.6) 616 (3.6) 670 (4.6) 701 (6.2) 266 Israel 296 (5.3) 332 (4.7) 396 (4.3) 545 (4.3) 601 (4.9) 634 (6.1) 269 BSJG-China 351 (6.7) 388 (5.9) 458 (5.9) 609 (5.8) 664 (5.6) 695 (6.2) 276 Malta OECD average 289 340 (5.9) (0.8) 331 373 (3.5) (0.7) 405 428 (2.5) (0.6) 558 553 (2.5) (0.5) 616 605 (3.0) (0.6) 648 634 (4.3) (0.7) 285 232 Country, economy, or province 227 Note: Countries, economies, and provinces have been sorted in ascending order by the difference in score points between the 10th and 90th percentiles. BSJG-China represents Beijing, Shanghai, Jiangsu, and Guangdong. The coverage of Argentina, Kazakhstan, and Malaysia is too small to ensure comparability. See OECD, PISA 2015 Results for a note regarding Cyprus. 75 Table B.2.5 Estimated average scores by language of the school system for Canada and the provinces: READING Anglophone school system Canada and provinces Francophone school system Difference between systems Average Standard error Average Standard error Difference Standard error Canada 527 (2.7) 526 (4.7) 1 (5.6) Nova Scotia 519 (5.1) 462 (7.6) 57* (9.2) New Brunswick 509 (6.6) 493 (6.3) 16 (8.7) Quebec 523 (6.0) 533 (5.3) -10 (8.3) Ontario 529 (4.5) 476 (5.0) 54* (6.4) Manitoba 501 (5.3) 461 (8.1) 40* (9.6) Alberta 534 (5.2) 487 (12.6) 46* (14.5) British Columbia * Statistically significant differences. 536 (5.6) 516 (14.9) 20 (14.6) Table B.2.6 Estimated average scores by language of the school system for Canada and the provinces: MATHEMATICS Anglophone school system Canada and provinces Francophone school system Difference between systems Average Standard error Average Standard error Canada 509 (2.6) 542 (5.0) -34* (5.5) Nova Scotia 497 (4.7) 491 (8.3) 7 (8.7) New Brunswick 488 (5.8) 505 (7.3) -17* (8.5) Quebec 505 (6.7) 549 (5.4) -44* (9.0) Ontario 510 (4.4) 496 (6.5) 14 (7.8) Manitoba 489 (4.5) 482 (8.9) 8 (10.7) Alberta 512 (4.7) 503 (12.4) 8 (12.8) British Columbia * Statistically significant differences. 522 (5.0) 531 (16.0) -9 (16.9) 76 Difference Standard error Table B.2.7 Estimated average scores by gender for Canada and the provinces: READING Females Canada and provinces Males Difference (female-male) Average Standard error Average Standard error Difference Standard error Canada 540 (2.5) 514 (2.6) 26* (2.1) Newfoundland and Labrador 514 (4.5) 496 (5.3) 18* (6.8) Prince Edward Island 534 (6.3) 497 (8.3) 36* (8.6) Nova Scotia 531 (5.3) 503 (6.0) 28* (5.5) New Brunswick 518 (4.9) 494 (7.0) 24* (6.6) Quebec 541 (5.0) 522 (5.9) 19* (5.7) Ontario 542 (4.7) 512 (4.8) 30* (3.7) Manitoba 512 (6.2) 486 (5.4) 26* (5.8) Saskatchewan 508 (4.5) 485 (4.0) 23* (4.7) Alberta 545 (6.1) 521 (5.2) 24* (4.8) British Columbia * Statistically significant differences. 549 (5.3) 522 (6.9) 27* (4.9) Table B.2.8 Estimated average scores by gender for Canada and the provinces: MATHEMATICS Females Canada and provinces Males Difference (female-male) Average Standard error Average Standard error Difference Standard error Canada 511 (2.6) 520 (2.9) -9* (2.8) Newfoundland and Labrador 477 (3.8) 494 (5.0) -17* (6.3) Prince Edward Island 499 (7.1) 499 (8.9) 0 (9.7) Nova Scotia 494 (4.8) 500 (5.7) -6 (5.3) New Brunswick 490 (4.8) 495 (6.8) -5 (6.1) Quebec 538 (5.2) 550 (5.7) -13* (5.2) Ontario 505 (4.7) 514 (4.7) -8* (4.2) Manitoba 485 (5.4) 493 (4.6) -8 (5.5) Saskatchewan 479 (4.1) 489 (3.9) -10 (5.5) Alberta 506 (5.2) 517 (5.2) -11* (4.4) British Columbia * Statistically significant differences. 517 (5.2) 527 (6.0) -10* (4.9) 77 Table B.2.9a Comparisons of performance, PISA 2000, 2003, 2006, 2009, and 2012, Canada and the provinces: READING 2003 2000 Canada and provinces 2006 Canada 534 (1.6) 528 (5.6) 527 (5.5) 524 (5.2) 523 (6.2) 527 (7.2) Newfoundland and Labrador Prince Edward Island 517 (2.8) 521 (6.2) 514 (5.9) 506 (6.1) 503 (7.0) 505 (7.6) 517 (2.4) 495* (5.8) 497* (5.7) 486* (5.5) 490* (6.5) 515 (9.1) Nova Scotia 521 (2.3) 513 (5.8) 505* (6.1) 516 (5.6) 508 (6.7) 517 (8.4) New Brunswick 501 (1.8) 503 (5.6) 497 (5.5) 499 (5.5) 497 (6.5) 505 (8.6) Quebec 536 (3.0) 525 (6.8) 522 (7.1) 522* (5.8) 520* (6.9) 532 (8.3) Ontario 533 (3.3) 530 (6.4) 534 (6.8) 531 (5.8) 528 (7.4) 527 (8.1) Manitoba 529 (3.5) 520 (6.3) 516 (6.1) 495* (6.1) 495* (6.8) 498* (8.4) Saskatchewan 529 (2.7) 512* (6.8) 507* (6.5) 504* (6.0) 505* (6.5) 496* (7.7) Alberta 550 (3.3) 543 (6.8) 535* (6.5) 533* (6.8) 525* (7.2) 533 (8.6) Average Standard error 2015 Standard error Average Standard error 2012 Average Average Standard error 2009 Average Standard error Average Standard error British Columbia 538 (2.9) 535 (5.9) 528 (7.5) 525 (6.5) 535 (7.4) 536 (8.8) * Statistically significant differences compared with PISA 2000. Note: The linkage error is incorporated into the standard error for 2003, 2006, 2009, 2012, and 2015. Also, for some provinces, the standard errors from 2000 to 2003, to 2006, and to 2009 differ from those in the previous PISA reports on trend results. These differences result from the change of the method used by the OECD to compute the linkage error. Table B.2.9b Comparisons of performance, PISA 2009 and 2012, Canada and the provinces: READING 2009 Canada and provinces 2012 2015 Average Standard error Average Standard error Difference Canada 524 (1.5) 523 (3.2) 527 (4.1) Newfoundland and Labrador 506 (3.7) 503 (4.5) 505 (4.9) Prince Edward Island 486 (2.4) 490 (3.7) 515* (7.0) Nova Scotia 516 (2.7) 508 (4.0) 517 (6.0) New Brunswick 499 (2.5) 497 (3.7) 505 (6.3) Quebec 522 (3.1) 520 (4.4) 532 (5.8) Ontario 531 (3.0) 528 (5.1) 527 (5.6) Manitoba 495 (3.6) 495 (4.2) 498 (6.0) Saskatchewan 504 (3.3) 505 (3.8) 496 (4.9) Alberta 533 (4.6) 525 (4.8) 533 (6.2) British Columbia 525 (4.2) 535 (5.2) 536 (6.5) * Statistically significant differences compared with PISA 2009. Note: The linkage error is incorporated into the standard error for 2012 and 2015. 78 Standard error Table B.2.10a Comparisons of performance, PISA 2003, 2006, 2009, and 2012, Canada and the provinces: MATHEMATICS 2003 2006 2009 Standard error Canada 532 (1.8) 527 (2.4) 527 (2.6) 518* (2.7) 516* (6.1) Newfoundland and Labrador 517 (2.5) 507* (2.8) 503* (3.5) 490* (4.2) 486* (6.4) Prince Edward Island 500 (2.0) 501 (2.7) 487* (3.0) 479* (3.2) 499 (8.5) Nova Scotia 515 (2.2) 506* (2.6) 512 (3.0) 497* (4.5) 497* (7.2) New Brunswick 511 (1.4) 506 (2.5) 504* (3.0) 502* (3.2) 493* (7.5) Quebec 536 (4.5) 540 (4.4) 543 (4.0) 536 (3.9) 544 (7.4) Ontario 530 (3.6) 526 (3.9) 526 (3.8) 514* (4.5) 509* (7.0) Manitoba 528 (3.1) 521 (3.5) 501* (4.1) 492* (3.5) 489* (7.0) Saskatchewan 516 (3.9) 507 (3.6) 506 (3.8) 506 (3.6) 484* (6.3) Alberta 549 (4.3) 530* (4.0) 529* (4.8) 517* (5.0) 511* (7.3) Average Average Standard error 2015 Average Canada and provinces Standard error 2012 Average Standard error Average Standard error British Columbia 538 (2.4) 523* (4.6) 523* (5.0) 522* (4.8) 522* (7.5) * Statistically significant differences compared with PISA 2003. Note: The linkage error is incorporated into the standard error for 2006, 2009, 2012, and 2015. Also, for some provinces, the standard errors from 2003 to 2006 and to 2009 differ from those in the previous PISA reports on trend results. These differences result from the change of the method used by the OECD to compute the linkage error. Table B.2.10b Comparisons of performance, PISA 2012, Canada and the provinces: MATHEMATICS 2012 Canada and provinces 2015 Average Standard error difference Standard error Canada 518 (1.8) 516 (4.2) Newfoundland and Labrador 490 (3.7) 486 (4.8) Prince Edward Island 479 (2.5) 499* (7.3) Nova Scotia 497 (4.1) 497 (5.8) New Brunswick 502 (2.6) 493 (6.2) Quebec 536 (3.4) 544 (5.9) Ontario 514 (4.1) 509 (5.5) Manitoba 492 (2.9) 489 (5.5) Saskatchewan 506 (3.0) 484* (4.6) Alberta 517 (4.6) 511 (5.9) British Columbia 522 (4.4) 522 (6.1) * Statistically significant differences compared with PISA 2012. Note: The linkage error is incorporated into the standard error for 2015. 79 Table B.3.1 Multiple comparisons of achievement for countries, economies, and provinces: SCIENCE Instructions: Choose a country, economy, or province from the left-hand column. Read across the row to compare its performance with that of Canada and the provinces, listed along the top of the chart. The symbols indicate whether its performance is above, below, or the same as* that of Canada and the provinces. For example, choose Ontario from the left-hand column. Its performance is below that of Alberta, British Columbia, and Quebec; the same as that of Canada, Nova Scotia, and Prince Edward Island; and above that of all other provinces. * (i.e., any difference is not statistically significant) 80 556 541 539 538 537 534 532 531 529 528 525 524 523 518 517 516 515 513 513 510 509 509 509 506 506 506 503 502 502 501 501 499 498 496 496 495 495 493 493 493 490 487 (1.2) (4.0) (4.3) (3.0) (4.7) (2.1) (2.7) (2.4) (1.1) (2.1) (3.9) (3.9) (2.5) (4.6) (4.5) (3.1) (5.4) (2.4) (1.3) (1.5) (2.6) (2.7) (2.3) (4.5) (3.2) (2.9) (2.4) (2.3) (2.4) (2.5) (2.4) (4.7) (2.3) (3.2) (3.1) (2.4) (2.1) (3.6) (2.3) (2.1) (1.6) (2.9) Saskatchewan Manitoba Newfoundland and Labrador New Brunswick Prince Edward Island Nova Scotia Ontario Canada Quebec British Columbia Alberta Standard Error Country, economy, or province Singapore Alberta British Columbia Japan Quebec Estonia Chinese Taipei Finland Macao-China Canada Vietnam Ontario Hong Kong-China BSJG-China Nova Scotia Korea Prince Edward Island New Zealand Slovenia Australia United Kingdom Germany The Netherlands New Brunswick Newfoundland and Labrador Switzerland Ireland Belgium Denmark Poland Portugal Manitoba Norway United States Saskatchewan Austria France Sweden Czech Republic Spain Latvia Russian Federation Average Average achievement significantly higher than comparison province or Canada. Average achievement not significantly different from comparison province or Canada. Average achievement significantly lower than comparison province or Canada. Table B.3.1 (cont’d) Multiple comparisons of achievement for countries, economies, and provinces: SCIENCE Instructions: Choose a country, economy, or province from the left-hand column. Read across the row to compare its performance with that of Canada and the provinces, listed along the top of the chart. The symbols indicate whether its performance is above, below, or the same as* that of Canada and the provinces. For example, choose Ontario from the left-hand column. Its performance is below that of Alberta, British Columbia, and Quebec; the same as that of Canada, Nova Scotia, and Prince Edward Island; and above that of all other provinces. * (i.e., any difference is not statistically significant) Saskatchewan Manitoba Newfoundland and Labrador New Brunswick Prince Edward Island Nova Scotia Ontario Canada Quebec British Columbia Alberta Standard Error Average Average achievement significantly higher than comparison province or Canada. Average achievement not significantly different from comparison province or Canada. Average achievement significantly lower than comparison province or Canada. Country, economy, or province Luxembourg 483 (1.1) Italy 481 (2.5) Hungary 477 (2.4) Lithuania 475 (2.7) Croatia 475 (2.5) Iceland 473 (1.7) Israel 467 (3.4) Malta 465 (1.6) Slovak Republic 461 (2.6) Greece 455 (3.9) Chile 447 (2.4) Bulgaria 446 (4.4) United Arab Emirates 437 (2.4) Uruguay 435 (2.2) Romania 435 (3.2) Cyprus 433 (1.4) Moldova 428 (2.0) Albania 427 (3.3) Turkey 425 (3.9) Trinidad and Tobago 425 (1.4) Thailand 421 (2.8) Costa Rica 420 (2.1) Qatar 418 (1.0) Colombia 416 (2.4) Mexico 416 (2.1) Montenegro 411 (1.0) Georgia 411 (2.4) Jordan 409 (2.7) Indonesia 403 (2.6) Brazil 401 (2.3) Peru 397 (2.4) Lebanon 386 (3.4) Tunisia 386 (2.1) Republic of Macedonia 384 (1.2) Kosovo 378 (1.7) Algeria 376 (2.6) Dominican Republic 332 (2.6) Note: Significance tests were not adjusted for multiple comparisons. Five per cent of the comparisons would be statistically significant by chance alone. The results of Argentina, Kazakhstan, and Malaysia are excluded because of insufficient coverage to ensure comparability (see Appendix B.1.2 for these results). 81