Ofsted: Worlds Apart? (1996)
Prepared for Ofsted by David Reynolds and Shaun Farrell, this review looked at the major international surveys of educational achievement between 1964 and 1990.
The text of Worlds Apart? was prepared by Derek Gillard and uploaded on 15 March 2022.
Worlds Apart? (1996)
A Review of International Surveys of Educational Achievement involving England
London: Her Majesty's Stationery Office 1996
of Educational Achievement
UNIVERSITY OF NEWCASTLE UPON TYNE
UNIVERSITY OF NEWCASTLE UPON TYNE
UNIVERSITY OF WALES, CARDIFF
© Crown Copyright 1996
ISBN 0 11 350085 8
Office for Standards in Education
Telephone: 0171 421 6800
We are grateful to numerous people for their comments in responding to early drafts of this review of literature:
Bert Creemers, Dean Fink, John Gray, John Haslett, David Hopkins, Wendy Keys, Neil Macintosh, John McBeath, Margaret Maden, Peter Mortimore, Kate Myers, Jean Ruddock, Pam Sammons, Jaap Scheerens, George Smith, Sally Thomas, Peter Tymms.
We are also grateful to colleagues in The International School Effectiveness Research Project (ISERP) for permission to make use of some collectively gathered data in advance of publication of the full study.
Errors of fact and interpretation we, as all authors, claim exclusively for ourselves.
Our remit in this study was to review the internationally comparative studies of educational achievement that have involved England. We interpreted our task as requiring us to collect the following types of literature:
We have organised our material for this report into three parts.
The Introduction describes the increasing internationalisation of the field of educational research, and looks at the reasons for this. It outlines why the issue of how different countries perform educationally is now at 'centre stage' in many societies.
Section Two looks at the findings of the major data sources in the field and the international achievement surveys of the IEA and the IAEP; it also attempts an assessment of their limitations as valid or true measures of country differences, and looks briefly at other smaller scale studies.
The Conclusion uses a wide range of published and unpublished studies in an attempt to explain the pattern of findings shown in Section Two, a pattern which does not portray English levels of achievement in a particularly favourable light. It represents an attempt not to test hypotheses about English performance, but rather to generate them.
We have concentrated exclusively upon Science and Mathematics as the curriculum areas in which to undertake comparison between England and other countries, for a number of reasons:
We have referred throughout to 'England' as the home sampling unit, but of course this is not always an accurate description of the origin of the sample of children. In some studies, 'England' means purely English samples are used - in others, Welsh children are also included in the survey sample. In two studies, the 'United Kingdom' is the sampling unit, although England naturally provides the highest proportion of children taking part. We make clear when we discuss each study, which countries within the United Kingdom we are concerned with; information about the make-up of the 'English' sample and about all the countries taking part in the studies will be found in Table Four, at the beginning of Section Two. It is also important to note that Scotland is included in many of the surveys in its own right: whilst we do not explicitly point up Scottish performance in every study, we return at the end of this review to the issue of variation in performance between England and Scotland.
We live in a world that is becoming 'smaller' all the time. The spread of mass communications, and particularly of satellite broadcasting, makes ideas that were formerly found only in isolated cultural niches globally available. The enhanced interactions between citizens of different countries through visits, vacations, migration and electronic contact are clearly both breaking down cultural barriers and yet, at the same time, also leading to a reassertion of cultural distinctiveness.
The educational world is also becoming 'smaller' all the time. In the United Kingdom, references to the superior achievements of 'Pacific Rim' economies or the 'Tiger' economies now pepper the speeches of Government and Opposition spokespersons (see Tony Blair's Platform column in The Times Educational Supplement of 23rd June 1995), as does acknowledgement of the educational reasons for their success. Only two decades ago, there was little reference in discussion of educational policies within the United Kingdom to 'overseas' evidence, save for occasional acknowledgements of the apparent success of Scandinavian comprehensive schools from the 'liberal' or 'left' wings (Reynolds et al, 1987) and of the success of German training and education-for-work provision (Prais and Wagner, 1965). In the debate about the necessity of educational reform in the mid 1980s, in fact, comparisons were usually made with Britain's own past, rather than with other contemporary countries (see Hargreaves and Reynolds, 1989; Ball, 1990).
Now all this is changing. The recent report Learning to Succeed (National Commission on Education, 1993) makes frequent reference to non- British societies. Indeed, one could not find better evidence for the changes taking place than the attention given by every 'quality' daily newspaper and the front page of The Times Educational Supplement to a recent report comparing teaching methods in Switzerland and England (Bierhoff, 1996). Comparable research on the relative educational performance of England and Germany in the mid-1980s barely rated a mention in the press of the time, although it attracted attention from within the ranks of educators and politicians. What factors are responsible for the increased internationalisation of educational discussion and debate?
Firstly, it is clear that economic pressures on traditional industrial societies have intensified in recent years with the emergence of what has been called the 'Asian economic miracle'.
Societies such as Korea, China, Japan and Taiwan have been regularly achieving growth rates of 6-7% per year, as against rates of only half that amount in the traditional economies of the OECD, including the United Kingdom. Thailand will, on present trends, exceed the United Kingdom in its Gross Domestic Product by the year 2010. Indeed, what has been called a 'recession' in the economies of North America and Europe since the late 1980s is mostly the result of a global redistribution of wealth towards the newly emerging economies of the East.
Explanations for the success of these economies have ranged widely and have included their strong family and community networks, their cohesive social structures, the pervasiveness of 'pro-social' attitudes and values encouraged by religious traditions and, frequently, their apparently very high levels of educational achievement.
The second set of reasons for the internationalisation of educational debate are connected with changes in the educational research communities in North America and Europe. Specifically:
(i) The emergence of organisations such as the European Educational Research Association and the International Congress for School Effectiveness and Improvement, and of journals such as School Effectiveness and School Improvement. This has led researchers in such areas as school effectiveness, educational evaluation, teacher effectiveness and educational policy to become increasingly aware both of intellectual traditions, and of educational processes and outcomes, that are different to those of their 'home' country. An example of this is the emergence of Dutch empirical research in the field of school effectiveness (Creemers, 1994; Creemers and Scheerens, 1989), which was largely unknown in the United Kingdom and United States as late as 1988, and which now is central to the effectiveness movement. The same inclusion of societies seems to be happening currently with Pacific Rim societies (see for example the work of Cheng, 1993).
(ii) Educational researchers have realised that they need to understand more clearly why some school effectiveness variables 'travel' across countries, whilst others do not. 'Top down' leadership from Principals is perhaps one of the most well supported of all the American school effectiveness 'correlates' within the 'five factor' theory originally formulated by Edmonds (1979). This has subsequently developed into a 'seven factor' theory by Lezotte (1989); yet in spite of the massive empirical support for the importance of this factor in the United States (e.g. Levine and Lezotte, 1990), Dutch researchers (van de Grift, 1990) have been generally unable to validate its importance in the Dutch educational and social climate (apparently Dutch Principals are often promoted out of the existing school staff and have a much more facilitative, group-based and perhaps 'democratic' attitude to their colleagues).
What explains why a factor is important in one country but not in another? Educational researchers have become interested in finding out.
Interestingly, it is the secondary analysis of existing international studies of educational achievement that have been a further spur to this trend. Scheerens et al (1989), for example, used a re-analysis of data from an IEA study to generate interesting material upon the
'context specificity' of factors associated with educational achievement in different countries, as shown in Table One at the end of this section. Whilst home background factors are associated across most countries with Mathematics achievement, and whilst expectations and 'opportunity to learn' appear to be consistently important factors across most countries, some factors appear to be associated with achievement only within certain national contexts. In the case of Belgium, class size is positively linked with achievement in the French speaking part of the country, but negatively associated with achievement in the Flemish speaking part.
Findings such as these at international level, and recent American research showing that different school effectiveness features prevail in different socio-economic contexts (Teddlie and Stringfield, 1993; Hallinger and Murphy, 1986), have propelled educational researchers increasingly in the direction of 'context specific' theories, in which effectiveness is contingent upon national contexts.
The problem is, though, that just as the educational world is aware that effectiveness factors may not 'travel' across countries, the political world is increasingly inclined to transfer features from one context to another; some American states have adopted shorter school holidays and lengthened the time pupils spend in school, simply because 'time to learn' is higher in educationally successful countries like Japan (see Business Week, 14th September, 1992).
(iii) Educational researchers have also increasingly come to realise that only international research can tap the full range in school and classroom quality, and therefore in potential school and classroom effects. The range of school factors concerning 'quantity' (such as size, financial resources, quality of buildings), and 'quality' (such as setting standards of achievement) is likely to be much smaller within countries, than between countries.
The existing estimates we have, based on all the empirical studies concerning the size of educational influences (schools and classrooms together) on students, have settled into a range of from 8% to 14% of variance in virtually all empirical studies (see Reynolds and Cuttance, 1992). It is likely that these estimates are merely artefacts of the studies' lack of school and classroom variation. The true power of education, researchers have begun to mink, may only be shown by internationally based research.
(iv) The fourth reason why educational researchers have been increasingly keen to internationalise their field is that they have become aware that international study is likely to generate more sensitive theories than those at present on offer, since both intellectually interesting and practical middle-range theories are connected with the ways in which school and classroom factors travel cross-culturally to a very varied degree. Why is it that the presence of 'an assertive Principal' is not a key factor in school effectiveness in the Netherlands - what is it in the local, regional or national ecology that might explain this finding? Answering this question inevitably involves the generation of more complex explanations that operate on a number of levels, and is likely to generate more complex theoretical explanations than those generated by the simple within country research findings.
Indeed, it is a review of what 'travels' and what 'does not travel', in terms of types of school and instructional factors, that has generated what is one of the most interesting new ideas within school effectiveness, since it is apparently the instructional variables - such as time on
task, opportunity to learn, structured teaching and high expectations - that 'travel' much more than do the standard school factors customarily used. The possibility that one needs different school processes in different countries to generate the same effective classroom processes is a potentially devastatingly important insight (see Reynolds et al, 1994).
(v) The fifth reason for the enhanced internationalisation of education, and educational research, is simply that people have become aware that other societies have educational features mat may be of interest to their own society. For educationists in societies like the United Kingdom, where education is often criticised for its less than exceptional quality, the realisation that other societies may provide 'new' factors or processes that are not tarred with the 'tried but found wanting' label has been of considerable interest.
It is clear that educational researchers are following some degree behind the general internationalisation of educational discussions that has been a feature of the last decade (see for example the work of Lynn (1988) and Howson (1991)). Nevertheless, the adoption of an international frame of reference is now proceeding very rapidly indeed, as we have seen.
There is no doubt that educational debate has become increasingly internationalised over the last decade. The professional community of educational researchers has developed internationally inclusive organisations, and economic pressures have propelled debate about the educational achievements of different countries to centre stage. This phenomenon is likely to be continued when the Third International Mathematics and Science Study (TIMSS) and the ISERP study are published later this year (see Reynolds et al, 1994 and Time for the West to go East', The Times Educational Supplement, September 29th, 1995, for details of the latter).
Such comparisons of different countries have potentially much use for researchers in England: they offer a chance to see factors in systems which do not exist in our own culture, and the possibility of developing an answer to the question of 'what factors travel, and why'.
In the next section, we move on to look at the comparisons that have been undertaken of educational achievement in England and in other societies. The directions in which our understanding has been taken by these and other studies form the contents of Conclusions.
Factors Associated with Effectiveness in Different Countries (from Scheerens et al, 1989)
Predictor Variables with significant Positive (p) or Negative (n) Associations with Mathematics Achievement
The Problems of International Comparisons
A number of different studies have been carried out over the last two decades that have attempted to describe and analyse the causes of international variation in educational achievement. Whether large scale (like those of the IEA or IAEP variety), or small scale (like the England/Germany comparisons of Prais and Wagner, 1985), and whether quantitative or qualitative, all investigations face the same basic problem: how does one measure the influence of the educational system and its responsibility for variation in educational achievements?
Table Two shows, as an example, the Mathematics performance of 13 year old pupils in eighteen different countries in 1992, together with the 'rank' obtained by the country and the percentage of answers correct on the Mathematics test used.
England appears to do relatively poorly in comparison to Pacific Rim societies such as China, Korea and Taiwan, a finding that will often be repeated as we progress through the various studies reviewed in this section. But how do we explain English low achievement? Is it that English pupils lack the strong respect for school learning that characterises pupils in Pacific Rim societies? Has the 'selective' educational history of the British system left its pupils with low expectations; or are there other factors at work?
Proficiency Test Scores in Mathematics from the International Assessment of Educational Progress 2 (IAEPM 2) 1990 (from Foxman 1992)
A second major problem also presents itself. Table Three, as an example, shows the distribution of educational qualifications within the workforces of various European societies.
Vocational Qualifications of the Workforce in Various European Societies, 1988-91 (from Prais 1994)
Again, the performance of Britain is poor by comparison with other European countries, But to what extent are we comparing 'like' with 'like'? 'Intermediate Vocational Qualifications' in Britain are defined as BTEC, HNC, HND and the like - do the German or French qualifications 'mean the same thing' as the British?
All studies in this field have, then, two basic problems - societies must be compared in their performance on the same skills, bodies of knowledge or tests, and attempts must be made to ensure that the educational causes of any differences are isolated from any other possible causes.
The majority of the studies referred to in this section have attempted to solve the first of these problems by using Mathematics and Science tests where the same definition of what is the 'right' answer applies across different cultures. 'Cohort' designs have also been used to control out non-educational influences. Cross-sectional studies of the kind that generated Table Two obtain measurements at a point in time, and then disentangle the various educational, social, economic and cultural influences that were responsible for these measurements over historic time. By contrast, cohort studies that look at children over time can study the relative gains children make, after allowing for the various different starting points in different countries. The 'true' effect of education in these countries can then be calculated and compared by 'controlling out' the background factors that are non-educational. The expense and complicated nature of cohort studies means, though, that they have rarely been used; given the problems involved in the longitudinal element of the Second International Mathematics Study (SIMS) (Garden, 1987), including limited participation, limited objectives and use of a very short time period, one can see why. The absence of 'cohort' studies leaves unsolved, however, the problem of the extent to which country differences reflect educational or non-educational influences.
Some Further Methodological and Practical Problems
Although the two problems noted above place limitations on the capacity of existing international surveys of achievement to definitively address the issue, 'Which countries are more effective educationally, and what is the educational contribution?', it is important to note that there are further limitations upon the IEA and IAEP achievement studies of the last twenty years (see for further discussion Goldstein, 1989, 1993; and Reynolds et al, 1994).
Of course, these surveys face the kinds of problems that are present in all cross-national research - of accurate translation of material, of ensuring reliability in the 'meaning' of questionnaire items (such as social class or status indicators for example), of inconsistencies caused by Southern Hemisphere countries having school years that begin in January, and of the need for retrospective information.
In certain curriculum areas, the cross-national validity of the tests gives cause for grave concern and the IEA study of Written Composition failed, for example, in its attempt to compare the performance of groups of students in different national systems that used different languages. This study concluded that the 'construct that we call Written Composition must be seen in a cultural context and not considered a general cognitive capacity or activity' (Purves, 1992, p199). Even the administration of an achievement test in the varying cultural contexts of different countries may pose problems, particularly in the case of England, where the test mode in which closed questions are asked in an examination style format under time pressure may not be frequently experienced. By contrast, the use of this test mode within a society such as Taiwan, where these assessment methods are frequently experienced, may inflate Taiwanese scores, just as it may depress English achievement below its 'real' level.
In addition to these basic problems that affect all large scale international comparative research, there are also specific problems concerning the IEA and IAEP studies we are about to look at:
It is with such caveats borne clearly in mind that we approach our review of the studies, the whole body of which is laid out in Table Four.
We begin with the subject area of Mathematics [we have eliminated from our discussion the IEA Pilot Study of Foshay (1962), that involved Mathematics (and Science) because of the small sample size (9,000 pupils in 12 countries)].
The IEA and IAEP International Studies of Educational Achievement
Table Four continued
Table Four continued
1964 - The IEA First International Mathematics Study (FIMS)
The IEA First International Study of Mathematics Achievement involved twelve countries. The target populations for the study were as follows:
1a. All pupils who were aged 13 at the date of testing.Sampling procedures in this study were quite rigorous, and the populations quite large (the sample for England alone was 12,740 pupils in 684 schools). In England, post compulsory schooling was (and remains) characterised by selection and subject specialism, and this fact should be borne in mind when comparing achievement scores for England with countries which offered either universal education to this stage (such as the United States), or countries which had a multi-disciplinary curriculum at this level (the baccalauréat system of several European countries). An additional concern is the fact that the tests had more items from certain Mathematics topics (e.g. arithmetic), which may have disadvantaged certain countries like England.
It is also important to note that students in England and Scotland began school at least one full year earlier than pupils in most other countries in the 1964 study, and in all the studies in this section.
Achievement Results from the IEA First International Mathematics Study 1964
Overall, the study showed that:
The Second International Study of Mathematics Achievement involved twenty countries. The target populations for the study were as follows:
Population A was defined as: All students in the grade in which the modal number of students had attained the age of 13.0 - 13.11 years by the middle of the school year (the comparable group in FIMS is Population 1b.)
Population B was defined as: All students who were in the normally accepted terminal grade of the secondary education system, and who were studying Mathematics as a substantial part (approximately 5 hours per week) of their academic programme. (Population B in England was taken from sixth forms only).
In England the intervening period between FIMS (1964) and SIMS (1982-83) had seen the transformation from a selective, to a predominantly comprehensive system. Retention rates at the pre-university level had also risen slightly, but continued to be among the lowest in comparative terms with the disparity increasing as several countries' retention rates had risen more during the same period.
For the 13 year old population, Japan again had the highest overall achievement. England was once more about average for the study and continued to lag behind the Netherlands, Belgium and others, while France had caught up and overtaken England.
Examination of Table Six suggests that achievement in England was again very close to the overall study mean. However, this second study included several low scoring developing countries which would suggest that achievement levels in England had not remained stable over time. Evidence supporting this interpretation is found by comparing England to France, which had actually moved ahead by some distance. Scotland, which trailed England in the earlier study, had also moved slightly ahead.
For students studying Mathematics at the pre-university level, Table Six demonstrates that England continued to occupy a high position. However, comparing the pre-university Mathematics specialists in FIMS and SIMS, it is notable that Japan had overtaken England and was joined by Hong Kong (who did not take part in the first study) as the highest achieving countries.
Achievement Results from the IEA Second International Mathematics Study 1982-83
Curricula characteristics of tests
If retention rates for Mathematics at the pre-university level are taken account of, Table Seven shows that the achievement means for England were produced by a much smaller 'elite' population than in any of the other participating countries, with the exception of Hong Kong and Israel. As with FIMS, it would appear that the achievement of England may have reflected the selective nature of the English system.
Retentivity in Mathematics Classrooms (Pre-university year) 1982-83
Other findings in SIMS reflect upon many of the current debates and concerns about education in England today. For example, it was found that class size at the 13 year old level across countries did not consistently relate to achievement (Japan being among the highest in terms of mean class size). SIMS also found that a high retention rate did not necessarily produce a detrimental effect on achievement. In fact, the highest performing students in all systems gained very similar standards of achievement regardless of varying retention rates.
In terms of the Mathematics curriculum, the major differences between systems occurred at the pre-university level, although the apparent consistency at 13 year old level in curriculum coverage left many unanswered questions when juxtaposed with achievement differences between countries. For example; the English curriculum was "very similar" to that in Japan and New Zealand, but pupil achievements between these countries were very different.
Finally, the issue of 'opportunity to learn' (or the extent to which pupils had covered the test items), which was raised in FIMS, received much more attention in the second study. Specifically, SIMS sought to identify patterns of variation between countries in terms of both 'intended curriculum' (as defined nationally), and 'implemented curriculum' (as reported by the Mathematics teachers). Furthermore, and of particular significance in relation to the large variance in achievement within the English system, SIMS also looked at the extent of variation within countries. The findings are reproduced below to demonstrate the strength of the relationship between achievement and 'opportunity to learn', and demonstrate how the characteristic variation in achievement within England reflected a worrying variance in students' exposure to Mathematics curricula.
Table Eight shows the data concerned with 'opportunity to learn' the Mathematics curriculum for students in the 13 year grade. It suggests that there is little difference between systems in terms of their intended curricula at this level (all had reasonably high matches with the curriculum as defined by the SIMS test items). However, when consideration is given to what knowledge students actually had access to, it is evident that there were wide discrepancies between countries, and, importantly, that countries also vary in terms of the fit between 'intention' and 'implementation'.
In England, it can be seen that the 'intended curriculum' was a good match for the items covered by the SIMS tests. However, when consideration is given to the 'implemented curriculum', practice did not follow intention. Across each domain covered by SIMS, the 'implemented curriculum' fell short of what was intended.
Table Nine presents the variation in curriculum implementation within systems reported by the SIMS. Essentially, the darker the shading, the more within system variation which occurred in terms of pupils' access to the Mathematics curriculum.
Variation in Opportunity to Learn Systems 1982-83 for students of 13 years
Variation in Opportunity to Learn, 13 year old population, Within Systems 1982-83
Table Nine clearly demonstrates that, in terms of the delivery of the Mathematics curriculum at the 13 year grade level, England had one of the highest levels of within system variation (Japan, France and the Netherlands had among the lowest). Perhaps the most worrying feature was that England appeared to have one of the highest levels of within system variance in opportunity to learn for arithmetic, the foundation of Mathematics learning. It is therefore unsurprising that this pattern was mirrored by large variations between pupils within England. This finding, linked to the comparatively large numbers of low achievers, explains why the achievement of our 13 year olds in Mathematics is only average in an international context.
Overall, the study showed that:
Assessing the extent of changes over time in achievement is clearly difficult, since different samples of children were used at different time points. The most accurate means of assessing changes between the two studies is to examine achievement on the 'bridge' test items - that is, those items which featured in the tests at both points. Table Ten shows the mean achievements for all countries who participated in both studies, for both the 13 year old grade and the pre-university Mathematics specialist populations.
General Patterns of Change in Mathematics Achievement (Based on 'bridge' Items Common To Both Studies)
The results demonstrate that in England, the mean achievement for all the 'bridge' items combined had fallen dramatically for the 13 year olds. It is also noticeable that the same can be said for several other countries at this age (exceptions are France and the Netherlands). There was a similar pattern for the pre-university population. Although here the achievement of students in England continued to be comparatively high, it is worth observing that Japan had overtaken England by a considerable margin, and that Finland and Sweden had closed the gap on England. The extent to which this may be due to changes in the English Mathematics curriculum between 1964 and 1982-3 is unclear.
Table Eleven demonstrates that only in England did achievement fall in all four main curricular areas. Although all countries' mean scores were reduced over time in arithmetic, England had among the highest reductions, which is surprising, since this was from an already low base. For algebra, England was the only country that had declining achievement.
Specific Changes in Mathematics Achievement by Curricula Areas (13 yr grade) (Based on 'Bridge' Items Common to Both Studies)
We present here only achievement percentages of the 'bridge' items (test items common to both studies) for those countries involved in both studies.
Note: the number of bridge items for each Subset are as follows: Arithmetic = 14 items, Algebra = 9 items, Geometry and Measurement = 7 items, Descriptive Statistics = 5 items.
Overall, the changes over time were as follows:
In addition to the IEA studies discussed above, there have been two studies of Mathematics undertaken by Education Testing Services (ETS) of the United States, entitled the International Assessment of Educational Progress (IAEP). These studies have employed a similar methodology, but have been restricted in the scope of their analyses, when compared with the IEA studies.
In the first IAEP study, participation was limited to six countries, with Canada divided into seven, giving a total of twelve education systems. The results are presented in terms of a Mathematics proficiency scale, with a range from 0-1000 and a mean of 500. As with the IEA studies, the target populations were defined as representative samples of 13 year olds from each system (the United Kingdom sample was taken from England, Wales, and Scotland, but not Northern Ireland). The Mathematics test consisted of 63 questions derived from the ETS item pool, which had been widely used in the United States.
Achievement Results from the IAEP First International Assessment of Mathematics (IAEPM 1) 1988
The findings here are consistent with those of the IEA studies. The United Kingdom achievement scores were again below the average for all countries in the study, although ahead of the United States. Canada (British Columbia) continued to have high achievement levels (as was shown in the SIMS) with Canada (Ontario) having achievement scores very similar to the United Kingdom. In Europe, the number of participating countries is limited, with Ireland and Spain showing similar levels of achievement to the United Kingdom. Another consistent feature is the superiority of Korea, and the size of margin it achieves over all other countries. Further discussion of the United Kingdom findings can be found in Keys and Foxman (1989).
In order to examine the consistency of achievement patterns in relation to specific Mathematics curriculum areas and opportunity to learn, Table Thirteen presents the achievement breakdown by the six curriculum areas represented in the IAEP test items. It also shows the 'implemented' curriculum in terms of 'opportunity to learn' defined by the Mathematics teachers involved in the study.
Percent Correct by Curriculum Area and Opportunity to Learn, (IAEPM1) 1988
Table Thirteen continued
Table Thirteen is again consistent with the findings of the IEA studies. Students in the UK continue to demonstrate comparatively high levels of achievement in areas such as geometry, and logic and problem solving, yet in comparative terms they continue to achieve poorly in the basics of 'numbers and operations'. It is noteworthy that 'numbers and operations' items represent a third of the entire test battery of IAEP, which would tend to work against the UK. With regard to 'opportunity to learn', Table Thirteen suggests a clear relationship with achievement in the UK for 'numbers and operations' and 'logic and problem solving', with the other curricular areas giving a mixed picture. A similar relationship is evident in other systems such as Quebec (French) and Korea which are two of the highest achieving systems. Obvious exceptions to this relationship are British Columbia, where low opportunity to learn ratings are associated with consistently high achievements, and Spain, where the reverse pattern is evident.
Overall, the study showed that:
This study (Foxman, 1992) built on the first, but is different in that twenty countries took part, and that 9 year olds were added to the 13 year old group that formed the target of the first study. Additionally, a wide range of data was gathered on educational factors (class size, amount of instruction, availability of resources etc.) and background factors (use of television, reading material in the home, views of Mathematics etc.) that might have influenced results. In this study, Scotland is analysed separately, whereas in IAEPM 1 it formed part of the English sample.
The skill areas covered at ages 9 and 13 were as follows:
Percentage Correct Items for Various Countries (IAEPM 2) 1990
We do not report this study in full, since many of the detailed findings of the study parallel those reported already for the other Mathematics studies in this review. However, it is worth noting that the response rate (below 70% at both ages), which was the lowest of any participating country, limits the confidence that can be placed in the findings.
Overall, the study showed (see Foxman 1992):
1970-72 - The IEA First International Science Study (FISS)
The first large scale international study of Science achievement was part of a wider six subject survey. The Science part of the study involved nineteen countries and the target populations were defined as follows:
I. All students aged 10.0 - 10.11 years at the time of testing. This age group was chosen because in most systems students were still taught predominantly by non specialist primary teachers.The sample for England consisted of 9,227 students, selected to be representative of the population as a whole. As with all the IEA studies, the response rates for England were comparatively low.
It should also be remembered' that the education system in England was at this time going through the transformation from a selective to a comprehensive structure and that the Science curriculum was being influenced by changes in both curriculum emphasis and in delivery methods. The timing of this study was therefore not ideal for England.
Table Fifteen shows that students in England at age 10 achieved below the mean for the study. There was also evidence of the same large variation which was a consistent feature of the Mathematics studies. The pattern of achievement at age 14 was once again markedly consistent, the mean achievement score being below the mean for the study and the variance in achievement within England being substantially larger than for most other countries. As with the Mathematics studies, it was noticeable that Japanese performance was consistently the best, although the performance of other countries did not appear to be consistent across curricular areas.
The achievement of students in the pre-university year was difficult to assess in comparative terms, since the results concerned the entire pre-university population and there was therefore some bias toward those countries which offered an undifferentiated curriculum at this level. This would have inevitably disadvantaged England, as well as other countries, whose pre-university education provisions were characterised by specialisation; although there remained the issue of retention rates which clearly favoured England. Table Fifteen shows that for the pre-university population in England, achievement was relatively better than that of the compulsory school age populations.
Achievement Results from the IEA First International Science Study 1970-72
In Table Sixteen it is apparent from the retention rates that the system remained more selective in England than in the majority of other participating countries. In addition to the generally low retention rates in England at the time of the first Science study, it was also evident that a lower percentage of those students who did remain in education at the pre-university level were likely to continue to study Science, when compared with most other participating countries.
Retentivity in Science Classrooms (Pre-university year) 1970-72
Returning to the 10 year old population, it is notable that the highest between-school variance occurred in those countries where the streaming of pupils was the norm, which include England. At this age, home background variables were highly associated with achievement variation between schools, and also with variation between pupils. There was a large negative relationship between the number of hours spent watching television and achievement in England and the US, a relationship which was found in almost all countries to some degree. In terms of Science teaching, the study found that an 'unstructured' approach was likely to be less successful than a 'structured' approach. Variation in achievement was related to opportunity to learn in several countries, including England. As with the first IEA Mathematics study, the influence of attitudes and motivation were unclear, with Japanese students again demonstrating the least favourable attitudes to Science, yet gaining the highest achievements!
At the 14 year old level, home background variables continued to be the most important factor explaining between-school variance in England, Scotland, and the United States, with father's occupation being the most prominent factor. In terms of both between-school and between-student achievement variance, the 'type of school' attended was found to be very influential in England. 'Opportunity to learn' was again a factor related to the variation in achievement between schools in those countries characterised by differentiated school types such as England. Contrary to the findings at age 10, the 14 year old population was found to benefit from a more 'flexible' approach to learning in most countries,
For the pre-university sector, home background variables had little or no influence in most of the countries (as with the Mathematics studies this may be attributed to the homogeneity of the populations), and it is not surprising that, for England and other selective systems at the pre-university level, learning conditions and 'opportunity to learn' were among the strongest factors explaining variance in achievement between schools, and between students in schools.
Overall, the study showed that:
1983-85 - The IEA Second International Science Study (SISS)
The second large scale international study of Science achievement was undertaken in the mid 1980s, and involved twenty-three countries, The target populations were defined as follows:
1. All students aged 10.0 - 10.11 year's at the time of testing. (Or all students in the grade where most ten year olds were to be found on the specified date of testing).As was the case for Mathematics, the intervening period between FISS (1970/72) and SISS (1983/85) had seen England transform its educational provision towards a comprehensive system. There had also been considerable changes in curriculum emphases and modes of delivery since the early 1970s. The second Science study was designed to focus on representative samples in each of the populations, in each of the participating countries. It is worth stating that, as with most of the IEA studies reviewed, the response rates for England were extremely low in comparison with those achieved in other countries. In the SlSS for example, English response rates were only about 70% for schools and students in population 1, about 60% for schools and nearer 50% for students in population 2, and less than 55% for schools in population 3.
Table Seventeen shows that the mean achievement of English 10 year olds was considerably below the mean for the study and that the achievement variation within England remained among the highest in the study, in spite of the move to a comprehensive system. A similar pattern was found at the 14 year old level.
In the case of the 14 year olds, the variation in achievement within England was consistently large, but was less than in several other countries, including Japan which was one of the highest achieving countries. It is interesting to note that Japan continued to feature among the highest achieving countries at all population levels, and that Hungary, which had comparatively high achievements in Mathematics for 13 year olds in the SIMS study, had the highest mean achievement in Science in the SISS for 14 year olds.
For pupils in the final year at school, achievement in England was among the highest, willie the extent of variation of pupil achievement was comparatively low, although greater wan the two highest achieving countries, Hungary and Sweden.
Achievement Results from the IEA Second International Science Study 1983-85 (Excluding Four Developing Countries)
Achievement Results from the IEA Second International Science Study (pre-university Science specialists)1983-85
Table Eighteen demonstrates that lie Science specialists in England achieved consistently well across each of the Science curricular areas identified. This finding is consistent with the previous Mathematics studies and suggests that the influence of a comparatively low retention rate and the continued subject specialisation at the pre-university level continued to favour England. The amount of variation demonstrated within the pre-university Science specialists was consistently below the mean for the study.
Several findings from the SISS are worthy of note regarding England. At the 10 year old level, there were very large between school differences in England, as well as in a few other countries, which contrasts with very low between school differences in achievement in Japan, Finland and most countries. The authors of the report note that "this large variation of achievement is surprising in some countries which pride themselves on equality of 'educational opportunity' and is clearly a point to which those responsible for the national planning of education could give attention" (Postlethwaite and Wiley, 1992, p 78). The lack of a national curriculum at the time of the SISS should be noted.
SISS also found a strong relationship between levels of achievement at 10 year old and 14 year old levels, which would indicate the importance of a good grounding in Science. The fact that English students achieve comparatively poorly in all branches of Science at the 14 year old level would seem to support this interpretation, although the sample tested in England at age 14 was younger than the samples tested from other countries.
Some investigation was also carried out into the extent to which there was variation between countries in their 'opportunity to learn' across the four subjects that comprised Science for Population Two (the 14 year olds). Table Nineteen shows the variation between countries on 'opportunity to learn,' with England towards the lower end of the range of countries in the curriculum exposure of its pupils to Biology, Chemistry, Earth Science and Physics.
Average Percent Opportunity to Learn by Subject and Country (14 year olds) 1983 -85
Although there was a moderate relationship between 'opportunity to learn' and achievement at between country level, stronger than the relationship with achievement within countries, adjusting country means to take account of this does little to change the overall ranking of countries, as Table Twenty shows.
Total raw achievement means by country, and adjusted means for opportunity to learn (14 year olds) 1983-85
Overall, the study showed that:
1988 - The IAEP First International Assessment of Science (IAEPS 1)
This study was linked to the IAEPM 1 study reported earlier, and took place within 12 different countries or regions (7 of the 12 settings were systems within Canada). The home sample used was a United Kingdom one, and the study followed the customary pattern of an achievement test (in this case administered to 13 year olds) and a set of further data on pupil background, pupil attitudes and the 'opportunity to learn' of different pupils rated by teachers. It is important to note that the United Kingdom response rate was 70%, compared to 89% or higher for other locations.
Table Twenty One shows the results, with the United Kingdom sample doing better in this study than in the other studies reported earlier. Countries were divided into three groups with Korea and British Columbia in the 'top' group, the United States in the 'bottom' group, and the United Kingdom in the 'middle' group.
Table Twenty One
Average Science Proficiency (IAEPS 1) 1988
Table Twenty Two
Percent correct by Curricular Area and Opportunity to Learn Ratings for Science IAEPS 1 1988
Table Twenty Two continued
Percent Correct by Curricular Area and Opportunity to Learn Ratings for Science, IAEPS 1 1988
Table Twenty Two shows the United Kingdom performance on the five sub-areas of scientific knowledge; it is clear that performance in Physics, and Earth and Space Sciences, was better than that in Life Sciences and Chemistry. This table also shows the 'opportunity to learn' scores and rankings. As has been the case in a number of the studies reported in this review, mere does not appear to be a close relationship between exposure to a topic and achievement level across the countries concerned. As an example, the good United Kingdom performance on Earth and Space Sciences is surprising given the very low exposure to these topics in schools. (The report notes the possibility that the information required to answer questions on these topics may have been obtained from other subject disciplines, or from non-school sources like the media).
Overall, the study showed that:
The slightly more favourable picture of English achievement in IAEPS 1 is also evident in the second Science Study (IAEPS 2). Table Twenty Three shows the results for students aged 9 and 13, with the English performance being slightly above the mean for all countries. Pacific Rim countries again do well at both ages, and Switzerland, together with Hungary, does particularly well in the European group of countries for the older age group.
At age 9, England does worse on Life Sciences relative to the average for all countries, and best on Earth and Space Sciences and Nature of Science. At age 13, the English performance is worst on Earth and Space Sciences, and best on the Nature of Science!
As with Mathematics, English pupils are given a relatively low amount of Science homework, with only 2% of 13 year olds undertaking four hours or more per week, a figure exceeded or equalled by all except two of the countries involved. England is also very high in the proportion of students undertaking Science experiments in school (practical Science).
Many of the attitudinal measures used in the study have perplexing results. For example, high scoring Pacific Rim societies, and (among the European countries) Switzerland, have students whose perceptions of their own ability are notably more modest than students in lower scoring European societies such as England.
The wider environmental variables used in the study generally confirm the impression given by other studies, with Science achievement in England related to parental interest, the number of books in the home (as a surrogate for social class), the amount of homework undertaken and the amount of listening reported in Science lessons.
Table Twenty Three
Overall Average Percentage Correct Of All Participants (IAEPS2) 1990
Smaller Scale Studies
There are a number of further studies of educational achievement involving England that merit attention, mostly involving bilateral comparisons of England and one other country. Interestingly, they tend to support the principal findings of the major surveys of achievement noted above.
(i) The Exeter Kassel Project (Burghes and Blum, 1995) is an ongoing comparison of Mathematics achievement in England and Germany, involving a cohort of pupils aged 13 and their performance on number, algebra, shape and space, and the handling of data.
Initial scores of the cohort in England were slightly below those in Germany (44.3 compared to 47.3, with the maximum being 150), but by the end of year 2, German scores were 61.8 by comparison with 54.5 for England. Although the study has not used elaborate statistical methods so far, it is notable that there is considerable variation in achievement gain in differing English schools, and that the German increases in achievement have been uniform across the ability range, in apparent contrast to England.
Further analysis, the expansion of the study to more countries, and an experimental phase in which methods found to be successful in certain countries (Hungary in particular) are introduced into English schools, are all in progress.
(ii) The early National Institute for Economic and Social Research studies of Prais and Wagner (1985) looked at Mathematics in England, Holland and Germany. They found that some students in Dutch schools from the bottom third of the ability range were able to solve algebraic problems usually tackled only by first year A level students in England.
There were marked differences in the answers to arithmetic questions given by 15 year olds in the lower half of the ability range. Sixty-nine percent of German students, but only 13% of English students, answered some simple division correctly. In a separate study looking at the lower section of the academic attainment range in each country, average scores of 12.9 were recorded in Mathematics for secondary modern school students in England and 22.4 for comparable Hauptschule students in Germany (Prais and Wagner, 1985). In fact, the average German score of 22.4 exceeded the average score of all students in England of 20.1. Taking account of the presence of a higher proportion of high attaining children in England than in Germany (5% against 2% scoring 50 or more out of a possible 70), pupils in England had a 60% higher variability in attainment than pupils in Germany in terms of the 'coefficient of variation'.
(iii) The Bierhoff and Prais (1995) Re-analysis of IAEP Mathematics 2 shows that the overall score obtained by the median or 'middle' Swiss student was only obtained by the top quartile of English students. This suggests that there are three times as many low achieving students in England as in Switzerland. Further analyses of Realschule students (those at schools catering for the bottom third of academic attainment) show them achieving scores above those attained by the average pupil in all schools in England (a score of 57% compared with 51%).
(iv) The Burghes (1995) Study shows other countries performing much better than England; the samples of 14 year olds scored as follows (maximum score is 50):
Whilst these differences may reflect the influence of a variety of social and cultural factors as well as educational ones, the tendency for the gaps between countries to widen as cohorts pass through and as children get older, suggests the influence of within-system factors of some kind. Because of the small samples used and problems of representativeness, it would be unwise to place too great a reliance on these studies.
(v) Prais (1995) suggests that 'organisational aspects' of the British educational system disadvantage school-leavers of average and below average ability. In particular:
There are also a number of secondary analyses of IEA and IAEP data that should be noted, although they tend to confirm the findings of the initial analyses. The Coleman (1975) re-analysis of IEA data, for example, showed that England had the highest level of family background effects of all the societies looked at, a finding also confirmed by later IEA studies. Scheerens et al (1989) re-analysed SIMS data and, although England is not included in the sample, there are fascinating variations between countries in the factors associated with achievement (as outlined in the Introduction), and in the relative strength of the school level and the classroom level in explaining variation in student achievement scores (Scandinavian countries like Sweden have very high variation explained by classrooms rather than schools, perhaps reflecting considerable central intervention to ensure similarity of catchment areas).
There have been some further attempts to rate countries on aspects of the organisation of their educational systems, such as the extent to which decisions are decentralised. But in spite of the attractiveness of using factors that have been collected in IEA studies in creative, yet low cost ways, the resulting analyses have been inconclusive. As an example, the Meuret and Scheerens (1995) classification of educational systems according to the proportion of decisions in which the school was involved, pointed to New Zealand (> 70%) and Switzerland (20%) as the two 'extreme' countries. Yet both achieved the very highest reading test scores in the OECD/CERI Education at a Glance publication (1993).
The Case So Far
We noted in Section Two that there were built in problems with the kind of studies that we have been looking at - the problem of comparing countries on a 'common currency' and the problem of separating out educational influences from the other significant factors. The studies we have reviewed solved the first problem by using standard achievement tests. The second problem was considerably more difficult to deal with, since 'at a point in time' comparisons reflect the influences of multiple factors. There is no doubt that social, cultural, economic and familial factors in different countries are of major importance in explaining performance - indeed, it would be utterly contrary to all the findings of school effectiveness research if this were not the case (see reviews in Reynolds and Cuttance, 1992).
Nevertheless, it must be clear that educational influences do stand out in the findings reported. No wider societal or cultural factors could be responsible for the huge difference in achievement in England between pupils in the years of compulsory schooling (5 - 16) and pupils aged 16-19. The marked differences between societies in the sub-areas of Mathematics have no known non-educational causes. And Mathematics and Science are subjects on which wider, cultural influences are least marked.
It is clear, then, that the educational systems of different societies are key factors in determining their educational achievement. These studies are remarkably consistent in that respect. For England, the studies suggest that:
When all studies point in the same direction, it would in our view need rather more than the above list of caveats to persuade one that the English performance is anything other than poor.
Why Worlds Apart?
In this section, we assess the factors that explain country differences in general and the rather poor English performance in particular. This search for explanations is something that has received much less attention than the issue of the size of the differences in achievement between countries. Indeed, we have seen that the IEA and IAEP studies are notably opaque when it comes to explaining their findings - not surprising, given the data that it is possible to collect on educational processes in studies of this kind.
Our difficulty in explaining, rather than describing, the differences between countries is magnified by the frankly inept contribution which the comparative education discipline has made over time (see the review in Altbach, 1991). A range of theories have been put forward in this field, without any apparent empirical backing. There are a large number of descriptive case studies of individual schools or educational systems, which it is impossible to synchronise because there are no common measures of outcomes or processes; there are also a number of descriptions of the range of educational, political, economic and cultural phenomena within different countries, which make no attempt to assess the contribution of the educational system compared with other factors.
There are, however, a number of small-scale, usually bilateral, comparisons of England with other countries and we have added to them by means of a comprehensive review of the educational literature ourselves. We have also had access to much unpublished data from our ISERP study.
What follows now are some hypotheses as to:
It is widely agreed that there are a variety of factors responsible for the high achievement scores of Pacific Rim societies. Among the cultural factors suggested are (Stevenson and Stigler, 1992; Her Majesty's Inspectorate, 1992; Thomas and Postlethwaite, 1983):
Among the systemic factors thought to be important are:
Discussion has also centred on the possible systemic features responsible for the high achievement levels of certain European societies. In Germany and Holland (Smithers and Robinson, 1991; Prais and Wagner, 1965; Bierhoff, 1996):
(i) There are close links between school leaving awards and future job opportunities, with particularly the apprenticeships that are the gateway to skilled working class employment being dependent upon good performance in school;In Switzerland, the high mean scores of pupils and the low range of scores in Mathematics and Science are thought to be the product of (Bierhoff, 1996; Bierhoff and Prais, 1995):
(i) High proportions of lesson time (50 - 70%) being used for whole-class teaching. This is not simply of the 'lecture to the class' variety, but high quality interactive teaching in which the teacher starts with a problem and develops solutions and concepts through a series of graded questions addressed to the whole class. Pupils working on their own in groups are correspondingly much rarer than in England;
Systemic factors in Hungary include (Burghes, 1995):
(i) More formal classroom teaching, with more teacher direction, more whole-class interactive instruction and more open discussion of students' mistakes.Growing Apart? Some Speculations
So far we have noted a wide range of explanations as to how the educational systems of different societies may generate differences in levels of educational achievement. Cultures other than England may have a variety of factors in their educational systems which are not present in an English context.
But what are the processes that are present within an English context - not just the absence of effectiveness factors seen elsewhere - which may be responsible for the English performance? What does the English experience actually look like? We use data, much of it unpublished, from the British part of the ISERP study to speculate on this point (see Reynolds et al, 1994 for an outline of the study and Reynolds et al, 1996 in Appendix A for further details of the study findings).
The first thing that is clear from our data is that there is already a wide range of achievements at age 7 amongst English pupils when they begin junior school. England, in the ISERP data, has a standard deviation of 92 (on a mean of 504), whereas Taiwan, for example, has a standard deviation of 56 (on a higher mean of 629). The heterogeneity of English society and the variation in parental environments, pre-school experiences and in infant school quality are likely to be the key explanations for this.
This wide range of achievement may well be magnified by the nature of English classroom and school processes, that is:
A pupil population which has considerable variation in achievement interacts with an educational system that displays considerable variation in quality, reflecting:
The ISERP longitudinal data suggests that the English system may be responding to the range of initial achievement by differentiation practices that are a perfectly understandable method of handling range, but which may, in their recognition of range, make it even wider. Another way of handling range is of course to impact upon it directly by reducing it, for example by the direct targeting of the lower achieving children for extra attention, and by 'holding pupils down a year' and 'putting them up a year', both of which actions considerably truncate range. Many European societies indeed combine action on range, with a reluctance to differentiate between and within classes. In Switzerland, for example, a differentiated secondary system at age 13 means that only 18% of pupils are grouped by ability in Mathematics between classes, and 19% of pupils within classes. By comparison, the English figures are 92% and 32% respectively.
Indeed, much of Europe now possesses a predominantly mixed ability and undifferentiated primary education system, where pupils repeat years if necessary, followed by a differentiated secondary system. This is not the English experience.
The Next Steps
Customarily, educationists have been highly ethnocentric in their knowledge bases of effective practices, although things are clearly changing as we noted at the beginning of this review. Our conclusion is that English educationists now need to look beyond their own geographical boundaries to see why it is that other countries, in particular those of the Pacific Rim and successful European countries such as Switzerland, may be doing better than we are.
To look at other, non-English contexts and assess which of their practices may be useful here is of course a slightly risky enterprise, intellectually and practically. Factors that work within one context may not work within another, or at least may not work as productively. Indeed, factors that are associated with success in a Pacific Rim culture which celebrates a very different view of the nature of humankind, and a very different view of the proper relationship between an individual and the collectivity, may need careful evaluation before they are adopted in schools of a different culture.
However, we would argue that the situation in which England finds itself is now so worrying, that the risk involved in looking outward and trying new practices is worth taking. Indeed, limited experimentation with non-British practice seems positively overdue. When such experiments have taken place within non-educational sectors of society - as with the British motor industry's use of a blend of British and Japanese practice - they have been productive for the professionals concerned and for the wider society. Variations in cultural context and traditions have never prevented management in any area from trying out ideas or reforms that have been introduced abroad, monitoring their effectiveness and then dispensing with them if they do not improve the situation.
We would suggest that educationists in England behave as we would urge our children to do. That is to look beyond the immediate restriction of tradition and geography and use an open mind to see if other countries have ideas and practices which we can adapt to our own system. The way to cease being 'Worlds Apart' is surely to adopt an open mind.
1. International Comparisons and Education Reform (1989) by A. Purves and published by ASCD in the United States is an excellent guide to, and summary of, many of the studies reviewed here.
2. D. Reynolds et al (1994) Advances in School Effectiveness Research and Practice (Oxford: Pergamon) is an introduction to the field of educational effectiveness, and is international in scope.
3. H. Bierhoff and S. J. Prais (1995) Schooling As Preparation for Life and Work in Switzerland and Britain (London: NIESR) is a small scale study of England and of Europe's highest scoring country on Mathematics, Switzerland. It is available (price £3.00) from the National Institute for Economic and Social Research, 2 Dean Trench Street, Smith Square, London, SW1P 3HE.
4. The IAEPM 2 Study Learning Mathematics and Science (1992) by D. Foxman, is available (price £10.00) from the National Foundation for Educational Research, The Mere, Upton Park, Slough, Berkshire, SL1 2DQ. The NFER also publish a free leaflet on The Third International Mathematics and Science Study (TlMSS), available from the same address.
5. S. J. Prais (1995) Productivity, Education and Training: An International Perspective (Cambridge University Press) provides a comprehensive and comprehensible summary of the detailed work of Prais and associates over we last decade.
6. Findings from the ISERP study are available in D. Reynolds, B.P.M. Creemers, S. Stringfield and C. Teddlie (1996) The International School Effectiveness Research Project: Some Further Findings (a paper presented to the American Educational Research Association, April 1996), which is available free of charge from David Reynolds, Department of Education, St Thomas Street, Newcastle upon Tyne, NE1 7RU.
7. Reviews of the methodological adequacy of the material reviewed here can be found in a special issue of the Comparative Education Review, volume 31, number 1 (February 1987).
A number of possible avenues of research have been suggested by our review of existing knowledge:
(i) England has, to our knowledge, undertaken no secondary analyses of international datasets, nor of the English/Welsh specific country data, in marked contrast to the situation in other countries (see for The Netherlands the re-analyses of IEA data by Scheerens et al, 1989). The imminent publication of the TlMSS results, and the availability of that data, creates an opportunity for such low cost work.
Altbach, P. (1991) 'Trends in Comparative Education' in Comparative Education Review, August, pp 491-507.
Anderson, L. W. Ryan, D. W. and Shapiro, B. J. (1989) The IEA Classroom Environment Study. Oxford: Pergamon Press.
Ball, S. J. (1990) Politics and Policymaking in Education. London: Routledge and Kegan Paul.
Bierhoff, H. (1996) Laying the Foundation of Numeracy: A Comparison of Primary School Textbooks in Britain, Germany and Switzerland. London: National institute for Economic and Social Research.
Bierhoff, H. J. and Prais, S. J. (1995) Schooling As Preparation for Life and Work in Switzerland and Britain. London: National Institute for Economic and Social Research.
Bronfenbrenner, U. (1972) 'Another World of Children' in New Society, 10th February, pp 278-286.
Burghes, D. and Blum, W. (1995) 'The Exeter Kassel Comparative Project: A Review of Year 1 and Year 2 Results' in Gatsby Foundation, Proceedings of a Seminar on Mathematics Education. London: Gatsby.
Burghes, D. (1995) 'Britain Gets a Minus in Maths', Sunday Times, 14th May, p 11.
Cheng, Y. C. (1993) 'The theory of the characteristics of school-based management' in International Journal of Educational Management, Vol. 7, No. 6, pp 6-17.
Coleman, J. S. (1975) 'Methods and Results in the IEA Studies of Effects of School on Learning' in Review of Educational Research, Vol. 45, No. 3, pp 335-386.
Comber, L. C. and Keeves, P. (1973) Science Education in Nineteen Countries. London: John Wiley.
Creemers, B. (1994) The Effective Classroom. London: Cassell.
Creemers, B. and Scheerens, J. (Eds) (1989) 'Developments in school effectiveness research', a special issue of International Journal of Educational Research, Vol. 13, No. 7, pp 685-825.
Edmonds, R. R. (1979) 'Effective schools for the urban poor' in Educational Leadership, Vol. 37, No. 15-18, pp 20-24.
Fogelman, K. (1978) 'The Effectiveness of Schooling' in Armytage, W. H. G. and Peel, J. (Eds). Perimeters of Social Repair. London: Academic Press.
Foshay, A. W. (1962) Educational Achievements of Thirteen Tear Olds in Twelve Countries. Hamburg: UNESCO.
Foxman, D. (1992) Learning Mathematics and Science (The Second International Assessment of Educational Progress in England) Slough: National Foundation for Educational Research.
Fuller, B. (1987) 'School effects in the Third World' in Review of Educational Research, Vol. 57, No. 3, pp 255-292.
Garden, R. A. (1987) 'The Second IEA Mathematics Study' in Comparative Education Review, Vol. 31, No. 1, pp 47-68.
Goldstein, H. (1989) Multilevel Modelling in Large Scale Achievement Surveys. Paper presented to the American Educational Research Association, San Francisco, March.
Goldstein, H. (1993) Interpreting International Comparisons of Student Achievement. Paris: UNESCO.
Goodman, R. (1995) 'Chasing Illusions: the Real Lessons From Japanese Schools' in Demos Quarterly, Vol. 6, pp 37-8.
Hallinger, P. and Murphy, J. (1986) 'The social context of effective schools' in American Journal of Education, Vol. 94, pp 328-355.
Hargreaves, A. and Reynolds, D. (Eds) (1989) Educational Policy: Controversies and Critiques. Lewes: Falmer Press.
Her Majesty's Inspectorate, (1992) Teaching and Learning in Japanese Elementary Schools. London: HMSO.
Howarth, M. (1991) Britain's Educational Reform: A Comparison with Japan. London: Routledge/Nissan institute.
Husen, T. (Ed) (1967) International Study of Achievements in Mathematics, Volumes One and Two. Stockholm: Almquist and Wiksell.
Keeves, J. P. (1992) The IEA Study of Science III: Changes in Science Education and Achievement, 1970 to 1984. Oxford: Pergamon Press.
Keys, W. and Foxman, D. (1989) A World of Differences (A United Kingdom Perspective On An International Assessment of Mathematics and Science). Slough: National Foundation for Educational Research.
Lapointe, A. E., Mead, N. and Phillips, G. (1989) A World of Differences: An International Assessment of Mathematics and Science. New Jersey: Educational Testing Services.
Levine, D. and Lezotte, L. (1990) Unusually Effective Schools: A Review and Analysis of Research and Practice. Madison: NCESRD Publications.
Lezotte, L. (1989) 'School Improvement based on the Effective Schools Research' in International Journal of Educational Research, Vol. 13, No. 7, pp 815-25.
Lynn, R. (1988) Educational Achievement in Japan: Lessons for the West. London: Macmillan/Social Affairs Unit.
Meuret, D. and Scheerens, J. (1995) An International Comparison of Functional and Territorial Decentralisation of Public Educational Systems. Twente: University of Twente.
Mislevy, R. J. (1995) 'What can we learn from International Assessments?' Educational Evaluation and Policy Analysis, Vol. 17, No. 4, pp 419-437.
National Commission on Education (1993) Learning to Succeed. London: Heinemann.
Pelgrum, W. J. and Plomp, T. (1993) The IEA Study of Computers in Education: Implementation of An Innovation In 21 Education Systems. Oxford: Pergamon Press.
Postlethwaite, T. M. and Wiley, D. E. (1992) The IEA Study of Science II, Science Achievement in Twenty Three Countries. Oxford: Pergamon Press.
Prais, S. J. (1995) Productivity, Education and Training: An International Perspective. Cambridge: Cambridge University Press.
Prais, S. J. (1994) 'Economic Performance and Education: The Nature of Britain's Deficiencies' Proceedings of the British Academy, Vol. 84, pp 151-207.
Prais, S. J. and Wagner, K. (1965) 'Schooling standards in England and Germany: some summary comparisons based on economic performance' in Compare, Vol. 16, pp 5-36.
Purves, A. C. (1992) The IEA Study of Written Composition II: Education and Performance in Fourteen Countries. Oxford: Pergamon Press.
Reynolds, D. and Cuttance, P. (1992) School Effectiveness: Research, Policy and Practice. London: Cassell.
Reynolds, D. Sullivan, M. and Murgatroyd, S. J. (1987) The Comprehensive Experiment. Lewes: Palmer Press.
Reynolds, D. Creemers, B. P. M. Stringfield, S., Teddlie, C., Schaffer, E. and Nesselrodt, P. (1994) Advances in School Effectiveness Research and Practice. Oxford: Pergamon Press.
Robitaille, D. F. and Garden, R. A. (1989) The IEA Study of Mathematics II: Contexts and Outcomes of School Mathematics. Oxford: Pergamon Press.
Rosier, M. J. and Keeves, J. P. (1991) The IEA Study of Science I - Science Education and Curricula in Twenty Three Countries. Oxford: Pergamon Press.
Scheerens, J. Vermeulen, C. J. and Pelgrum, W. J. (1989) 'Generalisability of instructional and school effectiveness indicators across nations' in International Journal of Educational Research, Vol. 13, No. 7, pp 789-799.
Smithers, A. and Robinson, P. (1981) Beyond Compulsory Schooling: A Numerical Picture. London: Council for Industry and Higher Education.
Stevenson, H. (1992) 'Learning from Asian Schools' in Scientific American, December, pp 32-38.
Stevenson, H. W. and Stigler, J. W. (1992) The Learning Gap: Why Our Schools Are Failing and What We Can Learn from Japanese and Chinese Education. New York: Summit Books.
Teddlie, C. and Stringfield, S. (1993) Schools Make a Difference: Lessons learned from a ten year study of school effects. New York: Teachers College Press.
Thomas, R. and Postlethwaite, N. (1983) Schooling in East Asia: Forms of Change. Oxford: Pergamon Press.
Travers, K. J. and Westbury, 1. (1989) The IEA Study of Mathematics I: Analysis of Mathematics Curricula. Oxford: Pergamon Press.
Van de Grift, W. (1990) 'Educational leadership and academic achievement in secondary education' in School Effectiveness and School Improvement, Vol. 1, No. 1, pp 26-41.
Walberg, H. J. (1991) 'Improving School Science in Advanced and Developing Countries' in Review of Educational Research, Vol. 61, No. 1, pp 25-69.