OVERVIEW

The data on which this research is derived comprises:

• the 107 school research reports;
• a survey completed by 3-4 staff in each project school; and
• follow-up visits to selected `case study' schools.

The authors of individual chapters derived their findings from either the complete data set of school research reports or from the subset of research reports, survey responses and case study schools relevant to a specific chapter focus.

The approach taken to research for individual chapters, is described below.

CHAPTER 2: EARLY LITERACY

This chapter examined the experiences of those schools in the IBPP that identified early literacy as their focus. Of the 107 schools, 23 nominated English or literacy as their focus, 16 of which were primary schools. Among these 16 schools, four had a programme that was operating across all of the primary years of schooling. The remaining 12 had an early years focus. This chapter seeks to answer the following questions:

• What kinds of programmes have the `literacy' schools implemented?
• What has been the motivation behind these programmes?
• What strategies have these schools employed to bring about change, and what factors have influenced successful change?
• What is the nature/quality of the evidence of success in these schools?

The evidence drawn upon to answer these questions comes from three sources, namely:

• the project reports of the 16 schools;
• responses of key staff in each of the 16 schools to a survey that was completed by all schools participating in the IBPP; and
• follow-up in-depth interviews undertaken in eight of the 16 early literacy schools judged to have achieved significantly improved outcomes.

CHAPTER 3: MATHEMATICS

The initial data source for this chapter was the research reports for the 9 schools that had mathematics as the major focus of their innovations. An additional 5 schools included mathematics within a broader project focus and their research reports provided contextual information. The survey data responses of the mathematics schools were also used for this chapter.

Additional data were collected from visits to six case study schools, three primary and three secondary schools. Interviews were conducted with key project personnel in these schools and meetings were held with groups of students.

The chapter analysed and Synthesised the 9 research reports, elaborated and extended by evidence gained from the visits and survey undertaken across all IBPP schools. Analysis was also made of pertinent references to mathematics in the school research reports of projects where mathematics was not the primary focus of the innovation.

CHAPTER 4: INFORMATION AND COMMUNICATIONS TECHNOLOGIES

The findings of the chapter were based on analyses of the research reports of the 20 schools that focused their innovations on ICT. The analysis of the school research reports focused on:

• descriptions of the aims of the projects and the strategies employed in their implementation;
• evidence of change to school practice and the sequences of change;
• attributions of obstacles to change; and
• evidence about the impact of the innovations on teaching and learning.

The data collection and analysis strategies that schools employed were also comprehensive, drawing from: surveys, interviews, focus groups, observation schedules, computer system logs, achievement tests and public examination results, teacher/student journals, computer maintenance records, standardised assessment inventories, evaluations of student work, student/teacher self- evaluations, document analysis, and needs analysis. Schools made attempts to triangulate the different forms of data they had collected and used case studies, structured vignettes, written testimonials, etc, to provide depth to their analyses.

In addition to completing a school research report, all schools responded to a survey completed by 3-4 staff members involved in the IBPP. Although anonymous, these responses were identified by a school code.

This report reflects the structure of the reports from the individual schools. It is principally a meta-analysis and Synthesis of the 20 school research reports from the technology-based innovations, elaborated and extended by evidence gained from visits to a sample of schools and the survey undertaken across all IBPP schools.

CHAPTER 5: MIDDLE-YEARS

Of the 107 IBPP schools, 33 nominated engagement of students in the middle- years as their key focus. This chapter focuses on the programmes in these 33 schools and seeks to answer the following questions:

• What kinds of programmes have the middle-years schools implemented?
• What has been the motivation behind these programmes?
• What strategies have these schools employed to bring about change and what factors have influenced successful change?
• What is the nature and quality of the evidence of success in these schools?

The evidence drawn upon to answer these questions comes from three main sources, namely the project reports of the 33 schools and the responses of key staff in each of the schools to a survey that was completed by all schools participating in the IBPP. In addition, visits to all schools took place. Further follow-up visits were made to a sample of schools judged to have achieved significantly improved outcomes. Interviews were carried out with relevant staff in those schools.

CHAPTER 6: LEADERSHIP

The research for this chapter incorporated a range of empirical and qualitative research procedures. Following analysis of the results of a comprehensive survey of IBPP participants to ascertain factors associated with the implementation of school-based initiatives, on-site focus group discussions were conducted with a range of participants in nine case study schools. The data collection and analysis process involved pairs of external researchers working with school representatives in validating data, forming concepts and generating theories.

Selection of case study schools

Schools were selected on the basis of documented improvement in student achievement in the IBPP. Five schools in the sample can be regarded as having provided substantive evidence of improved student achievement (Sunbeach Primary School, Sacred Mission School, West Town Secondary School, Christafam Secondary School, Potter's Gold Senior Secondary School) and four as having provided authoritative perceptual evidence (Worthwood Secondary School, Mt Ritchy Primary School, Holy Spirit Catholic College, Wave Secondary School). Case study schools were also selected to ensure representation from different States (Queensland, New South Wales, and Victoria) and a balance of State/Catholic/Independent systems, secondary and primary levels and urban/rural locations.

Data collection and analysis procedures

Focus group discussions took place in each case study school and followed a four-phase questioning strategy based on Padilla's `unfolding matrix' (1996). The central questions for the four phases were:

• What is the nature of the documented improvement in student achievement?
• What factors are perceived to be important in having facilitated the improvement in question?
• What roles did key actors play in the operationalisation of these factors?
• What leadership processes and concepts appear to underpin the processes and actions that have been identified?

Representative school groups (usually about six individuals, comprising the Project coordinator, principal and teachers involved in the innovation) responded to the first three questions through a range of individual reflection and interactive analysis. The fourth question was addressed differently. That is, the external researchers presented a series of propositions that they considered to capture the leadership implications of the dialogue that they had heard. The school cohorts then critiqued these propositions and developed from them a statement that they believed reflected the forms of leadership that underpinned their innovation.

Data analysis procedures varied in each State. In Queensland, a workshop for participants from the three schools was conducted over a weekend retreat; in New South Wales, the research team conducted an extended workshop with the State IBPP coordinator (who had participated in the visits to schools in that State) to Synthesise the outcomes of the four school visits; and in Victoria, the three- person research team convened a one-day workshop to Synthesise their individual school research reports. At the completion of this process, the research team met for two days to draw major conclusions. This process enabled generalisations to be identified for each case study, as well as across the research sample. A copy of the full draft chapter was circulated to the nine case study schools for validation.

Limitations of the Research Methodology.

The research into leadership in IBPP schools is characterised by a number of limitations.

First, while every effort was made to select schools that provided documented evidence of enhanced student achievement as a result of a school-based innovation, the consistency of the evidence provided varies across the sample. Most importantly, in five of the case studies the evidence that was provided was in the form of validated pre-post empirical data relating to student outcomes, but in the remaining four cases it was based largely in perceptions.

Second, the nine case studies that provided the basis for the research were self- identified innovative schools and in that sense must be viewed as atypical.

Third, with particular reference to the factors that contributed to the success of innovations in the case study schools, the data that were obtained relating to elements of the Research-based Framework for Enhancing School Outcomes (Leadership Research Institute, 1999) were broad in nature. While it is believed that these data were sufficiently comprehensive to enable confident assertions to be made about the importance of holistic approaches to school reform, they were not sufficiently detailed to enable essential features of the elements in question to be fully uncovered. Thus, for example, the dynamics of processes of School-wide Pedagogy are not considered in detail in this study.

Fourth, the concept of `School-wide Pedagogy' features strongly in the conclusions drawn from this research. While some aspects of pedagogy in all nine schools were indeed characterised by school-wide approaches to teaching and learning (for example whole-school literacy) most of the reforms did not involve all teachers or all pedagogical areas. Thus, the full complexity of the concept of school-wide may not be addressed in this study.

CHAPTER 7: INSTRUCTIONAL FLEXIBILITY

The research for this chapter is based on visits to 26 schools in six states and one territory. Four case study schools were selected as being highly flexible on the basis of information provided in school research reports. Visits to these four schools provided a basis for validating the full database of the 26 school research reports. Staff members actively involved in the innovation were shown a data summary sheet relating to their school and asked to comment on the appropriateness of the categories and the accuracy of the data recorded. In all four cases, more detailed information was provided in taped interviews that were transcribed. Excerpts from these interviews are included in this report.

CHAPTER 8: MANAGING INNOVATION

The principal data source is the full set of reports submitted by the 107 schools participating in the IBPP. All 107 school research reports provide accounts of effort directed at improvement. One strength of such a data source is that each school has chosen the events and actions considered by participants to be most significant. All schools had been provided with information from the IBPP team to indicate that autonomy was a matter of particular interest. However, the data about autonomy (and the flexibility that it allowed) was embedded in the reports in several layers of information about intentions, actions and evaluations regarding the innovations.

The basic questions that informed a content analysis of the school research reports focused on:

• evidence of change to school practice and the sequences of change;
• evidence that a school had used data in its decision making about change;
• evidence about the role of market pressure and competition on school innovation; and
• attributions of obstacles to change.

Once available, the full set of reports was read and a coding schedule was devised to assist with the quantification of information relevant to these questions. Relevant data in the reports were then coded and the results entered onto an electronic database. The project survey provided a valuable data source. The responses to relevant items were matched to the coding of the school research reports.

Visits to collect additional data were made to 26 schools in six states and one territory. Because of the significance of state contexts it was considered necessary to validate analyses found in school research reports in as many contexts as possible.