3.4 Speaking

The 1996 National School English Literacy Survey included an assessment of the speaking achievements of Year 3 and Year 5 students.

What Was Assessed?

In assessing levels of speaking achievement, special attention was paid to students' abilities to:

Within this broad framework, speaking assessment tasks were developed with reference to the outcomes of the English profile. Assessment procedures were designed to address a range of relevant profile outcomes in speaking (see pages 256 and 258-259).

How Was Speaking Assessed?

Students' levels of speaking achievement were assessed by having them complete a set of speaking tasks under controlled conditions ('common tasks') and by collecting records of speaking performances in the classroom within specified categories ('best work'). Classroom teachers made judgements about the quality of their students' speaking using provided assessment guides.

COMMON TASK

The speaking common tasks completed by students included:

Year 3 students retold their favourite narrative, and reviewed a character from the provided videotape. Year 5 students talked about their favourite TV show and discussed a poem in small groups in preparation for individual presentation and commentary.

Individual presentations required students to consider the ways in which spoken text is used to communicate meaning through:

Teachers made on-the-spot judgements of students' common task performances. Both Year 3 and Year 5 students completed two common tasks in speaking.

BEST WORK

Students' best work in speaking was assembled by teachers in three specified categories. Teachers were asked to base their assessments on two speaking performances/ presentations:

Teachers were asked to provide records made at the time of these student presentations or, if available, video or audio recordings of the presentations. Teachers made assessments of students' speaking performances by reviewing their written records, or the video or audio recordings. Two samples of speaking work were collected for Year 3 and Year 5 students.

The Speaking Scale

Students' performances on the National School English Literacy Survey tasks were used to construct a scale of increasing achievement in speaking. This scale is shown in Figure 3.24. The construction of the scale is described on pages 292-295.

The speaking scale is empirically based: in other words, it is based on an analysis of observed student performances in the Survey. The speaking indicators (descriptions of observed speaking behaviours) in Figure 3.24 are derived from the Survey tasks. Indicators which relate to performance elements of students' presentations are italicised.

The cluster of indicators lowest on the scale, including

expresses ideas simply and conveys limited meaning
presents a disjointed or incomplete story
shows limited understanding of the need to communicate with an audience
speaks inaudibly at times

describes the lowest level of achievement in speaking in the Survey.

The cluster of indicators highest on the scale, including

presents a well-reasoned account
displays a sense of key issues
consistently enhances presentation with relevant detail
begins to engage audience through language, gesture, tone

describes the highest level of achievement in speaking in the Survey.

On the right of Figure 3.24 an attempt has been made to show the approximate relationship between these clusters of indicators and the Speaking levels of the English profile.

At about Level 1, students express ideas simply, showing an emerging awareness of school purposes and expectations for spoken language.

At about Level 2, students speak with more confidence in structured school situations, showing a basic understanding of the demands of a formal speaking task.

At about Level 3, students begin to adapt spoken language to suit a formal context. They begin to organise their presentations and to consider audience response.

At about Level 4, students are aware of the ways in which speech can entertain, inform and influence others. They plan their presentations and show a consistent sense of audience.

At about Level 5, students effectively use appropriate language and organisational elements in formal presentations, and begin to experiment with ways to influence audiences.

Figure 3.24 Indicators on the Survey speaking scale

The Speaking Scale -- Examples

To illustrate the five levels on the Survey speaking scale, a number of samples of teacher records have been selected and are reproduced below. Because students' presentations were assessed separately for content and performance, some presentations were assessed at different levels on these two features (eg Level 3 for content, but Level 2 for performance). For purposes of illustration, the samples shown below are drawn from presentations which were assessed at the same Level on content and performance.

LEVEL 1 SPEAKING

The lowest rating assigned to student speaking was Level 1.  An example of a teacher's written record of a Level 1 speaking performance is shown below. This record suggests that the student's performance was brief, with ideas expressed simply. There was little understanding of the need to communicate with the audience.

Content: Level 1
Performance: Level 1

Teachers assigned a rating of Level 1 to about 6% of Year 3 students' common task speaking performances (see pages 277-279).

LEVEL 2 SPEAKING

An example of a teacher's written record of a Level 2 speaking performance is shown below. This record suggests that the presentation showed a basic understanding of the task, included key information, but was not well organised. There was little sense of addressing the audience. 

Content: Level 2
Performance: Level 2

Teachers assigned a rating of Level 2 to approximately 31% of Year 3 speaking performances and 14% of Year 5 speaking performances (see pages 277-279).

LEVEL 3 SPEAKING

An example of a teacher's written record of a Level 3 speaking performance is shown below. This record suggests that the student's presentation showed evidence of organisation (eg written text and props). The speaker showed a clear sense of audience most of the time.

Content: Level 3
Performance: Level 3

Teachers assigned a rating of Level 3 to approximately 46% of Year 3 students' spoken presentations, and 43- 48% of Year 5 students' speaking performances (see pages 277-279).

LEVEL 4 SPEAKING

An example of a teacher's written record of a Level 4 speaking performance is shown below. This record suggests that the student's performance was complete and organised, and the student showed a good, consistent sense of audience.

Content: Level 4
Performance: Level 4

Teachers assigned a rating of Level 4 to approximately 17% of Year 3 students' speaking performances and 32% of Year 5 students' speaking performances (see pages 277-279).

LEVEL 5 SPEAKING

The highest rating assigned to student speaking was Level 5.  An example of a teacher's written record of a Level 5 speaking performance is shown below. This record suggests that the student's performance was well organised, effectively used appropriate language, explored issues in detail, and engaged the audience through language, gesture, and tone.

Content: Level 5
Performance: Level 5

Teachers assigned a rating of Level 5 to approximately 6-9% of Year 5 students' speaking performances (see pages 278-279).

Measuring Speaking Achievement

Each student's performances on the common speaking tasks and classroom speaking presentations were used to construct a measure of that student's speaking achievement. Measures of speaking achievement were constructed in three steps. First, each student's ratings (both content and performance) on the two common speaking tasks were combined to provide a measure of the student's speaking achievement under controlled (test) conditions. Second, the student's ratings (both content and performance) on the two classroom presentations were combined to provide a measure of the student's speaking achievement under typical classroom conditions. Third, ratings on all four performances/presentations were combined to produce a 'combined' speaking measure for the student.

Figure 3.25 summarises the distributions of Year 3 and Year 5 students' speaking measures based on their common speaking task performances and on their classroom work. The distributions of speaking measures are shown here as 'box and whisker' plots which indicate the levels on the speaking scale achieved by 10%, 20%, 50%, 80% and 90% of students in each Year group. The number inside each box is the median of the distribution.

From Figure 3.25 it can be seen that the distributions of students' levels of speaking achievement are very similar, whether based on common speaking tasks or classroom work. This finding is different from the result in writing, where students performed better on classroom work than on the common tasks. The reason for this difference is probably that, in speaking, the two sets of speaking performances are completed under similar conditions and may not reflect the influence of drafting, re-drafting and conferencing.

Figure 3.26 shows the distributions of students' speaking achievement measures based on their two common speaking tasks and two classroom presentations combined. In constructing these combined measures, the common task and classroom work samples were weighted equally.

For the purposes of establishing baseline measures of Year 3 and Year 5 writing achievement against which performances in future English literacy surveys might be compared, the estimated Year 3 and Year 5 population means and standard deviations have been calculated and are reported in Table 3.18. The sampling standard errors of the means are shown. The estimated population means and standard deviations shown here are based on common task speaking only and reflect the decision to report speaking measures on a scale defined by setting the Year 3 mean at 300 and standard deviation at 100. The numbers of students on which these figures are based are shown in the left- hand column.

From Table 3.18 it can be seen that the difference between the average speaking achievement at Year 5 and the average speaking achievement at Year 3 is 75 points on the Survey speaking scale or 0.75 of the standard deviation at Year 3. From Figure 3.26 it can be seen that this growth from Year 3 to Year 5 is considerably less than a full profile level.

Table 3.18 Speaking means and standard deviations

Figure 3.25 Distributions of students' estimated speaking achievements

Figure 3.26 Distributions of combined speaking achievement estimates

Subgroup Performances

Figures 3.27 to 3.29 show the distributions of students' estimated speaking achievements for three subgroups of the population: male/female; English/other than English language background; and low, medium and high socio-economic background.

Figure 3.27 shows that Year 3 females are, on average, achieving at a higher level in speaking than Year 3 males. The median estimate for Year 3 females is 301 on the speaking achievement scale and the median estimate for Year 3 males, 267. This pattern is repeated at Year 5, where the median estimated achievement levels are 372 and 341 respectively.

There is no change in the relative speaking achievements of males and females between Year 3 and Year 5: females achieve about 30 points (0.3 of a standard deviation for all students) higher than males at both Year levels.

Figure 3.28 shows that students from English-speaking backgrounds are, on average, achieving at higher levels in speaking than students from homes in which the main language spoken is a language other than English.

The median speaking achievement estimate for Year 3 students from English-speaking backgrounds is 287, and the estimate for students from language backgrounds other than English, 256. This pattern is repeated at Year 5, where the median speaking achievement of students from English-speaking backgrounds is 357, and from language backgrounds other than English, 340. The difference between the median speaking achievements of these two groups of students decreases slightly from Year 3 to Year 5.

Figure 3.29 shows that students of high socio-economic status (SES) at each Year level are, on average, achieving at a higher level in speaking than students of medium SES who, in turn, are achieving at a higher level in speaking than students of low SES. Students in the high SES group are children of parents in upper professional and managerial occupations; students in the medium SES group, children of parents in clerical and skilled manual occupations; and students in the low SES group, children of parents in unskilled manual occupations.

The median speaking achievement of Year 3 students from high socio-economic backgrounds is 319, from medium SES backgrounds, 284; and from low SES backgrounds, 254. This pattern is repeated at Year 5, where the median speaking achievement estimates are 399, 348, and 337 for students from high, medium, and low SES backgrounds.

An interesting feature of Figure 3.29 is the similar achievement of Year 3 students from high SES and Year 5 students from low SES. The median speaking achievement of Year 3 students from high SES is only slightly lower than the median speaking achievement of Year 5 students from low SES.

A second interesting feature is the range of achievements for the Year 5 students from low SES. The least able students (below the 10th percentile) in this group are achieving well below the median speaking achievement of Year 3 students; the most able (above the 90th percentile) are achieving well above the median speaking achievement of high SES Year 5 students.

Figure 3.27 Distributions of male and female students' estimated speaking achievements

Figure 3.28 Distributions of English-speaking background and other than English-speaking background students' estimated speaking achievements

Figure 3.29 Distributions of high, medium and low socio-economic status students' estimated speaking achievements

Special Indigenous Sample

Students in the Special Indigenous Sample also completed the Survey speaking tasks. The performances of these students were used to estimate their levels of achievement on the speaking scale.

For each student participating in the Survey, an on-balance estimate of the student's level of achievement on the speaking scale was made from that student's performances on the two common speaking tasks. A second estimate for each student was made from the student's performances of the two best work speaking samples. Figure 3.30 shows the distributions of these two sets of estimates for both Year 3 and Year 5 students.

In interpreting the speaking performances of the Special Indigenous Sample, it must be remembered that this was not a representative sample of all Aboriginal and Torres Strait Islander students in Years 3 and 5, but a sample of students in schools reporting five or more Indigenous students at both Year 3 and Year 5.

Figure 3.30 shows that most Year 3 students in the Special Indigenous Sample are estimated to be working in profile Levels 1 and 2 in speaking; and most Year 5 students, in profile Levels 2 and 3. There is a greater spread of speaking achievements in the Special Indigenous Sample at both Years 3 and 5 than in the main sample.

An interesting feature of Figure 3.30 is the difference between the performances of the lower achieving students on common task and best work speaking.  In general, students in the Special Indigenous Sample perform better on classroom speaking tasks than on the common tasks developed for the Survey. Lower-achieving students, in particular, perform better on classroom-based work. One explanation may be that the speaking common tasks used in the Survey, despite their careful selection, were beyond the experiences of many students in the Special Indigenous Sample.

The mean and standard deviation of Special Indigenous Sample students' estimated levels of achievement on the speaking scale based on their common task performances are shown in Table 3.19. Notice that the large sampling error on the mean is the result of the relatively small number of students (left-hand column) in this sample.

Table 3.19 Special Indigenous Sample students' Speaking means and standard deviations

Figure 3.30 Distributions of estimated speaking achievements for the Special Indigenous Sample

Percentages Working in Profile Levels

Because the common speaking tasks and classroom best work samples in speaking were designed to provide information about the outcomes of the English profile for Australian schools, and because the content and performance indicators on the Survey speaking scale are based directly on these outcomes, it has been possible to 'map' the levels of the English profile on to the Survey speaking scale (as shown in Figure 3.24). Level 1 on the Survey scale corresponds to profile Level 1; Level 2, to profile Level 2; and so on.

This direct mapping of profile levels on to the speaking scale has made it possible to estimate the percentages of Year 3 and Year 5 students working in each profile level based on students' common task speaking and also on their classroom best work samples. These percentages are shown in Tables 3.20 and 3.21.

Table 3.20 Percentage of Year 3 students working in each profile level

Table 3.21 Percentage of Year 5 students working in each profile level

To make these estimates, it was necessary to decide what it meant for a student to be 'working in' a profile level. The rule applied here was a 50% rule. For example, a student whose common task speaking assessments were 2, 2, 3, 3 was considered to be 'working in' Level 3 because at least 50% of that student's assessments were at Level 3. (A student with the assessments 1, 2, 3, 4 would also have been considered, on balance, to be working in Level 3 because they achieved the same speaking score, 10, as a student with assessments 2, 2, 3, 3.)

The percentages of Year 3 and Year 5 students in the Special Indigenous Sample working at each profile level in speaking are shown in Tables 3.22 and 3.23.

Table 3.22 Percentage of Year 3 Special Indigenous Sample students working in each profile level

Table 3.23 Percentage of Year 5 Special Indigenous Sample students working in each profile level

Copyright © Commonwealth of Australia
DEST Web Site Privacy Statement
Disclaimer