3. Characteristics of Graduates

[next chapter] [previous chapter] [contents]

Gender
Age
Non-English-speaking Background
Previous Qualification
Attendance
Subsequent Study
Subsequent Employment

The Analyses

The Good Teaching Scale (GTS)
The Clear Goals and Standards Scale (CGSS)
The Appropriate Assessment Scale (AAS)
The Appropriate Workload Scale (AWS)
The Generic Skills Scale (GSS)
The Overall Satisfaction Item (OSI)

Summary

One purpose of the CEQ is to facilitate comparison between the course evaluations of graduates of similar courses from different universities. As with any relationship based on data from a cross-sectional survey, it is possible that observed differences between universities are the result of extraneous factors. For instance, a course might have a high mean for the Good Teaching Scale (GTS) not because of the quality of the teaching, but because a high proportion of mature-age students are enrolled in the course, and mature-age students are more generous in their assessment of teaching. Where information on possible extraneous influences, such as the age of the respondents, is available empirical investigations of their effects can be undertaken. The present chapter considers the influence of the following characteristics of graduates on their CEQ scale scores:

• gender of graduate;
• age of graduate;
• whether or not the graduate was from a non-English-speaking background;
• whether or not the graduate had some prior educational qualification and, if so, the level of that qualification;
• whether the graduate attended full-time or part-time;
• whether six months after completion of the course, the respondent was undertaking further study; and
• whether six months after completion of the course, the respondent was employed.

The wording of the questions is shown in Figure 1.

The reason why variables of traditional educational interest such as prior ability and socioeconomic status are not included in this list is that relatively few background questions were included in the 1994 Graduate Destination Survey (GDS). This is understandable because the GDS focuses on the employment and study destinations of graduates, not possible extraneous influences on CEQ scale scores. Nonetheless, the seven variables selected for analysis are pertinent and capture, either directly or indirectly, many of the possible sources of variation in CEQ scale scores.

Gender

There is considerable evidence to indicate gender differences in attitudes to formal education (Yates 1994). For many years females have had a higher Year 12 completion rate than males (Australian Bureau of Statistics 1995), a higher rate of entry from Year 12 to higher education (Department of Employment, Education and Training 1990b) and a higher graduation rate from university (Long, Carpenter and Hayden 1995). Even if gender were not known to be related to attitudes to education and educational outcomes, it is such a fundamental attribute for many social processes that its omission would be difficult to justify.

Age

Even when the sample is restricted to bachelor degree courses, higher education student ages are widely dispersed. Students over the age of 25 years make up more than half of all students in bachelor courses (Department of Employment, Education and Training 1993). Mature-age students often have a broader set of experiences gained outside the education sector which may influence their evaluation of university courses.

Non-English-speaking Background

Students from non-English-speaking backgrounds are identified in many recent educational and labour market policy documents as disadvantaged, presumably because of difficulties associated with teaching and learning in a second language. Nevertheless, young people from non-English-speaking backgrounds have higher Year 12 completion rates and higher entry rates from Year 12 to higher education. These may result from more positive attitudes to education (Williams, Long, Carpenter and Hayden 1993a; 1993b). While there may be substantial variation among non-English-speaking groups, a measure based on English-speaking background is of possible importance when considering attitudes to higher education courses. However, it is readily acknowledged that a problem with using grouped data in this area is that the process of aggregation of diverse elements may produce a questionable result. For example, the 1995 group of non-overseas students of non-English-speaking background came from homes in which 91 different languages were spoken, and there were substantial differences in participation in higher education among these different ethnic groups (Department of Employment, Education and Training, 1996: 146-7).

Previous Qualification

The completion of post-secondary qualifications prior to undertaking the university course provides students with a broader educational experience on which to base their perceptions of the particular course of study on which they are reporting for the CEQ. Alternatively, students who started but did not finish a qualification prior to the completed course of study reported in the GDS may confound the data.

Attendance

The lower graduation rates of part-time and external students are well documented (Long 1994; Long, Carpenter and Hayden 1995). The very fact that enrolment in higher education is unlikely to be their principal activity might be expected to place a different perspective on their course. It is also likely to result in different expectations of lecturers and the university as course requirements have to be completed around other activities. In this double sense, then, mode of attendance is likely to affect the evaluation of the course.

Subsequent Study

Some graduates continuing with further study will be those who enrolled in an articulated double-degree structure, such as arts-law. Another group consists of those who continue with postgraduate study. These are, presumably, the best and brightest products of higher education, and their positive attitudes may influence their CEQ responses.

Students who continue with study are likely to be high achievers within the higher education sector, and those who are interested in study. The arguments canvassed in the previous chapter, about the effect of the level of attainment on course perceptions, are relevant here. There is, however, a problem about the direction of causation with this variable. Some component of any relationship between CEQ scale scores and subsequent study may be because positive experiences of learning in higher education lead graduates to further study rather than vice versa.

Subsequent Employment

It is conceivable that graduates who find themselves unemployed or under-employed at the time of the survey, some months after completion of their course, may not have a very positive attitude to that course, regardless of the quality of instruction of the course. While the quality of instruction is something for which, within limits, universities can be held responsible, employment outcomes are usually considered to be less the province of the universities themselves (although the likelihood of employment may be important information for an individual considering enrolment in that course). It is readily acknowledged that a variety of factors influence student opinions. However, in view of the limited background information available for analysis, the present investigation assumes that university graduates' are fair-minded enough to ensure that their opinions of their courses of study are not influenced by subsequent employment outcomes. This assumption is based on the results of previous work by Ainley and Long (1994) which demonstrated that the influences of graduates' employment or study outcomes on the CEQ scale scores were negligible.

Figure 1: Questions from the Graduate Destination Survey for Graduate
                Characteristics

Note: Questions have been extracted from the questionnaire out of order and context. The way in which responses have been grouped is detailed in the  notes to Table4.

Table 4: Standardised Regression Coefficients for Graduate-level Variables on Course
              Experience Scale Scores by Field of Study

See Notes to Tables

The Analyses

Identifying the relationships between these seven factors and the CEQ scales is a first step towards examining their effect on differences between universities. This is the purpose of  Table4. The table presents the standardised regression coefficients for each of the factors for each of ten fields of study for the five CEQ scales, and the Overall Satisfaction Item (OSI).

A standardised regression coefficient shows the effect in standard deviations on some dependent variable of a change in one standard deviation in another variable. For example, the very first value in the table, 0.05, is the standardised regression coefficient for age predicting scores for the GTS for graduates of accounting courses. The interpretation of this coefficient is that an increase in the age of graduates by one standard deviation corresponds to an increase in GTS scores of 0.05 standard deviations.

The coefficients shown in  Table4 are adjusted for the effects of other factors shown in the table. The age coefficient of 0.05 for accounting graduates shows the effect of age holding the effects of differences in gender, non-English-speaking background, previous qualifications, mode of attendance, and study and work destinations constant. Importantly, but not evident from the table, the effect of university attended is also held constant.

The use of standardised coefficients allows more meaningful comparisons between variables, although it loses a little in terms of interpretability. For the effect of changes in age on mean scores for a particular course, we can make the following kinds of conversions. Given that the standard deviation of the age of graduates is about five years and the standard deviation of the GTS for accounting students is just under 40 points, the average age of students in a course would need to increase by five years to increase the mean score by just two points. Although there are no definite rules when interpreting the size of standardised regression coefficients, 0.10 might be considered small and 0.05 about the minimum worthy of any comment at all. The values in  Table4 suggest that there are few large relationships to be found between any of these factors and the CEQ scales. Although, there are many small relationships that are statistically significant (indicated by a ** at 0.01 and a * at 0.05).

The import of statistical significance is a moot point in this study. The GDS falls short of being a population survey only by non-response. The result, however, is not a simple random sample as required by the inferential statistics on which estimates of statistical significance are based. The indications of statistical significance provide a 'what if' approach. If the survey participants were a simple random sample, then the corresponding values would be (or would not be) statistically significant.

The following discussion of  Table4 deals with one scale at a time, and considers the fields of study as replications of relationships. This may not always be appropriate because some fields of study display quite different relationships on occasion. When necessary this will be noted.

The Good Teaching Scale (GTS)

Age has a positive effect on the GTS. When other factors are held constant, the older a student, the more positive the perception of the quality of teaching. The strength of this effect varies from a value of 0.24 (one of the largest values in the table) for history to 0.00 for medicine. Several of the coefficients are statistically significant and worth noting. There is no evidence to help explain why this should be the case. It is not, however, because of any relationship between age and the other six characteristics in the table, nor due to the university attended. In the absence of evidence it might be speculated that mature-age graduates have a more mellow attitude to lecturers or, simply, lower expectations. On the other hand, the difference might be due to the nature of the interpersonal relations between teaching staff and mature-age students. No doubt other explanations are possible.

There is little indication of any relationship between gender or non-English-speaking background and responses to the GTS. None of the coefficients is statistically significant and although there are some positive and negative coefficients around the 0.05 level, this is no more than might be expected if the overall coefficient really were zero. Similarly, having a previous qualification has little effect on the GTS, even though the value for accounting graduates in  Table4 is statistically significant.

Attendance on a part-time basis (or external or mixed mode) is clearly associated with lower GTS scores for several fields of study. Again the coefficients are small and the size varies across fields of study. In medicine and psychology, for instance, there is no effect.

Study in the year after course completion has a positive relationship with GTS scores for graduates in chemistry and psychology, but there is little evidence of a relationship for other fields of study. Employment in the year after course completion shows no significant relationship with GTS for any of the fields of study, although there are no negative coefficients.

The Clear Goals and Standards Scale (CGSS)

Age has a positive relationship with the CGSS, but only for economics and history. The effect is certainly more muted and less general than for the GTS. Again gender has no effect on the CGSS. For some fields of study, however, coming from a non-English-speaking background appears to have a negative effect on the extent to which graduates perceive that their course had clear goals and standards. It is tempting to interpret this as the consequence of difficulties with English language. It is interesting, however, that some of the fields of study for which the relationship exists, accounting and civil engineering in particular, are quite technical and might be thought to be fields in which any language effects would be minimal.

There is little effect of having a previous post-secondary qualification on the CGSS, although the relationship for accounting graduates is statistically significant and indicates that a previous qualification is associated with higher scores.

Being a part-time student seems to be associated with lower CGSS scores. All the coefficients are negative, although only the values for accounting and law courses are statistically significant. It is possible that part-time attendance might make it more difficult for students to understand the requirements of their course, and that this is be reflected in the scale scores.

Having a job in the year after course completion shows the most consistent relationship with the CGSS. Five of the ten fields of study have positive relationships with post-graduation employment. It seems that a successful employment situation after graduation is associated with higher CGSS scores. This relationship may be quite direct. Employment may lead to a positive retrospective interpretation of the course. It is also possible that graduates with better results are more likely to be employed, and that graduates with better results are more likely to believe that course standards and requirements were clear.

The Appropriate Assessment Scale (AAS)

Age has a clear positive relationship with AAS scores for six fields of study. For the remaining fields of study, chemistry, law, literary studies and medicine, there is no relationship. There is also some indication for accounting and computer science courses that females tend to have higher scores for the AAS, regardless of any other differences. However, the size of both these effects is very small. Graduates from a non-English-speaking background from computer science, literary studies and psychology courses are marginally more likely to perceive that the assessment of their course was oriented towards testing facts and memory rather than higher-order thinking skills. Having completed a post-secondary qualification before commencing the course, attendance mode, subsequent study and employment all had no systematic effect on perceptions of assessment, with the possible exception of employment status for medicine.

The Appropriate Workload Scale (AWS)

The relationship between age and the AWS varies quite substantially among fields of study. For civil engineering, there is a positive relationship between age and AWS scores, but for accounting, law, medicine and psychology courses there is a negative relationship. It is perhaps easier to explain a negative relationship, older graduates were more likely to have competing demands and inadequate time to cover the material. An alternative interpretation is that older students have a greater desire to engage in depth learning rather than surface learning, in understanding rather than knowing. This explanation might hold for females, graduates from a non-English-speaking background, and graduates with previous qualifications. Regardless of the reason, the values in  Table4 indicate that, in general, female graduates, graduates from non-English-speaking backgrounds, and graduates with previous qualifications have lower AWS scores. Mode of attendance and subsequent study and employment have little effect.

The Generic Skills Scale (GSS)

There are relatively few relationships between the GSS and any of the seven factors. For computer science courses younger graduates report higher GSS scores. For history courses older graduates report higher GSS scores. In both comparisons all other factors were held equal. The negative relationship is, possibly, the expected relationship. Older graduates have had more opportunities to develop many of the generic skills in other contexts, and hence the value-added by a higher education course is likely to be lower. The value of the coefficient for history graduates, however, is not consistent with this expectation.

There is little indication of any consistent relationship between gender and non-English-speaking background. It might have been expected that completion of a prior qualification would lead to lower GSS scores for reasons similar to those advanced for older graduates. Nevertheless, there is no relationship between previous qualification and GSS scores.

Mode of attendance shows what might be an expected relationship with GSS. Graduates who attended their course on a part-time basis had lower GSS scores, presumably because the other activities in which they were engaged tended to contribute to their generic skills. Their perception of the contribution of their university course to the development of these skills was therefore reduced.

The relationship with further study and the effect of subsequent employment are negligible for most fields of study and there is no consistent direction to any relationship across fields of study. The GSS scores are virtually unaffected by educational outcomes.

The Overall Satisfaction Item (OSI)

For three fields of study, economics, history and literary studies, there is a positive effect of age on responses to the OSI. For the remaining seven fields of study there is virtually no effect at all. Gender offers some suggestion of lower levels of satisfaction for female graduates in chemistry, civil engineering and law courses, but the coefficients are small and not statistically significant. For literary studies female graduates have higher levels of satisfaction. Non-English-speaking background also provides hints of a negative relationship with overall satisfaction, but few coefficients show little more than negligible effects and the relationship for economics is positive.

It would seem fair to suggest that the possession of a previous qualification has no effect on overall satisfaction scores. There are some indications of a negative relationship between part-time attendance and overall satisfaction, but this is largely restricted to accounting and chemistry courses.

There is no relationship between undertaking further study and OSI scores except for a positive relationship in psychology. Subsequent employment, however, shows a fairly consistent positive effect on OSI scores.

Summary

This chapter examined the effects of seven characteristics of graduates on their CEQ scale scores for ten fields of study. The characteristics were age, gender, English language background, previous qualifications, mode of attendance, participation in further study and employment status after course completion. The major finding was that any effects were generally quite small, and were rarely consistent across all fields of study.

Within this context, the largest and most consistent findings were:

• a positive relationship between age and the Good Teaching Scale;
• a positive relationship between age and the Appropriate Assessment Scale;
• female and non-English-speaking graduates reported lower scores for the Appropriate Workload Scale; and
• graduates in employment some six months after course completion reported higher levels of overall satisfaction with their course.

[back to top]