E     I     P
		Study of Non-Response to the 1996 Graduate Destination Survey
		97/10 Bruce Guthrie Graduate Careers Council of Australia Trevor Johnson Australian Council for Educational Research
		Evaluations and Investigations Program Higher Education Division Department of Employment, Education, Training and Youth Affairs

© Commonwealth of Australia 1997

ISBN 0 642 23669 0

This work is copyright. Apart from any use as permitted under the Copyright Act 1968, no part may be reproduced by any process without prior written permission from the Australian Government Publishing Service. Requests and inquiries concerning reproduction and rights should be addressed to the manager, Commonwealth Information Services, Australian Government Publishing Service, GPO Box 84, Canberra ACT 2601.

This report is funded under the Evaluations and Investigations Program of the Department of Employment, Education, Training and Youth Affairs.

The views expressed in this report do not necessarily reflect the views of the Department of Employment, Education, Training and Youth Affairs.

Contents

[top] [next chapter]

Acknowledgments

Executive Summary

1. Introduction

2. Review of the Non-Response Literature

3. The Non-Response Survey: A Comparison of Respondents and Non-Respondents

4. The Graduate Destination Survey: Reasons for Non-Response

5. Conclusion and Recommendations

Appendices

• Appendix A: Methodology for 1996 Survey of GDS Non-Respondents
• Appendix B: Standard Graduate Destination Survey Form

References

Acknowledgments

[contents] [next chapter]

Bruce Guthrie of the Graduate Careers Council of Australia (GCCA) and Dr Trevor Johnson of the Australian Council for Educational Research (ACER) designed the survey, conducted the data analysis, and prepared this report on the 1996 Non-Response Survey for the GCCA. On behalf of the GCCA, the authors thank the Department of Employment, Education, Training and Youth Affairs (DEETYA) for funding this project.

Project Director Roger Bartley (Executive Director, GCCA) joins the authors in acknowledging the valued advice of the GCCA Survey Management Group, and the essential contribution of the survey managers in the participating universities:

• Peter Cucchiarelli, University of South Australia
• Frank Curran and Paul Aldridge, James Cook University of North Queensland
• Jennie Curry, Charles Sturt University
• Virginia Gray, University of New England
• Christine Holmes and Don Johnston, Southern Cross University
• Brett Lacey and Kate Gemmell, University of Canberra
• Dave Lamb, University of Adelaide
• Wendy Marchment, Flinders University of South Australia
• Scott Martinson and Colleen Liston, Curtin University of Technology
• Helen Parker, University of Newcastle
• Robyn Peutherer, University of Technology, Sydney
• Marilyn Prestage, Edith Cowan University
• Alan Richardson, Queensland University of Technology
• Andrew Simpson and Joanne Tyler, RMIT

Executive Summary

[contents] [next chapter]

Since 1972, all Australian universities, in association with the Graduate Careers Council of Australia (GCCA), have invited their recent graduates to complete a Graduate Destination Survey (GDS) form. This is an annual survey, and since 1992 it has included the Course Experience Questionnaire (CEQ). The target population is the group of people, including overseas or international students, who had completed the requirements for a higher education qualification in the previous calendar year.

The present survey of non-response (NRS) had two main aims. First, to determine whether non-respondents differed from respondents, and if so, how they differed. Second, to determine the various reasons for non-response to the 1996 GDS as a way of refining the current methodology in an attempt to increase future response rates.

Seventeen universities originally agreed to participate in the NRS, but three did not return their NRS forms to the researchers at the end of the survey period. As a result, the sample on which the current report is based is smaller than the researchers would have wished, and this resulted in restricted analysis. Commentary based on disaggregations is necessarily cautious.

The report notes that whatever methods are used to alleviate non-response they are likely only to partially solve the problem. Notwithstanding that, any increase in response rates will mean less uncertainty about what graduates were doing shortly after graduation, and what they thought of their course (in terms of CEQ responses). In addition, any understanding that can be gathered concerning the remaining non-respondents and their motivations (or lack of motivations) can perhaps help to fill in the non-response information gap.

An important issue in this study was whether the answers to the GDS questions gathered from respondents were representative of the entire population of graduates. If non-respondents did not match respondents on various characteristics, it could mean that the GDS suffered from response bias, and that the GDS results might not be representative of the situation of all graduates. There were three important findings:

• In terms of the characteristics, the report suggests that non-respondents, as a group, do not entirely match respondents. The characteristics in question were sex, age, level of qualification, fee-paying status and mode of attendance. The differences found in some of these characteristics were relatively minor.

• In terms of field of study, the non-respondent group was found to be an adequate reflection of the respondent group.

• In terms of graduate destinations, the non-respondents were less likely to have been engaged in full-time study in 1996 and slightly more likely to have been available for full-time employment (that is, either employed full-time, or working part-time and seeking full-time employment, or not working and seeking full-time employment).

These results would raise some concerns that national GDS figures may not be entirely representative of those for the population. However, the previously noted small sample size may also be a factor in the lack of matching. There was a further important aspect uncovered:

• When full-time work force figures for respondents and non-respondents were examined at broad levels of aggregation, a high degree of consistency was found. Respondents and non-respondents were equally as likely to have been in full-time employment or seeking full-time employment. However, the size of the sample the researchers had to work with meant that breakdowns across even simple variables (such as sex) produced figures which were less consistent and which might be considered unreliable. This gives cause to regret the loss of three institutions from the original participating group.

• Overall, it seems that at high levels of aggregation, GDS data are likely to be reasonable indicators of the employment status of the population. When the data were disaggregated, some differences became apparent. However, the differences seen were differences of degree, and the overall employment-related GDS figures gathered from NRS respondents generally reflected those found in the main GDS.

Accordingly, in terms of GDS figures relating to graduates in and seeking full-time employment, data gathered in the GDS are likely to be reasonable indicators of the full-time labour market position of the population of graduates from that relevant cohort. More importantly, if GDS methodology remains consistent over the years, comparisons based on GDS figures over a number of years will continue to produce useful measures of change.

The reasons for non-response to the GDS constitute a complex issue, not easily resolved. There are probably two main (and unremarkable) strands to non-response as recorded in the NRS. The first relates to the physical issue of the delivery of the questionnaires. Graduates cannot respond if they do not receive the survey form.

The second relates to graduates' attitudes. They might be too busy to complete and return the form, or they might see it as being unimportant, or too long, or too confusing. Most likely, their reason for not responding will be represented by more than one of these attitudes.

For a good percentage, it may be that lack of personal organisation is the key factor. For others, the fact that they are still seeking employment may influence their decision not to respond. However, any effect of unemployment on non-response would seem to be a marginal one as there is little difference between respondents and non-respondents on this factor.

There will always be some level of non-response, but this can be reduced by finding solutions to a number of simpler problems.

The recommendations made include:

• The generation of pre-graduation and graduation-time publicity in a campus-based awareness program about the importance and benefits of the data collected in the GDS. This might help to reduce non-response while also highlighting the value to current students.

• The layout and wording of the current questionnaire should be considered, with a view to making the form attractive and easy to complete. An examination of returned GDS forms for markings or comments might highlight areas of confusion for respondents.

• The standard GDS questionnaire and any covering letters should carry prominent information about estimated completion time. Clear notice could also be given that respondents need not answer all the questions on the GDS form (such as the salary question), as this might reduce the level of non-response arising from concerns about privacy. However, this should not be done at the risk of encouraging total non-response.

• The standard questionnaire should ask graduates to note the date they completed the form. This would enable further research to be done on late response, which can be seen as an aspect of non-response.

• The GCCA should survey the covering letters and related materials used by universities and colleges which generate high response rates with a view to developing a 'bank' of useful letter templates for use by all institutions.

• Explanatory text on the GDS form and in related covering letters or any other accompanying documents could give information about the services offered by the university in the area of advice for employment seekers, course information, and the like. This might encourage those still seeking full-time employment at the time of the survey to respond.

• As a last resort, the Survey Management Group should investigate the possibility that a shortened version of the GDS form might encourage some non-respondents to complete and return their forms. This shortened form should be prominently labelled as a shortened version, and the estimated completion time should also be shown clearly. Appropriate covering letters should also note this information.

[return to top]