Technical Note No.2

This Technical Note provides a subset of the indicators published in Technical Note 1 (DEST 2004)[1].  This subset is calculated on more recent data for 2004 and includes only those indicators which have been used to assist in the allocation of the Learning and Teaching Performance Fund[2].

The relevant indicators are:

• graduate full-time employment, that is the number of Australian graduates in full-time employment expressed as a proportion of all domestic bachelor graduates available for full-time work (taken from the responses to the Graduate Destination Survey);
• graduate full-time study, that is the proportion of domestic bachelor graduates proceeding to further full-time study (taken from the responses to the Graduate Destination Survey);
• graduate generic skills, that is the level of satisfaction with generic skills acquired (taken from the responses to the Course Experience Questionnaire);
• graduate good teaching, that is the level of satisfaction with good teaching (taken from the responses to the Course Experience Questionnaire);
• overall graduate satisfaction, that is the proportion of graduates who were satisfied with the overall quality of their courses (taken from the responses to the Course Experience Questionnaire);
• student progress rates, that is the proportion of subject load passed by students (based on information supplied by universities).  Only commencing domestic bachelor students were included in the calculation of this indicator; and
• the inverse of the student attrition rates, that is the proportion of students who achieved a positive outcome of either completed in that year or were retained in the subsequent year (based on information supplied by universities).  Again only commencing domestic bachelor students were used.
  

The basic approach adopted in this study is similar to that used in Technical Note 1.  A regression analysis was used to identify a range of student background characteristics and other factors that might impact on student outcomes; and then further regression and probit analysis are used to estimate the independent influence of institution on student outcomes. The regression analysis is estimated for 2004 data except for the attrition rate which is estimated for students who commenced in 2003.

In consultations with the higher education sector some concerns about the adequacy of the indicators to allocate funds and the method of adjustment were raised.  In response, a review of the indicators and the method of adjustment was undertaken by Access Economics between January and April of 2005.  As a result of the review two changes have been implemented[3].  These are:

a) changes in the formulation of the student satisfaction variables as recorded in the Course Experience Questionnaire.  More specifically the student satisfaction with generic skills gained, teaching quality and the course overall is measured using the responses that indicated a satisfaction level of 4 (agree that they were satisfied) and 5 (strongly agree that they were satisfied)[4]; and

b) where appropriate, the application of a regression technique more appropriate to assessing differences in performance.  Here the Access Economics’ suggestion of using probit was adopted. 

In calculating the student outcome performance indicators adjustments have been made to take account of differences in the characteristics of universities and their student populations.  For example, part-time students have higher attrition rates than full-time students.  Institutions with a high proportion of part-time students would therefore have higher attrition than institutions with a high proportion of full-time students.   The adjustment aims to provide a ‘level playing field’ on which the performance of the institution can be assessed, independently of the characteristics of its student body. Both basic (‘crude’) and adjusted (‘adjusted’) indicators are presented at the end of this report.

Depending on the nature of the indicator and the availability of data, factors included in the adjustment process are:

• gender;
• age;
• English Speaking Background status;
• Indigenous status;
• level of study;
• mode of study (external/internal);
• type of study (full-time/part-time);
• broad field of education (*);
• residency status;
• disability status;
• location status, such as rural, isolated or metro;
• socio-economic status, two measures were used based on parents’ education and occupation background and on economic resources;
• new to higher education;
• basis for admission to university;
• Tertiary Entry Ranking (TER) score, as a proxy for the ability of students;
• size of institution, measured by the number of students within a particular institution and broad field of education; and
• labour market conditions using as a proxy the unemployment rate in the graduate’s home area at the time of the Graduate Destination Survey.

(*) Note: Previously nursing was identified separately from the remainder of the health category for all indicators.  Nursing is now identified separately for the GDS/CEQ indicators but not for attrition and progress.

The attached tables provide full results of both crude and adjusted indicators for Table A higher education providers only[5].  Results in this paper may differ from the results published by Graduate Careers Australia (GCA).  The differences are largely due to population definitions such as the inclusion of statistics for three year diplomas in the GCA data.

Ross Dunstan
Yew May Martin
Maureen Maclachlan

Student Outcome Indicators for the Learning and Teaching Performance Fund
	GDS-adjusted proportion in full time study	GDS-unadjusted proportion in full time study
New South Wales
Charles Sturt University	7.33	10.19
Macquarie University	22.82	27.08
Southern Cross University	14.14	14.34
The University of New England	15.90	13.85
The University of New South Wales	26.24	32.41
The University of Newcastle	15.60	18.83
The University of Sydney	26.12	30.47
University of Technology, Sydney	16.18	16.69
University of Western Sydney	20.78	17.66
University of Wollongong	32.04	34.40
Victoria
Deakin University	17.12	18.65
La Trobe University	21.73	25.98
Monash University	24.60	31.12
Royal Melbourne Institute of Technology	13.29	17.23
Swinburne University of Technology	13.69	18.02
The University of Melbourne	29.96	38.69
University of Ballarat	11.29	12.75
Victoria University of Technology	11.53	14.35
Queensland
Central Queensland University	15.54	13.01
Griffith University	21.76	23.26
James Cook University	12.18	11.12
Queensland University of Technology	20.37	19.75
The University of Queensland	31.42	38.85
University of Southern Queensland	13.83	10.33
University of the Sunshine Coast	10.11	13.24
Western Australia
Curtin University of Technology	17.78	21.05
Edith Cowan University	15.72	16.47
Murdoch University	11.86	24.13
The University of Western Australia	39.84	46.41
South Australia
The Flinders University of South Australia	23.45	28.12
The University of Adelaide	30.81	46.04
University of South Australia	14.51	14.32
Tasmania
Australian Maritime College	0.00	9.09
University of Tasmania	22.88	28.31
Northern Territory
Batchelor Institute of Indigenous Tertiary Education	NA	NA
Charles Darwin University	26.29	18.25
Australian Capital Territory
The Australian National University	18.18	36.08
University of Canberra	14.45	17.15
Multi-State
Australian Catholic University	14.67	12.15
All	24.15	24.15

Student Outcome Indicators for the Learning and Teaching Performance Fund
	CEQ-adjusted generic skills	CEQ-unadjusted generic skills
New South Wales
Charles Sturt University	69.33	72.34
Macquarie University	67.76	65.72
Southern Cross University	72.15	71.95
The University of New England	72.53	71.59
The University of New South Wales	63.48	64.88
The University of Newcastle	72.24	72.85
The University of Sydney	75.20	76.32
University of Technology, Sydney	71.98	72.30
University of Western Sydney	67.86	61.94
University of Wollongong	77.78	76.58
Victoria
Deakin University	72.23	71.81
La Trobe University	71.46	71.74
Monash University	71.67	71.51
Royal Melbourne Institute of Technology	67.79	66.59
Swinburne University of Technology	76.45	81.40
The University of Melbourne	73.91	74.64
University of Ballarat	71.97	71.96
Victoria University of Technology	74.86	81.27
Queensland
Central Queensland University	67.07	61.93
Griffith University	76.03	75.03
James Cook University	73.10	72.98
Queensland University of Technology	69.94	66.33
The University of Queensland	77.44	82.37
University of Southern Queensland	70.76	68.32
University of the Sunshine Coast	73.38	72.45
Western Australia
Curtin University of Technology	73.14	73.25
Edith Cowan University	70.82	68.02
Murdoch University	75.37	79.08
The University of Western Australia	74.14	75.52
South Australia
The Flinders University of South Australia	71.65	69.85
The University of Adelaide	63.92	65.24
University of South Australia	70.74	68.22
Tasmania
Australian Maritime College	79.05	90.91
University of Tasmania	66.51	68.22
Northern Territory
Batchelor Institute of Indigenous Tertiary Education	NA	NA
Charles Darwin University	69.00	64.68
Australian Capital Territory
The Australian National University	72.45	76.40
University of Canberra	73.96	72.01
Multi-State
Australian Catholic University	71.75	67.53
All	71.26	71.26

Student Outcome Indicators for the Learning and Teaching Performance Fund
	CEQ-adjusted satisfaction with good teaching	CEQ-unadjusted satisfaction with good teaching
New South Wales
Charles Sturt University	46.54	53.27
Macquarie University	45.80	44.54
Southern Cross University	44.59	50.35
The University of New England	56.00	60.16
The University of New South Wales	35.40	33.31
The University of Newcastle	43.73	47.17
The University of Sydney	48.52	49.48
University of Technology, Sydney	41.71	35.25
University of Western Sydney	43.13	41.63
University of Wollongong	51.94	50.27
Victoria
Deakin University	45.78	47.64
La Trobe University	45.93	47.50
Monash University	47.07	44.76
Royal Melbourne Institute of Technology	41.23	40.94
Swinburne University of Technology	51.15	52.91
The University of Melbourne	50.63	49.95
University of Ballarat	51.73	57.19
Victoria University of Technology	45.41	53.17
Queensland
Central Queensland University	46.80	43.88
Griffith University	45.79	47.01
James Cook University	47.03	50.76
Queensland University of Technology	42.82	35.57
The University of Queensland	54.37	56.45
University of Southern Queensland	48.76	46.32
University of the Sunshine Coast	58.69	62.94
Western Australia
Curtin University of Technology	47.35	46.20
Edith Cowan University	48.34	50.19
Murdoch University	55.88	60.92
The University of Western Australia	49.36	47.02
South Australia
The Flinders University of South Australia	47.19	51.92
The University of Adelaide	39.62	42.35
University of South Australia	40.77	40.03
Tasmania
Australian Maritime College	59.89	63.64
University of Tasmania	43.10	49.70
Northern Territory
Batchelor Institute of Indigenous Tertiary Education	NA	NA
Charles Darwin University	38.85	40.64
Australian Capital Territory
The Australian National University	48.30	52.55
University of Canberra	46.13	49.43
Multi-State
Australian Catholic University	49.94	46.69
All	46.73	46.73

Student Outcome Indicators for the Learning and Teaching Performance Fund
	CEQ-adjusted overall satisfaction	CEQ-unadjusted overall satisfaction
New South Wales
Charles Sturt University	69.35	74.31
Macquarie University	67.85	67.70
Southern Cross University	68.20	69.47
The University of New England	80.81	80.00
The University of New South Wales	64.49	66.04
The University of Newcastle	64.68	65.48
The University of Sydney	69.24	71.18
University of Technology, Sydney	71.61	70.26
University of Western Sydney	60.12	55.31
University of Wollongong	76.60	75.75
Victoria
Deakin University	69.81	68.67
La Trobe University	72.72	73.34
Monash University	71.41	68.45
Royal Melbourne Institute of Technology	66.65	63.65
Swinburne University of Technology	74.46	78.82
The University of Melbourne	72.61	75.37
University of Ballarat	73.28	71.25
Victoria University of Technology	71.32	76.72
Queensland
Central Queensland University	64.78	59.41
Griffith University	67.80	68.11
James Cook University	68.48	65.90
Queensland University of Technology	66.98	62.60
The University of Queensland	71.52	79.42
University of Southern Queensland	71.33	71.86
University of the Sunshine Coast	70.18	70.26
Western Australia
Curtin University of Technology	66.33	65.65
Edith Cowan University	67.42	66.11
Murdoch University	74.94	80.86
The University of Western Australia	69.25	69.57
South Australia
The Flinders University of South Australia	67.74	65.63
The University of Adelaide	60.92	62.77
University of South Australia	64.11	62.17
Tasmania
Australian Maritime College	80.83	90.91
University of Tasmania	63.20	64.14
Northern Territory
Batchelor Institute of Indigenous Tertiary Education	NA	NA
Charles Darwin University	61.72	60.00
Australian Capital Territory
The Australian National University	75.66	80.03
University of Canberra	70.30	70.11
Multi-State
Australian Catholic University	71.64	63.69
All	68.56	68.56

Student Outcome Indicators for the Learning and Teaching Performance Fund
	Adjusted progress rate	Unadjusted progress rate
New South Wales
Charles Sturt University	88.89	82.36
Macquarie University	87.45	88.52
Southern Cross University	89.83	85.18
The University of New England	87.44	79.95
The University of New South Wales	85.02	88.25
The University of Newcastle	89.01	89.98
The University of Sydney	87.13	91.35
University of Technology, Sydney	86.87	88.33
University of Western Sydney	88.49	85.39
University of Wollongong	88.33	89.65
Victoria
Deakin University	87.10	85.69
La Trobe University	85.18	84.49
Monash University	84.61	84.91
Royal Melbourne Institute of Technology	88.13	87.40
Swinburne University of Technology	91.15	87.61
The University of Melbourne	87.04	93.20
University of Ballarat	87.55	86.00
Victoria University of Technology	87.34	81.99
Queensland
Central Queensland University	80.39	76.49
Griffith University	86.59	87.08
James Cook University	79.52	81.35
Queensland University of Technology	84.06	86.40
The University of Queensland	88.24	91.99
University of Southern Queensland	78.87	73.67
University of the Sunshine Coast	86.42	85.35
Western Australia
Curtin University of Technology	86.61	87.96
Edith Cowan University	89.42	87.76
Murdoch University	84.58	84.81
The University of Western Australia	84.48	91.38
South Australia
The Flinders University of South Australia	85.47	86.91
The University of Adelaide	80.88	83.47
University of South Australia	84.40	82.05
Tasmania
Australian Maritime College	94.38	84.07
University of Tasmania	80.96	81.39
Northern Territory
Batchelor Institute of Indigenous Tertiary Education	NA	NA
Charles Darwin University	79.90	77.25
Australian Capital Territory
The Australian National University	86.47	89.82
University of Canberra	94.36	88.83
Multi-State
Australian Catholic University	91.51	94.25
All	86.30	86.30

Student Outcome Indicators for the Learning and Teaching Performance Fund
	Adjusted inverse of the  attrition rate	Unadjusted inverse of  the attrition rate
New South Wales
Charles Sturt University	85.81	78.31
Macquarie University	86.43	86.99
Southern Cross University	78.54	71.82
The University of New England	82.55	73.54
The University of New South Wales	85.71	88.94
The University of Newcastle	85.36	84.60
The University of Sydney	81.99	86.90
University of Technology, Sydney	86.42	89.07
University of Western Sydney	83.26	79.97
University of Wollongong	84.74	85.45
Victoria
Deakin University	83.48	79.87
La Trobe University	81.67	78.25
Monash University	85.04	85.86
Royal Melbourne Institute of Technology	83.01	83.16
Swinburne University of Technology	83.53	81.62
The University of Melbourne	86.00	90.82
University of Ballarat	84.67	78.59
Victoria University of Technology	80.46	75.74
Queensland
Central Queensland University	80.22	72.99
Griffith University	75.61	75.88
James Cook University	75.92	73.00
Queensland University of Technology	81.07	82.38
The University of Queensland	78.58	83.04
University of Southern Queensland	82.38	72.29
University of the Sunshine Coast	68.31	59.91
Western Australia
Curtin University of Technology	79.08	81.18
Edith Cowan University	80.72	75.61
Murdoch University	82.07	79.29
The University of Western Australia	80.80	87.42
South Australia
The Flinders University of South Australia	80.09	78.93
The University of Adelaide	81.54	81.83
University of South Australia	83.56	80.90
Tasmania
Australian Maritime College	78.86	71.14
University of Tasmania	84.87	80.63
Northern Territory
Batchelor Institute of Indigenous Tertiary Education	NA	55.56
Charles Darwin University	79.16	66.12
Australian Capital Territory
The Australian National University	85.15	86.19
University of Canberra	82.80	81.42
Multi-State
Australian Catholic University	81.09	79.68
All	80.92	80.92