Chapter 7. Methodology


Overview

Monash University, with its six Australian campuses and several overseas teaching locations, its ten faculties, its international focus and its different traditions and cultures arising from its pre-amalgamation components, is a very diverse institution (see Appendix 2). It may be viewed as a microcosm of the Australian higher education sector and an appropriate "laboratory" for analysing the key factors involved in the transition experience. Further, evidence from the literature indicates that the transition experience can vary according to the nature of the institution, the discipline area and the characteristics of the students , and the diversity of the Monash campuses, including their origins and the size of their student bodies, could be regarded as representative of different institutions, with faculties reflecting the range of discipline areas. Monash has therefore been our primary research focus, but some students from other universities have filled out questionnaires and attended focus groups, and a statistical analysis of first year commerce students at the University of Melbourne has been undertaken to complement some of this research.

 

Data and analysis

Data for this section of the project have been drawn from several sources and involve several population samples, some of which overlap.

Transition Questionnaires

Two questionnaires were issued to all 1997 first year students at Monash in undergraduate courses in four faculties Arts, Science, Engineering and Business and Economics, at five campuses, Berwick, Caulfield, Clayton, Gippsland and Peninsula. The sample studied thus included students from institutions and discipline areas with a variety of characteristics.

The aim of the questionnaires is to identify which factors claimed in the major studies as crucial to successful transition are applicable now in Australia’s largest teaching institution.

A total of 2072 responses were received for this first questionnaire. Campus responses were 1242 at Clayton, 507 at Caulfield, 90 at Peninsula, 149 at Gippsland and 84 at Berwick. The distribution by faculty was 705 in Arts, 586 in Business and Economics, 309 in Science, 241 in Engineering and 231 in double degree courses. The second questionnaire had a similar response rate.

The questionnaires (see Appendix 3) explored students’ expectations and dispositions on entry and attitudes and experiences after six months. Factors considered include those relating to student, social and institutional characteristics identified in overseas research studies on transition, based on theoretical models and related empirical studies. These questionnaires incorporate features used in previous research, such as the standard Student Adaptation to College Questionnaires (SACQ) or Anticipated SACQ (ASACQ) used in the USA as well as some from McInnis and James (1995), to enable comparisons to be made. Students were asked to rate items on a Likert scale ranging from 1 to 5.

A preliminary assessment of key factors and dispositions on entry, by way of the first questionnaire administered to all entering students in these faculties at enrolment in February 1997, was designed to ascertain students attitudes prior to undertaking tertiary study on a range of issues relating to expectations, preparation for higher education, commitment to the institution and course, academic motivation and approaches to learning. A second questionnaire sent to all students on re-enrolment in late second semester was designed to assess students’ reflections on their experience of transition along the same parameters as the first. The questions followed up issues from the first questionnaire in order to gauge whether experiences matched expectations, and explore issues of academic and social integration and academic motivation.

The results of these questionnaires were analysed using cross-tabulations and Analyses of Variance to identify significant effects. Reliability analyses and factor analyses were also undertaken to identify and construct psychometrically appropriate measures of student characteristics identified as significant in the literature review in Chapter 6. These include student psychological characteristics (academic preparedness, learning strategies, goal commitment and academic motivation) and institutional factors (institutional commitment, academic integration, social integration and course expectations). These constructs--and data on the educational background of family members, not available on data files--were used in the regression models in the following section.

 

Focus groups

The second source is focus groups undertaken by Peel in April, September and November 1997 and involving students from different faculties within Monash and other institutions. The focus groups were selected to contain representative samples of student characteristics (gender, school type, cultural background, rural/urban, living arrangements and means of tertiary entry), of the participating faculties and of different institutions. A total of 31 students from four different universities participated in discussions at Monash (Clayton and Caulfield) and Melbourne University. All major discipline areas were represented. Despite the small size of these groups, discussions were very fruitful in terms of providing detailed student perspectives on transition. In September, Peel used students participating in videotaped interviews for another project as a second round of focus groups. These groups comprised 37 students who attended seven different schools in 1996 and were enrolled in a wide range of courses at six different universities. In addition, some had withdrawn or deferred before or after first semester. Finally, Peel interviewed twenty additional students (in eleven different degree programs on six campuses of four different universities) in a third round of focus groups in November 1997.

The focus groups (for questions see Appendix 3) aimed to assess the extent to which expectations of tertiary study and student life matched experiences, including the nature and adequacy of advice received before course selection; adjustment problems, especially in terms of time management, study skills, disciplinary knowledge and assessment; the nature of the tertiary learning and teaching environment; experiences of social transition, especially the problem of isolation and making friends; and experiences of orientation, mentoring, learning communities, study skills training, counselling or other institutional programs for first-year students.

The focus groups also explored institutional factors which encourage a successful transition, including: strategies for the provision of information, advice and assistance; explanation and clarification of study objectives, outcomes and assessment; and how entering students anticipate, identify and confirm the university’s expectations of them. These data also provide the means to evaluate the success of institutional practices for managing transition and for providing students with a well-managed and coherent series of support structures, including feedback from departers on general and specific support and information strategies which might have induced retention.

The second and third focus group questions repeated most of the above material, but asked participants to reflect upon the whole year and on differences between first and second semester, and to reflect also upon the experiences of their friends and peers. Due to the strong unanticipated reaction to ‘virtual learning’ in the first focus groups, another question focused on the use of teaching technologies. In addition, the second and third rounds of focus groups asked students to specify particular attributes, skills and teaching characteristics of ‘good lecturers’ and ‘good tutors’, and to assess the performance of their best university teacher and other of their university teachers according to eight different criteria.

 

Related studies

The third source of data is a related project being undertaken by Peel. This is a longitudinal study of one transition cohort (1996-1997) through a combination of questionnaires, focus groups and videotaped interviews. This research involved 26 schools in metropolitan Melbourne and Gippsland and gathered 920 questionnaires from Year twelve students in 1996. Peel also spoke with 587 Year Twelve students in focus groups in 1996 and tracked the members of this group to their tertiary destinations in 1997.

Two questionnaires were sent to this group, of whom 553 had enrolled in a university or TAFE course, in April and September 1997.. The first questionnaire generated 188 responses (32%), and the second, sent in September, generated 153 responses (26%). The questionnaires gathered mostly qualitative data, and most respondents provided very detailed answers to questions about expectations, experiences and learning which complemented and expanded the information derived from the focus groups described above. Results from this study were particularly useful in identifying more precisely how the stages and key events of transition varied according to students’ personal characteristics, background and expectations.

 

University data files

The final source is Monash University student data files. The populations studied were students enrolled in fourteen first-year compulsory subjects in the Faculty of Business & Economics at the Clayton, Caulfield and Peninsula campuses in 1996 and 1997. As the Faculty is the largest of its type in Australia, its discipline areas ranging from "humanities type" management subjects to "science type" econometrics subjects, it can be regarded as a ‘microcosm’, with the results obtained relevant to other discipline areas. Some corresponding analysis has been undertaken at the University of Melbourne

Only students with a V-type VCE TER score were included and the unadjusted TERs were used. Such subject enrolments were in the range 200 to 500, and the pooled data involved nearly 5000 observations for each year. Data were extracted for students’ marks in first year university subjects, secondary school marks in pre-requisite and relevant subjects and TER scores, and on students’ personal characteristics. The latter included: secondary school type (government, independent, catholic, secondary TAFE and other); linguistic/cultural background, according to language spoken at home; gender; socioeconomic status from school postcode; distance from campus of enrolment (more or less than 10km from home); and age.

The analysis involves the use of factors suggested as relevant by this project’s Stage I and the descriptive analysis of Stage II, as well as the literature on student transition, attrition and performance overseas and in Australia surveyed in Chapter 6. The relationship between university academic performance in first year, overall and in particular subjects, was statistically analysed according to various input factors, particularly prior academic achievement, conditioned on other variables. Causal relationships were measured and hypotheses concerning key factors investigated, using appropriate statistical methodology.

Regression models were developed initially using data from the 1996 cohort, for which full university academic results were available, to enable a preliminary analysis of significant explanatory factors. Students’ university marks in each subject were regressed against VCE TER scores and other secondary school marks to determine how significant school achievement is in determining university academic performance. Similar models were estimated for the corresponding 1997 entry cohorts and also for the subpopulation of the 1997 group who completed the transition enrolment questionnaires. (A corresponding regression analysis was undertaken at the University of Melbourne for first year Bachelor of Commerce students in accounting and economics subjects in 1996). Finally, for the 1997 cohort who completed the Transition Questionnaires, a further set of regressions also included psychometric data obtained from the questionnaires.

University marks were regressed against VCE scores in each pre-requisite subject (English or English as a Second Language (ESL) for all courses, and Mathematics--Further Mathematics, Mathematical Methods, or Specialist-- Mathematics depending on the course) and associated 0-1 dummies to determine how significant these are in determining the variation in students’ university academic performance. In some subjects (Accounting, Economics, Business Management) first-year university marks were also regressed against marks in relevant VCE subjects and an appropriate 0-1 dummy. Linear, log-linear and logistic regressions were estimated using SHAZAM and the standard diagnostic and goodness of fit tests were undertaken.

To investigate different institutional characteristics, results for the individual campuses were compared. The same subjects are taught at Caulfield and Peninsula, so some analysis of different teaching regimes can be undertaken.

The input variables for the regression analysis thus included:

  • quantitative data from university files on input factors such as student secondary school performance in terms of overall TER and individual subjects for the full 1996 and 1997 Monash cohorts;
  • qualitative data from university files on student characteristics such as gender, age, residency, country of birth, language at home, socioeconomic status, secondary school attended, basis of admission, for the full 1996 and 1997 cohorts;
  • qualitative data, measured on Likert type scales and collected through the Transition Questionnaires, for factors identified by the theoretical models of Tinto, Bean, Spady, etc., and related empirical studies, involving social, institutional and personality characteristics, for the 1997 cohort responding to the questionnaires only.

The different kinds of data from these sources were matched by student identification number, enabling causal relationships to be measured and tested statistically, a feature unique to this section.


< Back 1 2 3 4 5 6 7 8 9 10 Next > Contents