Chapter 8. ResultsQualitative analysis Analysis of the qualitative information derived from focus groups, along with detailed responses to questionnaires in Peels related study, suggests important conclusions about transition experiences and the nature of the university teaching and learning environment. The differences between students generally enjoying university learning and study, and experiencing success, and those expressing (or observing among their peers) dissatisfaction, failure or disenchantment, can be largely explained in terms of four sets of factors. While students recognised and described a widely varying level of student commitment to successful transition and also showed the highly specific experiences of transition predicted in the literature, these factors do have particular implications for university teaching and administration, and help identify potential general strategies. The four factors and their specific features are:
1. The perceived quality of teaching, commitment to teaching in different subjects, courses, faculties and universities, and extent of teachers enthusiasm and interest, including actively welcoming of first-year students. Specific features:
2 The presence or absence of clear and effective information about subject and course objectives and assessment methods. Specific features:
3. Whether or not first-year students took advantage of, or had access to, a range of transition, orientation or introduction activities and student services after initial orientation. Specific features:
4. The extent to which first-year students were able to achieve a successful social transition, usually described in focus groups and questionnaires as more important, especially in the first few weeks, than academic transition. Specific features:
Overall, the research provides some crucial insights into the best future directions for first-year and broader undergraduate teaching, the identification and enhancement of academic teaching competencies, academic professional development, the balance between different teaching strategies and technologies, the provision of information to schools and school students, the marketing of the university environment, and much more. One of the more interesting findings was a perception that particular universitiesnotably Monash, but also Deakin (Burwood)were more adept, or at least more pro-active, in handling the transition from school to university. Students from Monash and Melbourne, most of whom have friends attending the other university, consistently used the phrase Monash works harder to explain differences between the institutions, especially in regard to student-based initiatives like the Host Scheme and the range and extent of orientation activities. The analysis suggests a number of possible institutional strategies for handling transition, which are outlined in our conclusions and recommendations. Four significant findings and implications for institutions stand out. First, the first year of university study may need to be reconceived, in response to students concerns about adjusting to a more independent style of learning and to different assessment practices and expectations. Students desire a significant break from senior secondary learning styles and the pressure of score-based expectation, and they consistently identify independent learning, self-discipline and self-reliance as basic goals in their university education. Yet many are unsure about how to achieve those goals, and often find that they are expected to somehow know how to be independent and self-reliant. Second, the repeated emphasis on face-to-face teaching, on access to academic staff, and on meeting senior university students before and during the early stages of transition were all unexpectedly strong, and suggest that tertiary institutions should be more concerned with recognising the important work done by senior students and student organisations in embedding first-year students within the social and teaching environment of the university, as well as the central role of the everyday, small-scale teaching environments which seem particularly important in bonding new students both to their specific courses and to the institution in general. Third, the consistent stress on social transition, on feeling welcomed and on the ability to make friends and feel at home on often dauntingly large and diverse campuses suggests that universities should seek to provide a range of opportunities for social interaction shortly before and shortly after the beginning of each year, that the importance of orientation as a social and academic transition be highlighted in university information for newly-enrolling students and be recognised through funding and good planning, and that strategies to welcome and value first-year students, as well as promote their interaction with each other (through informal project work, small-group teaching, icebreaker sessions and other more interactive and shared forms of learning) should form an important part of teaching, especially in the first few weeks, and especially in large, general faculties where few students will share the same classes. Finally, the identification of students experiencing transition problems needs to pay close attention to both academic and social factors, and to the likelihood that problems are significantly exacerbated by: expectations and self-perceptions that students should be self-reliant (which some misconstrue as able to help themselves and not seek assistance from others); failure or inability to engage with and become part of student-student conversation and interaction through which many academic and personal problems are allayed, worked out and often solved; and feelings of isolation and insignificance relayedmost often indirectlyin everyday teaching practice. It is difficult to draw strong conclusions in this matter. While disengaged students are perhaps unlikely to complete questionnaires or participate in focus groups, a small but significant group of disaffected first-year students were included in Peels research, and other participants provided detailed information about friends and peers who had withdrawn, experienced significant failure and/or suffered personal problems. What these students reveal are profound problems for institutions in the identification of individuals suffering transition difficulties which might nonetheless be partly addressed by various kinds of mechanismsof welcome, adjustment, care and informationsuggested in our conclusions.
Questionnaires The results of two 1997 Transition Questionnaires-one administered prior to enrolment and one six months later in mid second semester-were analysed. The mean scores and range of percentages from cross-tabulations were calculated for each questionnaire item. Significant differences (at 5% level) observed between either faculties or campuses or both are identified from an analysis of variance and noted. A reliability analysis and a factor analyses were undertaken to identify groups of questionnaire items which are valid manifestations of qualitative factors of interest. Items are grouped according to the factor identified for each group, with alpha reliability coefficients for the factor and mean scores for the items in parentheses.
Attitudes at Enrolment Decisions Enrolling students were asked, In deciding to go to university, how important were each of the following for you? and to score these items between 5 (very important) and 1 (not at all important).
Intrinsic interest On starting university, this was a highly important motivating factor (alpha=.4080) for students. This included items such as studying in a field that really interests me (4.8) and developing my talents and creative abilities (4.2) (with scores highest for Arts and lowest for Business & Economics and Science), and was considered of some importance by most (80%) students.
Vocational aspirations This factor (alpha=.5951) included to get training for a specific job (4.0) and to improve my job prospects (4.4). The latter was important for most (87%), with scores higher for the more vocational/professionally oriented faculties and lower for the traditional university campus. Social factors This factor (alpha=.5637),included being with my friends (2.2), few other opportunities because of the poor job market (2.4) and the expectations of my family (2.3), and did not rank high in importance, though some differences in faculty and campus were significant for the first.
Commitments Students were asked to Indicate the extent to which you agree with various statements, with scores ranged from 5 (strongly agree) to 1 (strongly disagree).
Institutional commitment Students at enrolment demonstrated a strong institutional commitment (alpha=.4885), which included items such as It is important for me to graduate from this university (4.7), and It is important for me to graduate from the course I am enrolling in (4.5) (with Gippsland highest).
Goal commitment This factor (alpha=.6188) included I am confident that I made the right decision in choosing to enrol in this course (4.3), I attempted to find out all I could about the course I am enrolling in (3.9), All the information I needed about my chosen course was available to me (3.6), and This course was one of my higher preferences (4.4). This item was most important at Clayton, least important at Caulfield).
Goals and motivations Integration Students recognised early the importance of academic and social integration, by giving low scores to this factor (alpha=.5743), which included getting good grades is not important to me (1.6), I find it difficult to get myself motivated to study (2.7), I usually mix with students of the same sort of background (2.9), I find it quite difficult to comprehend a lot of the material I am supposed to read (2.2) (highest at Gippsland, highest in Science, lowest in Arts), I feel uncomfortable participating in group discussions (2.4), and I only seriously study what's actually required by teachers (2.6) (highest in Engineering, lowest in Arts).
Lack of goals Few commencing students demonstrated a lack of goals (alpha=.5603), which included items such as I have no idea at all what I want to major in (2.3) (Arts highest), It will give me another three or four years to decide what I really want to do later on (2.6), (Engineering lowest, Clayton highest), and I am drifting into higher education without knowing if it is what I want to do(1.8).
Focus and organisation New students also appeared highly focussed or organised in their responses to this factor (alpha=.6194) which included such items as The qualification at the end of the course will enable me to get a good job (4.0) (highest in Engineering and Business & Economics, lowest in Arts, Clayton lowest), The course will help me develop knowledge and skills which will be useful later on (4.5) (highest at Berwick, lowest at Peninsula), I expect to get most of information I need from university teachers (3.2) (highest in Engineering, lowest in Arts), I really enjoy the practical and applied aspects of my subjects (3.8) (highest in Science and Engineering), Being at university will really help me get what I want in life (3.9), I regularly seek the advice and assistance of the teaching staff (3.4) (highest at Berwick), I expect to work much harder this year than in my final year of secondary education (3.7) (highest at Gippsland and Science and Engineering), and I am confident I know how to get the relevant information from lectures (3.4).
Academic motivation Most students also appeared at the outset to be highly academically motivated: this factor (alpha=.7611) included I will be able to study subjects in depth, and take interesting and stimulating courses (4.1) (highest at Berwick, lowest at Gippsland, highest in Arts), I want to develop as a person, broaden my horizons, and face new challenges (4.3) (highest in Arts, lowest in Engineering), I want to prove to myself that I can do it (4.0) (highest in Arts and at Peninsula, lowest at Clayton), I have a strong desire to do well in all my subjects (4.6), At school I have found most of my subjects really interesting (3.7), I get a lot of satisfaction from studying (3.5) (highest in Arts, lowest in Business & Economics and Engineering), and I enjoy the intellectual challenge of subjects I study (3.9). Most enrolling students also thought that Having done well at school, it seemed the natural thing to do (3.8) (with Clayton highest), and that The opportunities for an active social life and/or sport attract me (3.6). Responses varied for I know the type of occupation I want (3.6) (highest in Engineering, lowest in in double degrees, highest at Caulfield, lowest at at Clayton), and few thought that An important factor in choosing subjects is whether I will get high marks (2.8).
Attitudes in mid second semester Satisfaction and institutional perceptions Students were asked Thinking about your university course [or your experience of university study] in general, please indicate the extent to which you agree with the following statements, with scores ranging between 5 (strongly agree) and 1 (strongly disagree).
Satisfaction After six months at university, most students appeared to be reasonably satisfied. This factor (alpha=.8310) included these items: I am enjoying my university course more than I expected (3.24), I am confident that doing this course was the right decision (3.62), I am confident that attending this university was the right decision (4.11), I feel this course will really develop my talents and abilities (3.69), and Overall, I'm very satisfied with my university experience so far (3.48). At least half of the students surveyed had some idea of their course destination. 50% were in agreement with This course will give me good job prospects in the job market (3.43) (with lowest scores for Clayton and for Arts), and few thought that I'm not sure where my course is leading (2.80) (with highest scores for Clayton and Arts). Students generally appeared satisfied with their choice of course, with few thinking that "I will probably try and transfer into another course in 1998 (2.16), I will probably try and move to a different university in 1998; (1.54) or I would like to change courses because I am not enjoying this one (1.97). Most students after six months had a positive attitude to their course, indicated by Most of the students in my course are enjoying university education (3.21), and I seem to be getting less out of my course than most other students (2.46). Students (85%) opposed the view that It doesn't really matter if you enjoy your course or not (1.62), particularly at Clayton and Berwick and less so at Caulfield.
Study In terms of experiences of study, students responses varied for University study was harder than I expected at first (3.44), I find it easy to complete the required work for my subjects each week (2.93), and I think this course tries to cover too much too quickly (3.32) (lowest in Arts, highest in Engineering), but most felt I am confident that I am comprehending the most important information (3.49).A spread of responses was also evident in study consistency, a factor (alpha=.7438) identified by such items as I find it difficult to keep myself motivated to study (3.43), My study patterns are relatively inconsistent (3.31),and I did most of my work last semester in the final few weeks (3.01). Few thought that You could get through this course without working very hard (2.4), with 57% of students disagreeing or highly disagreeing, particularly at Peninsula and in Business and Economics.
Advice and institutional concern Perceptions varied about the provision of advice, a factor (alpha=.5954) identified by If I needed it, I know where to get advice on study or living problems (2.99), with agreement and disagreement evenly divided (both 36%), and lowest at Berwick, highest (3.4) at Peninsula. Most agreed that I have found most student advisers in my faculty very helpful (3.09). Students in their second semester appear to have divided perceptions about the institutions concern for them, a factor (alpha=.6868) as identified by two items: I think the university as a whole takes an interest in student welfare (2.95) and I think my faculty takes an interest in students' transition to university (2.99).
Teaching and learning Teaching Perceptions varied about teaching quality, a factor (alpha=.8373) identified by several items.. Less than half (44%) of all students either strongly agreed or agreed with the statement that Most of my lecturers are able to clearly communicate information (3.22), with faculty differences significant (lowest Engineering, highest Arts). Similarly only 47% of all students either strongly agreed or agreed that Most academic staff try to make the subjects interesting (3.33), again with significant faculty differences (lowest Engineering, highest Arts). Most (59%) agreed or strongly agreed that The quality of teaching in my course is generally good (3.57), and (62%) agreed or strongly agreed that Most of my lecturers seem enthusiastic about their subjects (3.65) again with Engineering lowest and Arts highest. Responses were also high for I am finding my studies challenging and satisfying (3.47), and In most subjects, teaching aids are used effectively (3.28)At least half (51%) of the students either strongly disagreed or disagreed that I would learn as much if I did most subjects on-line or by independent reading instead of attending classes (2.60) (most disagreement (61%) in Arts, least in Engineering), and 50% disagreed or strongly disagreed that I would like to have more computer-based learning (2.60) (again with Arts highest, Engineering lowest, and less disagreement(35%) at Caulfield and Peninsula).
The learning environment and assessment Later in the year students appeared reasonably comfortable in their environment, a factor (alpha=.7692) identified by three items. The majority (55%) of students either strongly agreed or agreed that I feel comfortable participating in group discussions (3.52), though less so in Science (44%), but only 46% of students either strongly agreed or agreed that I feel OK about asking questions in class (3.45), with fewer at Clayton and again in Science, and with Arts and Business and Economics highest. Most (74%) students either strongly agreed or agreed that I feel comfortable approaching my university teachers for help (3.50), though less so at Clayton and more so in Business and Economics and in Arts. About a third (36%) of students either strongly agreed or agreed, or were neutral (36%) about My tutor or demonstrator seems to take an interest in my progress (3.11), though responses were more positive in Arts. Most students agree that Sometimes I discuss my work with other students in the same subject (4.04). In terms of independent learning, most students claimed that I feel I am self reliant when locating resources for assessment tasks (3.94), and that I think I am being encouraged to be an independent learner (3.97). Few agreed that The assessment tasks have not been very challenging (2.25). Students found reasonably helpful feedback, a factor (alpha =.5881) which included I am getting helpful feedback from most of my lecturers and tutors (3.19) (with 39% of all students at least agreeing with this statement, though 39% were neutral, with the highest scores in Arts), and In most of my subjects, the assessment tasks are clearly explained (3.50).
Student life Students were asked to Indicate the extent to which you agree with these statements about student life, with scores ranging from 1 (strongly disagree) to 5 (strongly agree). Most (74%) either strongly agreed or agreed that I am enjoying being a university student (3.96) (particularly at Clayton). Nearly half (46%) either strongly disagreed or disagreed with I still don't really feel that I know where everything is on this campus (2.76), though fewer at the larger campuses. About half (49%) of all students either strongly agreed or agreed that I hardly ever stay on campus after my classes have finished (3.31), more so at Berwick and less so at Clayton, and more so in Business and Economics. Most (57%) students either strongly disagreed or disagreed with the statement I am involved in some extra-curricular activities (2.44), with differences observed by campus.
Work appeared to be of some concern, a factor (alpha =.6393) identified by, It is hard to fit in study when I'm working part-time as well (3.25), Worrying about money has had a real impact on my study (3.17), and Balancing social life, study, and work is my biggest problem (3.34). A friends factor (alpha =.5529) was identified by items, I have met a lot of new people and made some new friends (3.99) (with 73% in agreement, though fewer in Arts), Most of my friends seem to be balancing everything better than I am (2.67), and The most important support I get is from my friends at university (2.96) (with 35% in agreement, 32% neutral, and highest in Arts and Business and Economics). Most disagreed that I don't need as much support from my parents or family this year (2.91). Few appeared dissatisfied with student life, a factor (alpha =.6184) identified by items University hasn't really lived up to my expectations (2.76), and I seriously considered withdrawing or deferring from my course (2.30)
Preparation for university On both questionnaires students were asked, How well did your education or experiences prepare you for university? Scores ranged from 5 (very well) and 1 (not well). Students at enrolment generally felt academically prepared, but later in the year they were less confident.. This factor (changed from alpha = .75 in the first questionnaire to .72 in the second) included being able to work without much direction from a teacher (3.9 to 3.5), using resources like libraries and information technology (3.9 to 3.6), relevant subject knowledge developed in secondary education (3.9), (with scores highest at Clayton and in Engineering for these three), the study skills you need to carry out your work effectively (3.9 to 3.5), organising your own life, including part time work (3.7 to 3.3) (highest in Business & Economics), and feeling confident about approaching teachers for advice and assistance (4.1 to 3.7).
Understanding of learning In both questionnaires students were asked to Please rate these statements in terms of how close they are to what you understand "learning" to be. Scores ranged from 5 (very close) to 1 (not at all close). Several concepts were close to students understanding of learning at enrolment. Many perceived learning as accumulating information (alphas = .4822), a factor which included making sure you remember things well (3.65) (highest at Peninsula, lowest at Clayton), building up your knowledge by acquiring facts and information (4.3), and getting on with the things you've got to do (3.1) (highest at Gippsland, lowest at Berwick). More regarded it as personal development, a factor (alpha = .6853) which included developing as a person (4.2) (Berwick highest, Gippsland lowest, with 44% rating this as very close), using all your experience in life (4.1), seeing things in a different and more meaningful way (3.9) and being able to relate to people better (3.6). Also highly ranked was applying information, a factor (alpha = .5837) which included being able to use the information you've acquired (4.4), creating links between different kinds of information and different subjects (4.1) and thinking for yourself (4.4). Autonomous exploration (alpha = .5795), a factor which included understanding new material for yourself ( 4.25), mastering a particular method or discipline (3.85), and gaining ideas for knowledge you can acquire on your own (4.2), also was close to students understanding of learning. After six months at University, a factor analysis identified a single learning factor (alpha = .7976) for all these items. Most were in agreement with personal development (4.08), seeing things in a different and more meaningful way (3.89), being able to relate to people better (3.52), creating links between different kinds of information and different subjects (3.95), and making sure you remember things well (3.26), and agreement was lowest in Engineering and highest in Arts for all but the last. Scores were also reasonably high for using all your experience in life (3.85), being able to use the information you've acquired (4.25), understanding new material for yourself (4.12), getting on with things you've got to do (3.09), thinking for yourself (4.27) (with highest scores in Arts and Science), mastering a particular method or discipline (3.79), and gaining ideas for knowledge you can acquire on your own (3.91).
Student characteristics A question about the highest tertiary qualification of family members was included on the initial questionnaire, as such information was not obtainable from university data files. This related to parents and other siblings, with possible responses being none (1), currently studying (2), diploma (3), degree (4) and higher degree (5). Many had family members without tertiary qualifications, with means 2.16 for mother, 2.48 for father and 2.12 for other siblings. Campus differences were observed both for parents (highest for Clayton, lowest at Gippsland) and siblings (highest at Berwick, lowest at Gippsland). In general there appears to be a consistent difference between the results for Clayton, the campus of the traditional university, and the newer Monash campuses which are fairly similar.
Students were also asked in the second questionnaire, How many hours do you work in paid employment in an average week during semester?, with responses categories 0 (none), 1 (1-5 hours), 2 (6-10 hours), 3 (11-15 hours) , 4 (16-20 hours), 5 (more). Just over a quarter (27%) had no paid work, and about a third (32%) each worked up to 10 hours or between 10 and 20 hours per week. Fewer were in paid employment at Clayton and in Engineering.
Statistical Analysis Regression analyses were undertaken of students marks, in all first year Business and Economics subjects at three campuses in 1996 and 1997, as a function of their secondary school TER, scores in pre-requisite and relevant school VCE subjects and a range of personal characteristics. These applied to individual subjects in each first year compulsory subject, and to pooled regressions for the subjects at Clayton and at Caulfield/Peninsula (which have the same curriculum) respectively, and for all three campuses combined. Also analysed, for the cohort of students who completed the questionnaires, were regression models plus an extra set, which included the factors identified in the previous section in the reliability analysis. A linear or logistic specification appeared the most successful for the regression models generally. Initially a large range of exogenous variables was included and then restrictions were imposed. Selected results are presented in Appendix 4. The relevance of various factors in explaining first year university marks sometimes appears to differ according to discipline area and campus, but some features clearly emerge as significant overall. Factors which appear to be significant generally in explaining the variation in first year overall academic performance in the population studied are as follows.
Several factors varied in the significance of their influence on students academic performance in individual first-year subjects depending on the discipline area or campus. Relevant VCE subjects for accounting and economics differ in significance by campus, while for business management, the VCE subject appears less significant. Distance from campus of home residence appears to have a negative significant influence only in some cases. Similar results reflecting the importance of Mathematical Methods for first-year accounting, economics and econometrics/business statistics subjects, particularly at Clayton, were observed for individual subjects at the University of Melbourne, except that the TER did not appear as relevant in explaining variation in tertiary academic performance. However, this could be explained by the fact that their cohort corresponded to students enrolled in one course (Bachelor of Commerce) so the range of TERs was small. In contrast, the Monash cohort included students from many undergraduate courses enrolled in Business and Economics subjects and their TERs varied across a 40-point range. Ideally we would like to be able to identify which teaching and learning environments affect performance, using statistical methods in addition to our qualitative analysis. We included dummy variables as a proxy for levels of teaching resources allocated by department. However, as other factors, such as grading policies, also differ by department it is difficult to draw inferences from any significant differences observed. The nature of the curriculum varies to some degree between the traditional campus at Clayton and that at Caulfield/Peninsula. This, as well as institutional characteristics of the campuses, may account for differences observed in performance in subjects in the same discipline areas. For the cohort completing the 1997 Transition Enrolment Questionnaire, the significance of the qualitative factors, identified in the analysis of the previous section, in predicting overall academic performance was analysed. Several factors, such as an understanding of "learning", appear to differ across campuses. However, two factors appear significant overall: motivated and academically and socially integrated` students generally achieve higher marks, all other things being equal. The quantitative results clearly demonstrate that the more academically successful students are those who know the type of occupation they want, are motivated to study, study more than the required minimum, consider good grades important, comprehend the required reading material, and feel comfortable participating in group discussions and mix with students from various backgrounds.
Conclusion This research confirms that there are common transition issues which are likely to arise in all institutions and for all groups within each transition cohort. In general, the students who participated in these questionnaires and focus groups shared many of the experiences of students described in Section I of this study. A majority had some difficulties matching experiences with expectations and motivations, and grew slightly less confident about their ability to function effectively in the university teaching and learning environment. However, most reported themselves satisfied, and were more likely to describe their first year in positive rather than negative terms. But the data suggest that there are also issues (as well as feasible practices which might deal with them), which are specific to particular institutional environments (and to specific teaching and learning environments at the faculty and even department level within institutions) and to particular groups within each cohort, whether identified by social factors (such as rural students), educational background, or motivations and perceptions upon commencement. As a large and diverse institution, Monash shows particularly clearly both common issues and important differences. Indeed, the Monash campuses offer something of a microcosm of the Australian university system, and it is likely that the highly variegated pattern revealed here will be repeated throughout the system as a whole. For instance, the Monash variations from a common pattern of transition problems include the different orientation and early learning experiences within different campuses and courses, differences between those first-year students enrolled mainly in common subjects and those able to choose from a wide variety of subjects, and the extent to which particular faculties and departments successfully welcome and embed students into some form of corporate identity. The qualitative data (which include students from other universities) also suggest that students educational background, varying approaches to learning developed in different secondary school environments, balance of vocational and general educational objectives, and expectations of the purposes of tertiary study, lead them into particular institutions and courses and have a major impact on the likelihood of successful transition. Within each transition cohort, then, common problems will be shaped, magnified and affected by specific combinations of expectations and early experiences, and by specific teaching and learning environments. The anticipation, measurement and management of those problems must therefore combine general strategies to alleviate factors which lead to unsuccessful transition with more fine-grained, student-focused and institution-sensitive practices. |