Repositioning Quality Assurance and Accreditation in Australian Higher Education

Executive Summary

Introduction

1. This report has been prepared for the Commonwealth Department of Education, Training and Youth Affairs (DETYA) to assist in developing an improved national approach to both quality assurance and accreditation. The current arrangements are inadequate and Australia needs a more rigorous quality assurance and accreditation system: to protect the international reputation of our higher education; for public accountability purposes; to inform student choice; and to promote and improve quality processes and outcomes as well as disseminate good practice.
2. Various models for a new approach to quality assurance and accreditation have been reviewed, including refinement of the current Australian model, the recently modified New Zealand model, the new United Kingdom model, the model used by the Vocational Education and Training (VET) system in Australia and a ‘Modern Australian Model’.
3. The Modern Australian Model provides separate arrangements for both institutions which have been given power to accredit their own courses and for non-self accrediting providers. For institutions with power to accredit their own courses, the main requirements will be: (a) rigorous scrutiny of financial and quality aspects before founding legislation is passed or other authorisation is given; (b) annual publication of Quality Assurance and Improvement Plans for the forthcoming triennium; (c) a quality audit of each institution every five years following a detailed self-assessment; and (d) compliance with any additional measures which may be necessary to ensure the maintenance of acceptable high standards of degrees. For non self-accrediting providers, the main features may include: (a) rigorous scrutiny of provider capacity before accreditation; and (b) review of provider performance and accredited courses every five years.
4. The task for the project was to develop the Modern Australian Model as an alternative to the other four models; advise under whose authority it should be run and whether a legislative base is needed; assess whether it would be sensible and appropriate to make use of the AQF; elaborate the possible nature of the five yearly self-assessments for self-accrediting institutions; comment on the desirability of focussing on outcomes and standards as well as processes; consider how to achieve rigour and independence while retaining the cooperation and confidence of universities; and advise on the role of professional associations within the model and the nature of the audit of the courses of non self-accrediting providers. We were requested to evaluate the Modern Australian Model against the following criteria: credibility; effectiveness; ability to provide legal clarity for students and providers; ability to promote and enhance improvement and good practice; how well the model could build on the key features of the current system and possibly exploit the role of professional associations in accrediting courses; minimum bureaucracy; and cost.

Quality Assurance and Accreditation

5. Quality assurance in higher education is defined as systematic management and assessment procedures adopted by a higher education institution or system to monitor performance and to ensure achievement of quality outputs or improved quality. Quality assurance aims to give stakeholders confidence about the management of quality and the outcomes achieved.
6. Accreditation refers to a process of assessment and review which enables a higher education course or institution to be recognised or certified as meeting appropriate standards. In Australia, the term accreditation has developed three specialist meanings: a process of review or assessment conducted by a government agency to enable a Minister or approved authority to recognise and approve a higher education institution or course; a process of review carried out by a government registration body to enable graduates of particular courses to practise in the particular State or Territory; and a process of assessment and recognition carried out professional associations. In this report, we are primarily concerned with the first usage.
7. Quality in the context of higher education can be defined as a judgement about the level of goal achievement and the value and worth of that achievement. It is also a judgement about the degree to which activities or outputs have desirable characteristics, according to some norm or against particular specified criteria or objectives.
8. Over the past decade, extensive experimentation has taken place internationally with quality assurance and how it is managed. The literature reporting these developments points to tremendous variety in approaches and methods. The main approaches can be summarised under the following headings: the agency or unit with responsibility for the management of quality assurance; participation in reviews and other activities; the main methodologies employed; the focus of quality assurance activities; the purposes of such activities; and reporting and/or follow-up.
9. The most common pattern at national level is for responsibility to lie with a specialised government agency, or with the central department responsible for higher education coordination. In a small number of countries, however, responsibility lies with an agency set up by higher education institutions themselves.
10. An important variation between quality assurance systems is whether participation is voluntary or compulsory. Many countries began with institutional audits on a voluntary basis. Generally, however, with national reviews of disciplines, participation is compulsory and, even when participation is voluntary, strong moral and professional pressures usually produce in a high level of participation.
11. Most quality assurance mechanisms depend on one or a combination of a limited number of methodologies, the most important of which are self-studies or self-evaluation; peer review by panels of experts; use of relevant statistical information and performance indicators; and surveys of key groups, such as students, graduates and employers. At the national level, the most common forms of assessment are 'horizontal' reviews of disciplines and 'vertical' evaluations of institutions.
12. Quality assurance programs serve a variety of purposes but generally their primary purposes are a combination of public accountability, efforts to ensure credibility, improvement and renewal. In some cases, there is a gap between stated purposes and actual purposes, and frequently there is tension between accountability and improvement purposes.
13. Reporting and follow-up activities are important, with a major challenge being to devise fair and effective methods likely to lead to improvements without damaging the institution being reviewed. Various approaches are used with regard to the distribution of reports. In some cases, reports are provided solely to the institution concerned but increasingly the practice is to make the results more widely available

Higher Education Providers and Current Arrangements

14. For the purposes of this study, Australian higher education providers can be categorized into five distinct groups: public universities and other public higher education institutions established under State, Territory and Commonwealth legislation; non-government institutions which operate under their own legislation and have self-accrediting powers; institutions not established by legislation but who have been given government approval to operate; private providers whose courses have been accredited by State or Territory accrediting agencies; and private providers whose courses have not yet been accredited.
15. Accreditation of higher education institutions and courses is under the control of State and Territory Governments who view this responsibility as flowing from their responsibilities for education under the Commonwealth constitution. Generally, the relevant legislation makes provision for private providers to secure accreditation and approval to offer courses. In other cases, legislation provides for accreditation of both institutions and courses.
16. The considerable differences between the provisions of State and Territory legislation and the criteria and processes constitute a major problem needing attention. In a number of cases, legislation provides control over the use of the terms ‘university’ and ‘degree’, and over degree titles. The most detailed legislative controls operate in Victoria, New South Wales, Queensland and Tasmania. Additional protection is afforded with regard to the establishment and recognition of universities by other Commonwealth, State and Territory legislation.
17. Recently the States and Territories have agreed on procedures for considering applications and authorisation to offer higher education courses in two or more States and Territories, and operational guidelines to achieve this were endorsed by MCEETYA in April 1999.
18. The current system of quality assurance operates at a number of levels and includes the activities of professional associations and associations and networks set up by groups of universities for benchmarking and other quality assurance purposes.
19. Internal quality assurance processes in universities include: assessment of new courses and units of study; reviews of courses, units and departments; student evaluation of teaching; use of external examiners for higher degree research theses and sometimes bachelors honours theses; surveys of graduates and employers; use of performance indicators; benchmarking and participation in benchmarking networks; and special projects for the improvement of teaching and special awards for teaching excellence.
20. The current national quality assurance mechanisms include: reports by universities on quality assurance and improvement plans; encouragement of innovation and good teaching; and publication of Characteristics and Performance of Higher Education Institutions.
21. Various professional bodies conduct accreditation of professional courses in fields such as medicine, law, engineering and architecture. Professional associations have formed a peak body, the Australian Council of Professions.
22. Special Commonwealth mechanisms, consisting of legislation and a register of courses, provide protection for international students. For example, The Education Services for Overseas Students (Registration of Providers and Financial Regulation) Act 1991 helps ensure that only quality courses are offered to foreign students studying in Australia.
23. While there are various desirable characteristics of the current quality assurance and accreditation arrangements, at the same time there are clear weaknesses that need attention. The major weaknesses in quality assurance is lack of a national agency that can publicly vouch for the quality of Australian higher education while with regard to accreditation there is need for uniform approaches and criteria across States and Territories and a better system of reporting and providing public access to information concerning which courses and institutions have been accredited.

Changing Quality Environment and the Modern Australian Model

24. Important recent changes have taken place in the quality assurance environment. These can be categorised under the headings of: globalisation and changes in educational technology; international recognition of qualifications; recent changes in quality assurance in other industrialised countries; new quality assurance arrangements in ‘off-shore’ education countries; the needs of Australia’s education export industry; increased accountability pressures at home; incidents with private providers and increases in the number of private providers; and complaints from applicants seeking accreditation.
25. The Modern Australian Model of quality assurance and accreditation has many strengths. It makes important distinctions between the functions of accreditation and quality assurance, and between the possible treatment of self-accrediting institutions and non-self accrediting providers. DETYA documentation specifies that quality assurance and accreditation mechanisms should satisfy a number of criteria: the mechanisms relating to self-accrediting institutions should not be solely at their discretion; there needs to be some external review or audit of the claims made by institutions about quality and standards; the mechanisms should be credible with international and domestic interest groups and be able to protect the international reputation of Australian awards; the mechanisms should help satisfy Australian taxpayers of value for money; any audit mechanism should have rigour, but at the same time be cost effective, not unnecessarily intrusive and be able to retain the cooperation of public universities; and the mechanisms should provide legal clarity for students and providers and be able to promote good practice and facilitate improvement.
26. Refinement of the current Australian model of accreditation and quality assurance would provide valuable improvements but Australia’s arrangements would still fall far behind international good practice and do little to provide additional safeguards for the education export industry, or to lend additional international credibility to Australian awards. The recently modified New Zealand Model provides for a national government agency but there is some uncertainty about how successful will be the plan for the new agency to approve various accrediting bodies. The new United Kingdom model is still developing its procedures but to date its proposals have been somewhat controversial and have yet to secure support from the well-established universities. The current VET model of accreditation and quality assurance is now well accepted in the VET sector and widely supported by industry but this model does not appear suitable for the higher education sector.
27. Important models not canvassed in DETYA documentation are the Dutch and French models of quality assurance. The Dutch model is based on a well-organised program of disciplinary reviews, while the French model uses both disciplinary reviews and institutional audits. The Dutch program is operated by the VSNU, the association representing the heads of Dutch Universities, while the French program is the responsibility of a special government agency.

Accreditation of Courses and Institutions

28. Any process of accreditation would need to be concerned primarily with: approval for new universities to operate, to use the name university, and to offer degrees and other awards; approval and accreditation of courses of study leading to degrees and other awards by other higher education providers; and re-accreditation of institutions and awards.
29. There is strong support for the proposition that accreditation is a matter for government and not the higher education sector, and that the States and Territories should continue to exercise their responsibilities in this area. Continuation and extension of the present work being undertaken by the MCEETYA Multilateral Joint Planning Committee seems likely to produce a professional national approach to accreditation.
30. To date the Multilateral Committee has put most of its efforts into developing a common approach to the accreditation of universities. This has raised issues about the distinguishing characteristics of universities today in Australia and what criteria should be used in accrediting new and overseas universities. There differences in views over use of the titles of ‘university’ and ‘degree’. State and Territory officials see value in maintaining and strengthening current controls over these titles.
31. Perhaps more important for the Multilateral Committee will be to develop uniform protocols for the recognition of new and overseas universities and agreement on the criteria to be applied. There appears to be agreement that criteria should include topics such as financial viability, the legal basis of the institution, and the processes of governance, internal quality assurance and accountability but less agreement about whether the criteria should include quantitative indicators with regard to staff, buildings and facilities, and library holdings and specialised laboratories. Other issues that need attention include: protocols and procedures for the accreditation of institutions other than universities; whether the recognition of new and overseas universities should automatically carry with it the rights of self-accrediting powers; whether all institutions need some form of accreditation before their courses can be accredited; requirements with regard to ‘out-state’ Australian institutions operating in other States and Territories; whether accredited institutions should be required to seek special approval to offer courses to international students at special international student campuses; whether universities and other self accrediting institutions need special approval to enter into franchise arrangements to offer higher education courses with non accredited institutions such as VET providers; and whether legislation in all States and Territories should provide for both the accreditation of institutions and courses.

Quality Assurance and Improvement Plans

32. Continuation and strengthening of the current requirements of the Commonwealth with regard to institutional quality assurance and improvement plans appears to be a well-conceived and sensible strategy. Good management practice requires that all institutions should have in place appropriate quality assurance and improvement plans, and submission of these to some outside body provides useful discipline for institutions to keep plans up to date. Such plans should cover all major aspects of operations, including ‘off-shore’ efforts and distance education offered internationally.
33. While some universities have made major advances in benchmarking, this development has not been uniform across the sector and that it may be helpful to provide additional assistance. Institutions should be encouraged to document in detail their monitoring and quality assurance procedures for ‘off-shore’ efforts.
34. With regard to quality assurance plans currently required by DETYA, it will be important that discussions take place with any new quality assurance agency to ensure that DETYA requirements do not in conflict with any documentation requirements for institutional audits.

Quality Audits and a New Quality Agency

35. A non-intrusive and sensibly conceived quality assurance mechanism involving both the higher education sector and the State and Territories seems likely to attract considerable support. Certainly there is wide appreciation of some of the strong influences that require establishment of a new national mechanism.
36. One important issue is the legal basis of any new agency, the governance structures to be employed and its accountability arrangements. In our discussions four main models were identified: a Ministerial Committee set up by the Commonwealth Minister; an agency established by Commonwealth legislation; an agency established by joint Commonwealth and State legislation; and an agency set up as a company, possibly responsible to MCEETYA.
37. Our preferred model for the new quality assurance mechanism and agency is as follows:

• A new quality assurance mechanism should be established as a joint Commonwealth, State/Territory, and higher education initiative with the aim of strengthening public accountability, protecting academic standards and the reputation of Australian higher education providers and awards, and promoting good practice in quality assurance. We suggest that the new mechanism should be called the Higher Education Quality Assurance Council.
• The central function of the Council will be conduct of program of institutional reviews or audits. Review teams will carry out site visits, following completion of self-assessments carried out by institutions, which will include reviews of the processes of managing quality including monitoring performance and benchmarking. Institutions will provide review teams with a report of their self-assessments, together with documentation on institutional mission and objectives, quality assurance and improvement plans, details on methods used to monitor and benchmark achievements and the results of monitoring and benchmarking. Participating institutions will be reviewed every five years.
• The Council will be established an independent agency, at ‘arms length’ from both government (Commonwealth and State) and from the higher education sector. It will be governed by a board consisting of an independent Chair, two Commonwealth nominees, two members representing the States and Territories, two representatives of the higher education sector and one representative drawn from those professional associations involved in accreditation within the higher education sector. The Executive Director will be an ex-officio member and the board will have the power to coopt up to two additional members with special expertise in academic audits and assessment. Commonwealth representatives will be appointed by the Minister for Education, Training and Youth Affairs, while the two State and Territory representatives will be appointed by MCEETYA. Members will serve four year terms.
• Funding for the work of the Council will come from annual grants from the Commonwealth and from the States and Territories, and annual membership fees paid by individual higher education institutions who wish to participate in the program of reviews.
• The terms of reference of the Council will be as follows:
  • to review within participating higher education institutions the mechanisms for quality assurance, monitoring performance and academic standards, and enhancing quality;
  • to publish the reports of reviews;
  • to report publicly from time to time on the effectiveness of quality assurance procedures in participating institutions, the extent to which procedures ensure academic standards and reflect good practice in maintaining and improving quality, and other relevant matters;
  • to identify and disseminate good practice in quality assurance in higher education;
  • to undertake and sponsor studies related to effective quality assurance management practices and academic standards in higher education.
• In carrying out reviews, review teams appointed by the Council will focus particularly on:
  • appropriateness of quality assurance and improvement plans in relation to institutional contexts and missions;
  • rigour of the mechanisms employed to review courses and academic organisational units, and monitor performance against institutional plans;
  • effectiveness in monitoring outcomes and in benchmarking, both nationally and internationally; and
  • success in communicating the results of the monitoring outcomes and academic standards to stakeholders.

Review teams will report to the Council.

• Each year the Council will draw up a program of reviews for the following year, after consultation with institutions likely to be reviewed.
• Review panels, generally of no more than five members, will be appointed by the Council. Members of review teams will be drawn from the higher education sector, the Commonwealth and the States. Members may also be drawn from the professions and professional associations, and from business and industry. Review panels will normally visit institutions for two consecutive days after the institution has completed a self-assessment and supplied other documentation as required. Institutions offering courses ‘off-shore’ for international students should document in detail the procedures followed for safeguarding and monitoring quality, and the results of any assessments.
• Following the visit of the review team, the draft report will be forwarded to the institution for comment. Once the report is completed it will be considered by the Council and then published. Copies will be provided free to DETYA, State and Territory accrediting agencies, all participating higher education institutions, and relevant professional associations. For each review, a single report will be prepared and published.
• Should a review reveal serious weaknesses, the institution concerned will be given up to 12 months to correct weaknesses prior to a supplementary review. Failure to rectify weaknesses would be a matter for DETYA to address (in the case of Commonwealth funded institutions) or for the relevant State or Territory accrediting agency. One possible action would be to remove the name of the institution from the AQF list of accredited institutions until such time that as minimum standards are achieved.
• Every effort should be made to encourage private universities and non self-accrediting institutions to participate in the review program.
• Prior to arrangements for the Council being finalised, the higher education sector should be consulted about the proposed terms of reference for the Council, the composition of the Council’s board and the method of conducting reviews.