STANDING COMMITTEE ON PUBLIC ACCOUNTS

MEMBERSHIP LIST

1^st Session, 41^st Parliament

Ernie Hardeman

Chair

Lisa Macleod

Vice-Chair

PEGGY SATTLER regularly served as a substitute member of the Committee.

William Short

Clerk of the Committee

Erica Simmons
Research Officer

Contents

Preamble   1

Acknowledgements   1

Background   1

Follow-up Audit, 2014  2

Status of Actions Taken on Auditor's Recommendations: Summary  2

Issues Raised in the Follow-Up Audit and Before the Committee   2

Consolidated List of Committee Recommendations   4

Preamble

On Wednesday April 1, 2015, the Standing Committee on Public Accounts held public hearings on the follow-up audit (section 4.11 of the Auditor General's 2014 Annual Report) of University Undergraduate Teaching Quality administered by the Ministry of Training, Colleges and Universities.

The Committee endorses the Auditor's findings and recommendations, and presents its own findings, views, and recommendations in this report. The Committee requests that the Ministry provide the Clerk of the Committee with written responses to the recommendations within 120 calendar days of the tabling of this report with the Speaker of the Legislative Assembly, unless otherwise specified.

Acknowledgements

The Committee extends its appreciation to officials from the Ministry of Training, Colleges and Universities (the Ministry) as well as representatives of Brock University, the University of Ontario Institute of Technology, and the University of Toronto. The Committee also acknowledges the assistance provided during the hearings and report writing deliberations by the Office of the Auditor General, the Clerk of the Committee, and staff in the Legislative Research Service.

Background

A value-for-money audit of university undergraduate teaching quality was reported on in the Auditor's 2012 Annual Report. The objective of the audit was to assess the extent to which the Ministry and selected Ontario universities support, assess and periodically report meaningful performance information on the quality of  instruction provided to undergraduate students. A follow-up audit was published in the 2014 Annual Report.

The 2012 audit found that although the universities visited by the Auditor were not formally measuring and reporting on teaching quality, all three institutions already collected information that could be useful in the measurement of teaching quality. All universities, including the three audited, ask undergraduate students to evaluate their courses and professors, but only one in four of Ontario's 20 publicly-assisted universities make the results of these evaluations available to students to assist them in making their course selections.

Although all three universities audited provided annual performance appraisals of full-time faculty, these appraisals did not always provide substantive feedback on teaching performance or encouragement to improve teaching performance where warranted. None of the three universities provided performance appraisals to sessional (contract) instructors.

Follow-up Audit, 2014

Status of Actions Taken on Auditor's Recommendations: Summary

In 2014, the Ministry and the three universities visited by the Auditor indicated that progress had been made in implementing  many (70%) of the recommendations in the 2012 Annual Report, which were focused on maintaining and enhancing teaching quality. The Auditor noted that "universities need to better ensure that teaching quality is valued, encouraged and rewarded."[1] The 2014 follow-up audit found that a few of the recommendations from the 2012 audit had been fully implemented by at least one of the universities visited. Of note:

· One university implemented, in some of its divisions, an online course evaluation system with results made available to most students, allowing them to make more informed decisions when selecting courses.

· All three universities increased the professional development opportunities available to faculty.

· Further progress is needed in evaluating the use and performance of sessional instructors.

· The Ministry started collecting additional data as part of the Ontario University Graduate Survey and had started to publish additional results on graduate employment outcomes. However, this data was only published at an aggregate provincial level and was not available at the university or program level to enable students to make more informed decisions on university and program selection.

Issues Raised in the Follow-Up Audit and Before the Committee

Ministry representatives told the Committee that with 66% of Ontarians now holding post-secondary credentials, the government has made significant progress towards the goal of having 70% of Ontarians hold post-secondary education credentials by 2020. In addition, the Ministry has negotiated and signed "strategic mandate agreements" with all publicly assisted post-secondary institutions. Teaching and learning are key components of these agreements and are explicitly linked to the quality of student experience.

Ministry representatives emphasized that the Ministry does not deliver education but plays a role in oversight, funding, and accountability of the sector.

The Higher Education Quality Council of Ontario (HEQCO), an agency of the Ministry, is working with post-secondary institutions on student learning outcomes. To improve the collection of student-level data, the Ministry is working with post-secondary institutions to implement the Ontario Education Number (OEN), which is a unique identifier assigned to each student.

University representatives noted their enhanced focus on teaching excellence in recent years. All three universities have teaching and learning centres that offer workshops and professional development to faculty members and other course instructors wishing to improve or adapt their teaching skills, develop new pedagogy, or incorporate technology in the classroom. In addition, the creation of a new category of teaching-stream faculty has helped to reduce reliance on part-time and sessional instructors at one university. Universities regard teaching effectiveness as an important component of tenure and promotion assessments.

Committee members noted the importance of looking at student outcomes after graduation, and Ministry representatives explained that the use of key performance indicators is an important tool for assessing these outcomes. Since February 2015, the Ministry has made more data available and is collecting and publishing data on these outcomes by institution in 26 program categories. This data looks at whether graduates consider their areas of study and skills acquired at university to be relevant to their employment after graduation. Committee members noted that the Baccalaureate Graduates Survey, which measures employment and further education outcomes for graduates of universities in British Columbia, is a useful approach.

University representatives discussed some of the many approaches used to improve the student experience at the undergraduate level, ranging from experiential and work-integrated or co-op learning, to ensuring that first year students experience small group seminars. Every university program in Ontario is subject to periodic evaluation by the Ontario Universities Council on Quality Assurance (the Quality Council). The cycle of review should not exceed eight years.

The Committee noted the Auditor's recommendations regarding performance feedback for sessional instructors. One university representative explained that every course offered, whether taught by a full-time faculty member or a sessional instructor, is evaluated each time it is given. Across the three universities, the proportion of courses taught by sessional instructors ranges from 14% to 33%. The Committee noted the importance of providing performance evaluations and feedback on teaching effectiveness to instructors, whether contract or full-time.

Consolidated List of Committee Recommendations

The Standing Committee on Public Accounts recommends that:

1. Universities continue to take steps to make the results of course evaluations available to students to assist them in making their course selections.
2. Universities conduct performance appraisals of sessional instructors; and examine the impact on teaching quality of the use of sessional instructors.
3. Universities provide substantive feedback to full-time faculty on teaching performance, and encouragement to improve teaching performance where warranted.
4. The Ministry of Training, Colleges and Universities identify effective tools for measuring employment and further education outcomes for graduates of Ontario universities.
5. The Ministry of Training, Colleges and Universities make data on graduate employment outcomes at the program and university level publically available to assist students in making informed decisions on university and program selection.