Standing committee on public accounts

employment ontario

(Section 3.04, 2016 Annual report of the office of the auditor general of ontario)

2^nd Session, 41^st Parliament
66 Elizabeth II

ISBN 978-1-4868-1074-1 (Print)
ISBN 978-1-4868-1076-5 [English] (PDF)
ISBN 978-1-4868-1078-9 [French] (PDF)
ISBN 978-1-4868-1075-8 [English] (HTML)
ISBN 978-1-4868-1077-2 [French] (HTML)

The Honourable Dave Levac, MPP
Speaker of the Legislative Assembly

Sir,

Your Standing Committee on Public Accounts has the honour to present its Report and commends it to the House.

Ernie Hardeman, MPP
Chair of the Committee

Queen's Park
December 2017

Standing committee on public accounts

Membership list

2^nd Session, 41^st Parliament

ernie hardeman

Chair

lisa macleod

Vice-Chair

Bob Delaney percy hatfield

Vic Dhillon randy hillier

han dong monte kwinter

john fraser

PEGGY SATTLER regularly served as a substitute member of the committee.

katch koch

Clerk of the Committee

erica simmons

Research Officer

Contents

Introduction 1

Acknowledgements 1

Background 1

Skilled Trades and Apprenticeships 2

Funding 3

Audit Objectives and Scope 3

Main Points of Audit 4

Issues Raised in the Audit and Before the Committee 5

Finding full-time employment 5

Funding for the Employment Service program 6

Overdue loans to apprentices 6

Overpayments to clients 7

Evaluating service providers 7

Ministry follow-up on action required by service providers 8

Apprenticeship program completion rates 9

Examination preparation initiative 10

Financial incentives for employers 11

Monitoring quality of apprenticeship programs 11

Alternative pathways to apprenticeship training 12

Data on labour needs 12

Public reporting of Employment Ontario outcomes 13

Duplication of employment and training services 13

Consolidated List of Committee Recommendations 15

Introduction

On May 3, 2017, the Standing Committee on Public Accounts held public hearings on the audit (Section 3.04 of the Auditor General's 2016 Annual Report) of Employment Ontario administered by the Ministry of Advanced Education and Skills Development.

The Committee endorses the Auditor's findings and recommendations, and presents its own findings, views, and recommendations in this report. The Committee requests that the Ministry provide the Clerk of the Committee with written responses to the recommendations within 120 calendar days of the tabling of this report with the Speaker of the Legislative Assembly, unless otherwise specified.

Acknowledgements

The Committee extends its appreciation to officials from the Ministry of Advanced Education and Skills Development and from the Ontario College of Trades. The Committee also acknowledges the assistance provided during the hearings and report-writing deliberations by the Office of the Auditor General, the Clerk of the Committee, and staff in the Legislative Research Service.

Background

Employment Ontario provides employment and training programs and services for job seekers, related information for both job seekers and employers, apprenticeship training to students seeking certification and employment in a skilled trade, and literacy and numeracy skills to people who lack basic education necessary for employment. Services for employers include providing additional training to their employees, placing job postings on the Ministry's job bank website, applying for incentives to hire people, and training apprentices in skilled trades.

These programs and services are funded by the Ministry of Advanced Education and Skills Development (Ministry), formerly known as the Ministry of Training, Colleges and Universities, and the majority are delivered by third-party agencies. As of March 31, 2016, Employment Ontario offered 27 programs and services via 400 third-party (mostly not-for-profit) service providers at about 740 service delivery sites. Service providers include community-based employment service providers, literacy providers such as publicly-funded school boards, colleges of applied arts and technology, and other non-college apprenticeship training delivery agents such as unions, employment associations, and large employers.

The 27 programs and services are clustered under four categories:

· Employment and Training – ten programs and services providing supports to job-seekers and grants for employers.

· Apprenticeship – nine programs and services ensuring workers receive the combination of workplace and classroom training needed to become certified and employed in a skilled trade.

· Foundational Skills – two programs and services offering literacy and numeracy skills upgrading for those lacking the necessary basic education for employment, and bursaries for internationally-trained professionals completing postsecondary education programs to learn Canadian standards applicable to their professions.

· Labour Market – six programs and services providing employment planning and capacity building at the community level and employment services following large-scale layoffs.

Employment Ontario is administered by the Ministry's Employment Training Division (Division), which manages four regional offices (Central – Toronto; Eastern – Ottawa; Western – London; Northern – Sudbury), which are further divided into 39 local field offices. In total, the Division has over 800 staff. The Ministry develops policy for adult education and labour market training. It also works with the Ontario College of Trades to set standards for occupational training, such as trade certification and apprenticeships.

Skilled Trades and Apprenticeships

Ontario has 156 skilled trades in four main sectors:

· construction (e.g., electricians and plumbers);

· motive power (e.g., automotive mechanics);

· industrial (e.g., welders, tool and die makers); and

· service (e.g., hairstylists).

Apprenticeship combines on-the-job and in-class training in a skilled trade. The certification process differs by trade and is intended to ensure that an apprentice has the technical knowledge and hands-on skills to meet industry standards.

People working in a compulsory trade must be certified through a final examination process in order to practice legally in Ontario. In Ontario 22 of the skilled trades (mainly in the construction and motive power sectors) are designated as compulsory.

People working in some voluntary trades (such as general carpenters or industrial electricians) may also choose to get certified but it is not a legal requirement.

The Ontario College of Trades (College) regulates the skilled trades, including how and what people must learn during an apprenticeship (i.e., on-the-job training requirements and curriculum for in-class training).

A person must be a member of the College's Apprentices Class to work as an apprentice in an Ontario apprenticeship program established by the College. To be a member, one must hold a Registered Training Agreement with the Ministry.

The Ministry issues a Certificate of Apprenticeship upon completion of all requirements (on-the-job and in-class training) of the apprenticeship program for the given trade.

The Ministry issues a Certificate of Qualification to apprentices who write and pass a final certification exam given by the Ministry on behalf of the College of Trades for their trade and register with the College as a journeyperson.

Funding

In 2015/16, the Division spent $1.3 billion (of which approximately two-thirds or $841.1 million came from the federal government) to support employment programming. (Some of this programming had previously been delivered in Ontario by the federal government.) The Ministry receives federal funding under three separate agreements with the federal government. The agreements include reporting requirements such as audited financial statements, annual plans outlining priorities and planning activities, performance targets, and results.

Nearly 90% of the total transfer payments provided by Employment Ontario to third-party service providers are for programs and services under the Employment and Training, and Apprenticeship categories.

Audit Objectives and Scope

The audit assessed whether the Ministry has effective systems, processes, and procedures in place to

· ensure that programs and services are being delivered in accordance with established program requirements;

· ensure that the Ministry and its delivery agencies are providing programs and services to clients in an economical and efficient manner; and

· measure and report on the effectiveness of the programs in meeting their objectives.

The audit focused on the major programs offered within the Employment and Training, and Apprenticeship categories. Audit work was primarily conducted at the Ministry's corporate office, two of its four regional offices, and six local field offices in those regions.

The two regional offices selected for detailed audit work (Central and Western regions) collectively served 78% of employment service clients and 74% of active apprentices in 2014/15. These regions also accounted for 72% of transfer payments for employment services and 60% of transfer payments for apprenticeship training.

Main Points of Audit

The Auditor found that

key programs offered by Employment Ontario are not effective in helping Ontarians find full-time employment. Although the Ministry is redesigning some of its existing programs, more attention is needed to increase their effectiveness and improve efficiency.

In addition, the Auditor explained that the Ministry

lacks the detailed and timely labour market information necessary to both improve existing programs and develop new ones to meet the current and future labour needs of Ontario.

Significant issues found in the audit include the following:

· The majority of employment and training program clients are unsuccessful in finding full-time employment in their chosen career.

· Overpayments to clients who do not complete programs are not being recovered.

· Less than half of those who begin an apprenticeship program in Ontario complete it.

· The Ministry needs to better analyze and address reasons for low apprenticeship completion rates.

· Financial incentives to employers may not be encouraging apprenticeship certification.

· The number of apprentices at risk of non-completion remains high even after implementation of a monitoring strategy.

· The Ministry's monitoring of apprenticeship training is limited.

· The Ministry lacks necessary data to ensure that Employment Ontario programs meet current and future labour needs.

Issues Raised in the Audit and Before the Committee

Significant issues were raised in the audit and before the Committee. The Committee considers the issues below to be of particular importance.

Ministry representatives told the Committee that last year, the Employment Ontario network helped more than 8,600 Ontarians through Second Career funding, more than 42,000 learners through the Literacy and Basic Skills Program, and more than 104,000 students find summer jobs. More than 25,600 people registered as apprentices. Through the Rapid Re-employment Training Service, Employment Ontario responded to more than 125 layoffs affecting 9,700 employees.

At the time of the Auditor's report, the Ministry was working on a long-term transformation agenda to modernize and integrate Ontario's employment and training programs, and to improve the efficiency of Employment Ontario program delivery. Following the recommendations made by the Premier's Highly Skilled Workforce Expert Panel, the Ministry is also putting into place a strategy to help the workforce adapt to the current and future demands of a technology-driven, knowledge-based economy.

The Ministry is inviting apprentices, employers, training delivery agents and other partners to participate in a facilitated discussion on apprenticeship modernization. The goal will be to develop an action plan that will result in an improved experience for apprentices and employers, in support of building Ontario's highly skilled workforce.

Ministry staff noted that jurisdictions across Canada and the United States are also struggling with how to retrain laid-off workers for jobs in the new knowledge-based economy. Many of these workers do not have the educational background needed to access these jobs, and are not in a position to leave the labour force for an extended period in order to attend education programs. The aim of every Ministry program is to maximize opportunities for good employment.

The Committee heard that Employment Ontario receives 70% of its funding from Ottawa. Because this funding is derived primarily from Employment Insurance (EI) monies, the federal government requires the programs it funds to serve EI recipients. However, Ministry staff explained that 70% of Employment Ontario clients are not receiving EI. The Ministry is requesting more flexibility to make it possible to tailor programs to best meet the needs of Ontarians.

Finding full-time employment

The Auditor found that, based on Ministry data, employment and training programs and services resulted in relatively few clients finding full-time employment, or employment in their field of training.

Committee Recommendations

The Standing Committee on Public Accounts recommends that:

The Ministry of Advanced Education and Skills Development should establish specific outcome measures and associated targets for each Employment Ontario program, and take corrective action where program outcomes do not meet targets.
The Ministry of Advanced Education and Skills Development should ensure follow-ups are conducted with Employment Ontario program participants at three, six, and 12 months after receiving services.

Funding for the Employment Service program

Funding for third-party service providers of the Employment Service program is determined by the targeted number of clients to be served, labour market indicators, and location indicators. However, the Auditor found that the Ministry is using indicators from 2009/10 and some service providers are consistently serving fewer clients than they are funded for.

The Committee heard that in November 2017, newly released census data on the labour market will be incorporated into Employment Ontario's calculations. Also, Ontario's Highly Skilled Workforce Strategy includes the development of more local, relevant, and timely labour market data. The Ministry's Ontario Job Futures publication will be updated to provide outlooks from 2017 through 2021 and, beginning in fall 2017, phased improvements to the labour market information website will include regional and local content.

Committee Recommendations

The Standing Committee on Public Accounts recommends that:

The Ministry of Advanced Education and Skills Development should

a) regularly update the labour market information (with a focus on regional labour markets) used to determine funding allocations to service providers; and

b) ensure that funding to service providers reflects the actual level of services provided.

Overdue loans to apprentices

The Ministry provides interest-free loans ($300-$800) to first-year apprentices for the purchase of tools, repayable within one year after obtaining certification or within six months of withdrawing from the program. The Ministry has provided $32.8 million in loans since the program began in 1998. As of March 31, 2016, $18.2 million in loans was outstanding but the Ministry did not have reliable information about how much of the outstanding loan balance is overdue.

The Auditor had recommended that the Ministry "should proactively monitor apprentices' status in the program to quickly identify the date they either complete or withdraw from the program." The Committee learned that the Ministry is working to improve the repayment rate for loans for tools by linking information about loan recipients to apprenticeship completion status in the Employment Ontario Information Systems database.

Overpayments to clients

The Auditor found that in the last three fiscal years (2013/14 to 2015/16) $30.1 million (6% of total program funding) in overpayments was forwarded for collection to Ontario Shared Services (Ministry of Government and Consumer Services). During that same period, Shared Services wrote off $26.6 million in overpayments to Second Career clients.

The Committee heard that beginning in February 2017, the Ministry increased its reporting requirements to ensure that Ministry staff conduct their monitoring and reconciliation duties in order to reduce overpayments to Second Career clients. The Ministry is exploring the use of improved information technology supports in order to better manage client cases and reduce the number and level of potential overpayments to clients. A number of changes have also been made to the Ministry's information systems and business processes. For example, the Ministry now reconciles receipts quarterly instead of at the end of a contract to reduce the number and level of potential overpayments.

Moving forward, the Ministry plans to review the impact of these changes and analyze the need for additional measures. The Ministry will continue to consult with its partners on the feasibility, and potential impact on clients, of requiring receipts before providing funding.

Committee Recommendation

The Standing Committee on Public Accounts recommends that:

The Ministry of Advanced Education and Skills Development should assess the advantages of providing funding to clients in advance of getting receipts for only the initial one or two months of instalments, and then require receipts prior to providing funds for remaining instalments.

Evaluating service providers

The Committee asked how the Ministry is evaluating service providers and heard that risk assessments of service providers are conducted as part of ongoing risk management processes. This helps to inform the Ministry when service providers are underperforming and action needs to be taken. The action taken may involve either a directed improvement process or an official review process.

The Committee asked about the sharing of best practices. Ministry staff explained that a service delivery advisory group representing a cross-section of the service provider network enables the Ministry to do both formal and informal consultations including sharing best practices.

Committee Recommendation

The Standing Committee on Public Accounts recommends that:

The Ministry of Advanced Education and Skills Development should incorporate long-term outcomes of clients' employment or training status into the measure of service provider effectiveness.

Ministry follow-up on action required by service providers

The Committee asked how the Ministry is following up on noncompliance or underperformance by service providers. Ministry staff explained that letters have been issued to service providers whose program outcomes did not meet the provincial service quality standards for the Employment Service program and the Literacy and Basic Skills Program. The Ministry is exploring whether to expand these actions to more programs, such as the Youth Job Connection, and to provide more training to staff to ensure consistent use of these monitoring frameworks.

The Ministry also took corrective action for issues of noncompliance related to following up with clients, late reporting, and inaccurate financial forecasting. In total, 96 service providers were placed on official review in 2015/16, and they were required to provide the Ministry with action plans to address their own noncompliance. Failure to do that could result in a wind-down of or amendments to their contracts.

The Ministry is currently reviewing its approaches to monitoring, and consolidating its program accountability frameworks to inform the development of provincial strategies. A first-phase environmental scan has been completed. Later this year, the Ministry will implement new system-wide processes to track quarterly activities, highlight risk levels, and allow for documented, strategic follow-up with service providers.

A review of efficiency targets for the Employment Service is currently underway, and once the review is complete, the Ministry will implement parameters for service providers to set efficiency targets for contracts, effective for the next contract timeframe (April 2018).

Committee Recommendations

The Standing Committee on Public Accounts recommends that:

The Ministry of Advanced Education and Skills Development should employ enhanced monitoring methods for all service sites that fail to meet either the minimum Provincial quality standard or their targeted service quality scores; and ensure corrective action is taken within established timelines.
During monitoring of service providers, the Ministry of Advanced Education and Skills Development should identify any common deficiencies and address these system-wide.

Apprenticeship program completion rates

The Auditor found that the average completion rate for apprentices in Ontario in the five-year period from 2011/12 to 2015/16 was 46% for a Certificate of Apprenticeship and 47% for a Certificate of Qualification. Completion rates were substantially higher (59%) for those training for a compulsory trade (in which certification is a requirement of employment) than for a voluntary trade (35%).

The Committee heard that the Ministry has initiated a review to update the apprenticeship workplace monitoring strategy. The objectives of the strategy are to minimize the number of apprentices who become at risk for non-completion, and to strengthen relationships with sponsors. A number of strategies for apprenticeship completions have also been implemented including financial incentives for progress and completion, and monitoring sponsors with apprentices who are not progressing in their training and apprentices who are between sponsors.

The Ministry has also introduced new monitoring pilots to help apprentices who need essential skills upgrading. For example, if apprentices need help with numeracy and literacy, they will be directed to the Literacy and Basic Skills Program. If apprentices are having difficulty finding a job, they will be directed to the Employment Service program. In addition, the Ministry is reviewing funding and supports for both compulsory and voluntary trades, to determine their impact on completion rates.

In 2013, the Ministry developed the Apprenticeship Monitoring Strategy to identify and follow up on those apprentices considered at high risk of non-completion. However, the Auditor found that the number of apprentices at risk of non-completion remains high despite the implementation of this strategy. The Ministry is re-examining the monitoring methods it uses to identify those apprentices most at risk of not completing a program.

In order to improve training for apprentices, the Ministry is working with colleges to review the in-class student engagement and satisfaction survey. The survey findings will be used to improve the training as well as survey administration. The annual apprenticeship survey is being reviewed to ensure that it includes mandatory questions that will offer some insight into why apprentices withdraw from their programs.

The Ministry agrees with the Auditor that it would be helpful to compare Ontario's apprenticeship program with other jurisdictions and to develop a standard methodology across Canada for calculating apprenticeship completion rates.

The Committee also heard that many apprentices do not need a licence to practice their trade and may, if offered a job, decide not to complete the apprenticeship.

Committee Recommendations

The Standing Committee on Public Accounts recommends that:

The Ministry of Advanced Education and Skills Development should

a) develop methods to gain more insight into the factors causing apprentices to withdraw from apprenticeships and implement strategies to address these factors;

b) implement strategies to improve completion rates for apprentices in both compulsory and voluntary trades;

c) evaluate whether it should change the funding allocated to apprenticeship training in voluntary trades; and

d) take a leadership role in working with other provinces to develop a standard methodology for calculating apprenticeship completion rates across Canada.

Examination preparation initiative

In 2010/11, the Ministry began funding examination preparation courses in six high-demand skilled trades to prepare apprentices for their final certification exam. Ministry staff told the Committee that the strategies implemented to improve apprenticeship completion rates include expanding exam preparation courses and making exam preparation a mandatory component for 11 of the trades.

Committee Recommendations

The Standing Committee on Public Accounts recommends that:

9. The Ministry of Advanced Education and Skills Development should

a) evaluate the outcome of expanding the examination preparation course to more high-demand trades and, if positive results are found, further expand it to other compulsory trades;

b) consider making the examination preparation course mandatory for apprentices who have previously failed their trade certificate exam; and

c) review and adjust funding for the examination preparation course to ensure it is comparable to rates paid to training delivery agents for regular in-class training courses.

Financial incentives for employers

While the Province offers a number of incentives to employers to hire and train apprentices, the Auditor concluded that these incentives "are not aligned with the goal of improving apprenticeship completion rates." The incentives include the Apprenticeship Training Tax Credit ($15,000 in total per apprentice for the first
36 months of training), a signing bonus ($2,000 per apprentice paid within six months of registering), and a completion bonus ($1,000 per apprentice paid at time of completion).

The Committee heard that the Ministry recently undertook employer engagement that focused on financial supports in the apprenticeship system. The Ministry is focusing on the use of tax credits to encourage businesses to help apprentices gain the certification and skills they need to maximize their own earnings, improve their labour market mobility, and contribute effectively to the economy. The Ministry is now working with the Ministry of Finance to review the Apprenticeship Training Tax Credit.

Committee Recommendation

The Standing Committee on Public Accounts recommends that:

10. The Ministry of Advanced Education and Skills Development should redesign the financial incentives offered to employers in order to encourage both program registration and completion.

Monitoring quality of apprenticeship programs

The audit found that the Ministry had not developed specific policies or guidelines for ongoing monitoring of on-the-job training, nor does it generally monitor the quality of in-class training unless there is a complaint. In addition, although the Ministry's monitoring of the apprenticeship program is conducted primarily through surveys, the information collected was of limited value. For example, for more than half of the questions in the Apprenticeship Survey the Ministry did not compile and analyze survey results separately for those who completed their apprenticeship program and those who withdrew.

Committee Recommendations

The Standing Committee on Public Accounts recommends that:

11. The Ministry of Advanced Education and Skills Development should

a) implement policies and guidelines for ongoing monitoring of on-the-job and in-class training of apprentices;

b) regularly analyze completion rates by training delivery agents and employers to identify trends and take corrective action; and

c) identify and address factors that may be preventing apprentices from passing the final qualification exam.

Alternative pathways to apprenticeship training

The audit found that three Ministry programs designed to expand access to apprenticeship training have not been as effective as the Ministry originally expected. The Ministry said that these programs are currently being redesigned and will be targeted to those individuals mostly likely to pursue a registered apprenticeship. In June 2016, the Cabinet accepted recommendations for addressing some of the barriers to apprenticeship participation that were tabled by the Premier's Highly Skilled Workforce Expert Panel.

Data on labour needs

The audit noted that although the Ministry collects labour market information, it lacks the regional data on labour force supply and skills demand "needed to make effective decisions with respect to setting priorities and targeting funding in skills training and education." While the Ministry provides more than $6 million to the network of 26 local community-based Workforce Planning Boards across the province to assess local labour market conditions, the Auditor found that this information is not factored into funding or programming decisions of the Employment Service program.

The Committee asked how the Ministry is ensuring that Employment Ontario programs are more responsive to labour market needs. Ministry representatives told the Committee that they are exploring some new and enhanced sources of labour market information, for example, real-time information on job postings at the community level. The Ministry is also assessing the feasibility of using regional labour market projections from an external service provider to create information resources for the public.

Committee Recommendations

The Standing Committee on Public Accounts recommends that:

12. The Ministry of Advanced Education and Skills Development should

a) regularly collect forecast labour force data by region and occupation, factoring in new graduates and migration trends, and use longer projected forecasts (such as ten years); and

b) evaluate the work of the Workforce Planning Boards, and use the findings of the evaluations of the Local Employment Planning Councils pilot project, in informing decision-making, and take corrective action where needed.

Public reporting of Employment Ontario outcomes

The Auditor noted that while the Ministry publicly reports on a number of outcomes for its Postsecondary Education Program, there is little public reporting of Employment Ontario outcomes.

The Ministry has started to release employment and training data to the public through the Open Data Directive. In February 2017, the Ministry released its Employment Ontario geo hub, which provides interactive data that includes outcomes for a variety of programs, divided up by the 26 local Workforce Planning Boards and the eight Local Employment Planning Councils (pilot) across Ontario. Data will soon be publicly released on apprenticeship registrations and completions, with further apprenticeship datasets released later in 2017.

The Ministry conducts in-depth evaluations to assess the performance and outcomes of Employment Ontario programs. Evaluations of the Literacy and Basic Skills Program and the Second Career program were publicly released in April.

Committee Recommendations

The Standing Committee on Public Accounts recommends that:

13. To assist job-seekers and those considering apprenticeships and/or education for employment purposes, the Ministry of Advanced Education and Skills Development should

a) establish yearly reportable outcome measures for employment and skills development programs; and

b) publicly report information useful to job-seekers and those seeking skills training, apprenticeships, or upgrading such as the number of Employment Service clients who find employment in (or outside) their fields, or who take further training, as well as apprenticeship pass rates and the percentage of apprentices who find employment in their fields.

Duplication of employment and training services

The Auditor found that almost 40% of the service providers funded by the Ministry of Citizenship, Immigration and International Trade were also funded by the Ministry of Advanced Education and Skills Development to provide similar services.

Staff explained that the Ministry is committed to working with other ministries to avoid duplication. A representative of the Ontario College of Trades noted that that the College shares responsibilities for apprentices with the Ministry, and both are working to streamline the registration process.

Committee Recommendation

The Standing Committee on Public Accounts recommends that:

14. The Ministry of Advanced Education and Skills Development should establish timelines for streamlining and integrating employment and training services offered by Employment Ontario and by the Ministry of Citizenship, Immigration and International Trade.

Consolidated List of Committee Recommendations

The Standing Committee on Public Accounts recommends that:

The Ministry of Advanced Education and Skills Development should establish specific outcome measures and associated targets for each Employment Ontario program, and take corrective action where program outcomes do not meet targets.
The Ministry of Advanced Education and Skills Development should ensure follow-ups are conducted with Employment Ontario program participants at three, six, and 12 months after receiving services.
The Ministry of Advanced Education and Skills Development should

a) regularly update the labour market information (with a focus on regional labour markets) used to determine funding allocations to service providers; and

b) ensure that funding to service providers reflects the actual level of services provided.

The Ministry of Advanced Education and Skills Development should assess the advantages of providing funding to clients in advance of getting receipts for only the initial one or two months of instalments, and then require receipts prior to providing funds for remaining instalments.

The Ministry of Advanced Education and Skills Development should incorporate long-term outcomes of clients' employment or training status into the measure of service provider effectiveness.
The Ministry of Advanced Education and Skills Development should employ enhanced monitoring methods for all service sites that fail to meet either the minimum Provincial quality standard or their targeted service quality scores; and ensure corrective action is taken within established timelines.
During monitoring of service providers, the Ministry of Advanced Education and Skills Development should identify any common deficiencies and address these system-wide.
The Ministry of Advanced Education and Skills Development should