Standing committee on public accounts
metrolinx – public transit construction contract awarding and oversight
(Section 3.09, 2016 Annual report of the office of the auditor general of ontario)
3rd Session, 41st Parliament
67 Elizabeth II
ISBN 978-1-4868-1787-0 (Print)
ISBN 978-1-4868-1789-4 [English] (PDF)
ISBN 978-1-4868-1791-7 [French] (PDF)
ISBN 978-1-4868-1788-7 [English] (HTML)
ISBN 978-1-4868-1790-0 [French] (HTML)
The Honourable Dave Levac, MPP
Speaker of the Legislative Assembly
Sir,
Your Standing Committee on Public Accounts has the honour to present its Report and commends it to the House.
Ernie Hardeman, MPP
Chair of the Committee
Queen's Park
May 2018
Standing committee on public accounts
Membership list
3rd Session, 41st Parliament
ernie hardeman
Chair
lisa macleod
Vice-Chair
Bob Delaney percy hatfield
Vic Dhillon randy hillier
han dong liz sandals
john fraser
katch koch
Clerk of the Committee
ian morris
Research Officer
Contents
Issues Raised in the Audit and Before the Committee 3
Canadian National and Canadian Pacific Railways 9
Consolidated list of Committee Recommendations 12
Introduction
On February 21, 2018, the Standing Committee on Public Accounts (the Committee) held public hearings on the audit (section 3.09 of the Auditor General's 2016 Annual Report) of Public Transit Construction Contract Awarding and Oversight administered by Metrolinx. Senior officials from Metrolinx and the Ministry of Transportation participated in the hearings. (For a transcript of the Committee proceedings, please see Committee Hansard, February 21, 2018.)
The Committee endorses the Auditor's findings and recommendations and presents its own findings, views, and recommendations in this report. The Committee requests that Metrolinx provides the Committee Clerk with written responses to the recommendations within 120 calendar days of the tabling of this report with the Speaker of the Legislative Assembly, unless otherwise specified.
Acknowledgements
The Standing Committee on Public Accounts extends its appreciation to officials from Metrolinx and the Ministry of Transportation for their attendance at the hearings. The Committee also acknowledges the assistance provided during the hearings and report writing deliberations by the Office of the Auditor General of Ontario, the Clerk of the Committee, and staff in the Legislative Research Service.
Background
Metrolinx is an agency of the Ministry of Transportation responsible for operating a network of train and bus routes across more than 11,000 square kilometres in the Greater Toronto and Hamilton Area (GTHA). Valued at $11 billion, Metrolinx uses about 680 km of railway track on seven train lines, 66 train stations, and 15 bus terminals spanning from Hamilton in the west, Barrie in the north, Oshawa in the east, and Lake Ontario in the south. In total, about 69 million passenger boardings occur annually on Metrolinx vehicles. Also, there are about 70,000 parking spots in 10 multi-level parking garages and 139 surface parking lots. Throughout the network, there are about 470 bridges for pedestrians and trains. Metrolinx currently owns 79% of the track it operates on, while CN owns about 10% and CP 11%.
Metrolinx was established in 2006 as a planning agency and merged in 2009 with GO Transit (GO), which had been operating the regional transit system since 1967. With this change, Metrolinx became responsible for operating, maintaining, and expanding GO's network of trains and buses. Increasing public transit capacity is a high priority for Metrolinx. Under the government's 25-year Big Move plan, announced in 2008, about $27 billion is earmarked for new public transit infrastructure over the next 10 years (2016-17 to 2025-26).
In the past five years Metrolinx has completed around 520 construction projects at a total cost of about $4.1 billion, with an average cost per project of just under $8 million. These projects included building new parking lots, expanding GO railway tracks, building tunnels and bridges for trains, and upgrading existing GO stations.
Metrolinx's construction projects proceed differently depending on the contractor it works with. Of the $4.1 billion spent over the past five years, about $3.4 billion (82%) was on projects where Metrolinx contracted out all of the work. External firms designed the project, constructed it, and oversaw it. In most cases, Metrolinx contracted with separate companies to design the project and to construct it (this is the traditional model for the delivery of construction projects).
The remaining $725 million (18%) was paid to the Canadian National Railway (CN) and the Canadian Pacific Railway (CP). GO originally used existing CN and CP track, and as demand for GO train service increased, bought as much CN and CP track and surrounding land as it could. GO also paid CN and CP to construct more track lines on their land and paid them, as per the terms of their agreement, to use the lines. This continued after Metrolinx assumed responsibility for GO, necessitating the hiring of CN or CP as the sole contractor for these projects.
Main Points of Audit
The audit found that Metrolinx does not have adequate processes in place to consistently ensure value for money in its delivery of construction projects. Because of deficiencies noted in its oversight processes around construction contracts, and deficiencies audit staff confirmed in a sample of contracts, there is a risk that Metrolinx is spending more than what is required, and a significant risk that this will continue.
Metrolinx awards contracts to poorly performing contractors that submit the lowest bids-it does not track contractors' past performance and does not consider contractors' ability to deliver completed projects on time. This has resulted in Metrolinx incurring additional costs. Metrolinx has had many years to implement a contractor performance management system but still has not done so.
For contracts with CN and CP, because Metrolinx does not verify charged costs, does not ensure that charged costs are reasonable, and does not check that the new parts it requests on a project (and pays for as new, as opposed to less expensive recycled ones) are new, it does not know that it is getting what it has paid for. It has also been paying excessively high mark-up rates charged by CN for building new rails for Metrolinx (CN's mark-up rates are specified on its invoices, while CP's are not as clear).
Issues Raised in the Audit and Before the Committee
Significant issues were raised in the audit and before the Committee. The Committee considers the issues below to be of particular importance.
Design Consultants
The audit found that Metrolinx takes little action to recover costs and prevent reoccurrences of the additional costs associated with design consultants' errors and delays. In a sample of six projects whose total initial construction costs were over $178 million, design consultants' errors and omissions cost an extra $22.5 million. No repercussions followed in these cases, and Metrolinx did not factor in this poor performance when selecting these design consultants for future projects.
In its response to the Auditor's recommendations, Metrolinx indicated that it is currently implementing a new Benefits Realization and Management program to help ensure that capital projects achieve the intended outcomes as documented by the approved business case. This program will introduce stronger alignment to the business case requirements, strengthen governance over capital project delivery, and provide stage gates at key points that will enable business sponsors to have adequate oversight and monitoring of project scope and status, as well as a robust risk management process to help optimize project outcomes. This process will be supported by a suite of performance measures for each project to help demonstrate that value for money has been achieved.
Metrolinx noted that it is in the final stages of implementing a process to ensure that reviews of consultants' work are occurring and documented in a consistent manner across the Capital Projects Group (CPG). This will provide the information needed to manage consultants' output and address design errors and omissions. To date, the Engineering and Design Standards (E&DS) group has documented the roles and responsibilities of the various CPG teams within a new procedure. This procedure has been shared in workshop sessions with staff along with re-confirming accountabilities.
As part of a suite of procedures and processes along with associated systems (Primavera Contract Manager v14–CM14), Metrolinx has refined and introduced a claims and dispute resolution procedure to enable staff and vendors to address concerns impacting a design or construction. With this process, staff have the guidelines describing how costs of errors and omissions will be recovered. With the information captured in the system, staff will be able to inform a claim and build the documentation required for recovering the costs from design errors and omissions. The CM14 system has been implemented for all Capital Projects Group (CPG), Rapid Transit (RT), and Regional Express Rail (RER) contracts. Objective-based in-class and one-on-one training on systems and associated procedures and processes, along with summarized work aids, have been provided to project teams.
In June 2015 as part of its overall Vendor Relationship Management program, Metrolinx completed the system design of a comprehensive Vendor Performance Management (VPM) program designed to capture and quantify the performance of its vendors on a contract-by-contract basis and also across the Metrolinx organization where multiple contracts exist with a particular vendor. The performance information will be used as part of the evaluation process for awarding future contracts to vendors. Since January 2015 publicly tendered contracts have included contract-specific scorecards that will be used by Metrolinx to evaluate the vendor's performance against specific Key Performance Indicators (KPIs).
Metrolinx procedures and guidelines for the VPM program have been posted on Metrolinx's external website. Training and information sessions have been conducted both for internal and external stakeholders. Metrolinx uses a computerized system to support the VPM program. The system is used to document scorecards; capture, analyze and track all VPM data on a contract-by-contract and on a per vendor basis; and to provide vendors with access to their applicable VPM information and scores. The system is currently populated with 85% of the project meta-data. The first scorecard evaluations were completed and entered into the system in September 2017.
Metrolinx issued a tender in December 2017 as a pilot project using VPM generated vendor performance data as part of the evaluation process to award the contract. Metrolinx has and continues to exercise the rights in its procurement processes to not award contracts to vendors who have poor past performance on Metrolinx or other projects using information collected as part of the reference checking process.
Nine contract management procedures have been developed to enable tracking of cost overruns and to define the roles and responsibilities of the staff involved in recovering the overrun.
The evaluation and awarding of the contract resulting from the pilot tender using VPM, and the lessons learned exercise from this pilot, are expected to be completed by the end of March 2018. At that time, project meta-data for contracts subject to Metrolinx's VPM program will be entered into the computerized system and scorecard evaluations will be completed on an ongoing basis. Starting in April 2018, tendered contracts will use vendor performance management data as part of the evaluation criteria for awarding new contracts to vendors.
In order to address project timeline concerns and delays, Metrolinx has implemented scheduling requirements to set the expectations of its vendors. Metrolinx has documented roles and responsibilities for reviewing and accepting construction contractors' baseline (planned) schedules, against which construction progress will be measured and subsequent schedule updates made through the construction period. More stringent contract provisions have been developed for inclusion in new Metrolinx-issued construction contracts going forward, requiring timely and comprehensive schedule submissions from construction contractors and promoting compliance with requirements by incorporating penalties for non-compliance. The enhanced contractor scheduling requirements and review protocols will encourage consistency across submissions from various contractors, aid interface management, and provide a tool to support early identification and resolution of issues that may affect on-time performance.
Metrolinx has communicated these new requirements within the vendor community and the requirement is now included in recent construction contracts.
With the implementation of an integrated program delivery team by end of fiscal 2017-18, Metrolinx stated that there will be more, and stronger, schedule oversight to support each project.
Committee Recommendations
The Standing Committee on Public Accounts recommends that:
1. Metrolinx provide the Committee with its most recent results, including results from 2016-17 and 2017-18, of additional costs incurred by Metrolinx attributed to errors and omissions by design consultants.
2. Metrolinx provide the Committee with results of its evaluation and lessons learned exercise from the pilot tender using Vendor Performance Management (VPM), which was expected to be completed by March 2018.
Construction Contractors
Contractors' Performance
The audit found that Metrolinx does not appear to be addressing problems caused by construction contractors that have a history of poor performance on Metrolinx projects, with the exception of two cases where Metrolinx took the past unacceptable performance of contractors into consideration.
In its response, Metrolinx stated that, beginning April 2018, it expects that tendered contracts will include vendor performance management data as part of the evaluation criteria for awarding new contracts.
Metrolinx added that it is proactively ensuring contractor safety performance by implementing the Certificate of Recognition (COR) program as a mandatory requirement on all construction procurements. COR is a leading industry sponsored safety standard that ensures the contractor has in place a comprehensive health and safety management system. Construction tenders of all values will be mandated to be COR-certified by the fourth quarter of 2017-18.
Liquidated Damages
The audit found that Metrolinx rarely takes action against contractors for tardiness. Late construction projects have created additional costs. For eight projects whose total initial budget for oversight services was $1.35 million, delays caused a further expense of over $2 million. Although Metrolinx could charge contractors "liquidated damages" (a pre-determined amount to cover additional oversight costs if a project is late), it has not always included this in its contracts. As well, based on information provided to the Auditor by Metrolinx, the agency has rarely sought action against contractors for the recovery of additional costs.
In its response, Metrolinx stated that it is developing and documenting a standard approach for the inclusion of liquidated damages (LDs) in its construction contracts where a reasonable estimate of the potential material loss can be estimated. The approach will include guidelines for project managers on how to calculate the appropriate LDs. Metrolinx noted that inclusion of LDs is a standard part of its tender templates that can be activated on a project-by-project basis. LDs will be applied on all contracts where Metrolinx is reasonably able to calculate a pre-estimate of material costs that would be incurred as a result of delays caused by the contractor. Training for 200-250 project managers and staff will be included as part of Contract Management training initiatives.
With the implementation of an integrated program delivery team by end of fiscal 2017-18, Metrolinx stated that there will be more, and stronger, schedule oversight to support each project. This will ensure teams have the right focus and resources to enable Metrolinx to realize the improvements to timely delivery.
Safety
The audit found that Metrolinx does not take action against contractors that breach safety regulations during construction. The Auditor noted that in all of Metrolinx's audits of compliance with safety regulations at construction sites over the past three years, contractors breached regulations.
In its response, Metrolinx stated that its audit process will be strengthened by requiring follow-up on all safety audits. Metrolinx noted that it has an established Construction Safety Management Program (CSMP) that includes mandatory safety training for all workers where Metrolinx is Constructor, including workers of subcontractors doing construction on rail corridors. Over 20,000 workers have been trained.
Metrolinx stressed that safety is a key Metrolinx priority. It follows a progressive safety discipline process as Constructor. The CSMP includes a three strike safety infraction process that can result in removal of workers from site on Metrolinx projects. Where Metrolinx is Constructor, it is able to audit, direct for compliance, and confirm compliance of all safety breaches.
Where a construction contractor has been procured (and Metrolinx is not the Constructor), Metrolinx includes various remedies in its contracts, including strict requirements to fix issues where safety breaches occur. As the project Owner, Metrolinx performs safety observations and audits, and provides this information to contractors to remediate in accordance with contract terms. Where safety breaches or safety incidents do occur, contractors are required to develop remedial plans and Metrolinx will conduct and document the results of follow-up audits to verify that the remedial plans have been implemented.
Metrolinx will continue to implement the Owner Construction Safety Management Program. Full implementation and quality audits for conformance to the CSMP will continue through fiscal year 2018. Metrolinx has prepared employee safety training, part of which highlights what staff can and cannot do to improve safety assurance without taking on additional liability.
Metrolinx as Owner will introduce re-audit for compliance of contracted safety provisions that are found to be in breach of contract (including both Metrolinx and regulatory requirements). Implementation will continue in fiscal year 2018 in alignment with safety training. Metrolinx has requested contractors' monthly safety data within new contracts. Going forward, Metrolinx will use this information to develop and evaluate safety trends from these indicators. This will continue to be undertaken as newly contracted works begin. Metrolinx will begin delivering employee safety training in the first quarter of fiscal year 2018.
Committee Recommendations
The Standing Committee on Public Accounts recommends that:
3. Metrolinx provide the Committee with a summary of the number of safety breaches found in its audits of contractors at construction sites over the past year.
4. Metrolinx provide an update on its Contract Management training initiatives, including its development of a standard approach for the inclusion of liquidated damages (LDs) in its construction contracts.
Deficiencies
The audit found that Metrolinx is not diligent in ensuring that contractors fix deficiencies in their work in a timely manner. In three-quarters of the projects reviewed, the Auditor noted that contractors took an average of almost eight months to fix outstanding deficiencies (much longer than the industry standard of two months).
In its response, Metrolinx indicated that it has refined and introduced the Contract Completion and Close-Out procedure, as well as a Substantial Completion procedure to address the Auditor's recommendations. These new procedures form part of a suite of procedures and processes along with associated systems (CM14).
Committee Recommendations
The Standing Committee on Public Accounts recommends that:
5. Metrolinx provide the Committee with
a) a list of projects completed over the past year and the corresponding length of time needed to fix outstanding deficiencies in each project; and
b) where projects did not meet the industry standard of two months to fix all deficiencies, provide a reason why this was not done.
6. Metrolinx provide the Committee with an update on its Contract Completion and Close-Out procedure and its Substantial Completion procedure, and explain how these procedures have assisted Metrolinx in closing out projects within the industry standard of two months.
Subcontractors
The audit found that Metrolinx has not addressed the risk of poorly performing sub-trades being selected by the contractor. The audit noted that Metrolinx allowed contractors to subcontract up to 100% of the work on their projects. Metrolinx has experienced significant issues with sub-trades-to the extent that agency staff have requested that Metrolinx pre-screen sub-trades to ensure that those with a poor work history do not jeopardize project timelines.
In its response, Metrolinx stated that it has incorporated the requirement for a Quality Management Plan (QMP) to be submitted by every consultant and contractor before undertaking any work. Metrolinx uses that submitted QMP to periodically audit against the project at phased completion of major works. QMP requirements have been incorporated into contract requirements for AFPs and large consultant contracts. Audits are being performed against the AFP project company and the large consultant contract providers.
Metrolinx implemented a new Vendor Performance Management (VPM) system, which takes into account past performance in order to be able to influence future contract awards. Key Performance Indicators included in the VPM evaluation scorecard will be reviewed and will be factored into the evaluation. Should a sub-trade of a contractor fail to perform, it will be reflected in the contractor's VPM score, which will then be used to evaluate and award the contractor future Metrolinx projects. This ensures that the contractor is motivated to effectively manage the performance of its sub-trades.
Metrolinx's VPM program will be entered into the computerized system and scorecard evaluations will be completed on an ongoing basis. Starting April 2018, tendered contracts will use vendor performance management data as part of the evaluation criteria for awarding new contracts to vendors.
Committee Recommendations
The Standing Committee on Public Accounts recommends that:
7. Metrolinx provide the Committee with the results of its procurement team's analysis of what type of mechanism or clause will be implemented in contracts to ensure that a substantial scope of work is not subcontracted.
8. Metrolinx provide the Committee with its evaluation of the benefits of implementing a pre-screening process for large subcontractors.
Accounting System
The audit found that Metrolinx's enterprise management system does not have a control in place to ensure that payments exceeding approved budgets have been approved for over-expenditure. As a result, project staff must manually keep track of project expenditures to ensure that they are within the budget.
In its response, Metrolinx stated that in advance of the implementation of its Oracle Procurement module, all purchase orders deemed to be completed were closed out and not converted into the new module. A total of 10,367 purchase orders were closed through the data cleansing process.
As part of a suite of procedures and processes along with associated systems (CM14), Metrolinx has refined and introduced the Payment procedure, Work Release procedure, Contract Change and Amendment procedure, and the Contract Completion and Close-Out procedure.
Using these processes, staff will have the guidance and the information needed within CM14 to ensure that payments will only be issued up to the approved budget and purchase order limits, and purchase orders will follow the proper close-out procedure upon project completion. Improvements to the interface between CM14 and Oracle Procurement will allow an automated process to close out purchase orders on completed projects. This work is targeted for completion in the fourth quarter of 2017-18.
Canadian National and Canadian Pacific Railways
The audit found that Metrolinx pays Canadian National Railway (CN) and Canadian Pacific Railway (CP) without verifying most costs. Metrolinx's projects with CN and CP are costed in one of two ways. With some CN projects, CN provides an estimate of the total costs, and that estimate becomes the lump-sum amount Metrolinx ultimately must pay for the project. With other CN projects and almost all CP projects, CN or CP invoices Metrolinx based on the project's time and materials. In all cases, Metrolinx pays CN and CP without verifying most costs. Compared to other rail companies that work for Metrolinx, CN charged Metrolinx significantly higher materials and labour costs. Specifically, materials costs were about 60% higher and labour costs were 130% higher. Information on CP's costs were not detailed enough to allow the Auditor to perform the same comparison.
Additionally, the audit found that Metrolinx pays CN and CP excessive markup rates on projects. All contracts with CN and CP are sole-sourced. CN's mark-up rates on labour and parts are set in a long-term agreement with Metrolinx. These rates are as much as 74% higher than industry benchmarks. Metrolinx has not negotiated any mark-up rates with CP, and they are usually not transparent. The Auditor found that CP disclosed their mark-up rates in only one of the projects sampled, and they were about 30% higher than industry benchmarks.
In its response, Metrolinx stated that it has refined and introduced a suite of procedures and processes along with associated systems (CM14) to enable staff to manage all facets of a contract, whether with a vendor or a railway. The processes will help staff ensure invoice line items are validated, assess the reasonableness of costs against the budget through comparison with Metrolinx estimates and forecasts, and perform second-party compliance audits for safety and quality management.
Metrolinx has historically-required audit rights on CN and CP contracts, but these rights were not always consistently defined. Metrolinx has included audit rights in all new agreements with CP and CN. These provisions include the ability for Metrolinx to perform audits on CN and CP invoices. Metrolinx (both the Rail Corridor Infrastructure group and Capital Projects group) has recently exercised its right to audit, and initiated third-party reviews of work done by CN and CP, respectively.
Metrolinx engaged PricewaterhouseCoopers (PwC) to conduct contract compliance assessments as part of its governance procedures to effectively manage costs and budgets with its contractors. PwC performed an independent contract assessment, which included the review of the CN Master Operating Agreement and Master Construction Agreement, and the CP Commuter Operating Agreement and key supporting documentation, as well as interviews with key Metrolinx personnel involved in managing these Agreements. This work was completed in spring 2017, and PwC's final report was delivered on July 5, 2017. PwC's findings were reviewed by staff and recommendations are being considered as part of ongoing negotiations.
Metrolinx staff has internally reviewed the terms of the Master Construction Agreement. Based upon both the PwC report and Metrolinx's internal review, Metrolinx staff is in the process of developing a list of potential updates to the Master Construction Agreement, including proposed amendments to the commercial terms. Given the level of complexity related to the Master Construction Agreement, as well as other ongoing negotiations with CN, an estimated timeline for completion of a new Master Construction Agreement cannot be provided.
A list of work required on CN and CP corridors to support Regional Express Rail (RER) and other projects (including various light rail transit [LRT] projects) is being developed. In addition, a single point of contact has been identified within the legal department, to oversee the negotiation of all agreements with CN and CP. These steps will enable coordinated negotiations and ensure the inclusion of appropriate contract terms in all agreements with CN and CP.
Committee Recommendations
The Standing Committee on Public Accounts recommends that:
9. Metrolinx provide an update to the Committee on its efforts in negotiating with CP to incorporate the allowance of audits of CP invoices.
10. Metrolinx provide the Committee with an update on its review and renegotiation of the Master Construction Agreement with CN.
11. Metrolinx provide the Committee with an update on the results of third-party reviews conducted on CN and CP projects.
12. Metrolinx provide to the Committee, after negotiations with CN and CP have been completed, its yearly operating cost resulting from the 20% of the lines that it operates on CN and CP lines.
Stage Gate Process
The Committee heard that Metrolinx has implemented a stage gate (or decision points) process. This process-which is a best practice used in other jurisdictions-allows for project decisions to be taken in an orchestrated way at each of the stage gates. All of the different players involved in a contract know the decisions made at each particular gate. These gates are at the point of feasibility, followed by optioneering (a way of assessing alternative design options, analyzing their long-term capital and operational costs to identify those with the lowest price tag over the entire lifecycle), preliminary design, design, and procurement.
Consolidated list of Committee Recommendations
The Standing Committee on Public Accounts recommends that:
- Metrolinx provide the Committee with its most recent results, including results from 2016-17 and 2017-18, of additional costs incurred by Metrolinx attributed to errors and omissions by design consultants.
- Metrolinx provide the Committee with results of its evaluation and lessons learned exercise from the pilot tender using Vendor Performance Management (VPM), which was expected to be completed by March 2018.
- Metrolinx provide the Committee with a summary of the number of safety breaches found in its audits of contractors at construction sites over the past year.
- Metrolinx provide an update on its Contract Management training initiatives, including its development of a standard approach for the inclusion of liquidated damages (LDs) in its construction contracts.
- Metrolinx provide the Committee with
a) a list of projects completed over the past year and the corresponding length of time needed to fix outstanding deficiencies in each project; and
b) where projects did not meet the industry standard of two months to fix all deficiencies, provide a reason why this was not done.
- Metrolinx provide the Committee with an update on its Contract Completion and Close-Out procedure and its Substantial Completion procedure, and explain how these procedures have assisted Metrolinx in closing out projects within the industry standard of two months.
- Metrolinx provide the Committee with the results of its procurement team's analysis of what type of mechanism or clause will be implemented in contracts to ensure that a substantial scope of work is not subcontracted.
- Metrolinx provide the Committee with its evaluation of the benefits of implementing a pre-screening process for large subcontractors.
- Metrolinx provide an update to the Committee on its efforts in negotiating with CP to incorporate the allowance of audits of CP invoices.
- Metrolinx provide the Committee with an update on its review and renegotiation of the Master Construction Agreement with CN.
- Metrolinx provide the Committee with an update on the results of third-party reviews conducted on CN and CP projects.
- Metrolinx provide to the Committee, after negotiations with CN and CP have been completed, its yearly operating cost resulting from the 20% of the lines that it operates on CN and CP lines.