2 Report of the AuditorAuditor General Report de Montr?al of the General of the Ville of City Ville and Montr?al to thethe Council de to the Urban Agglomeration Council to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 For the Year Ended December 31, 2012 Situation du Bureau du v?rificateur g?n?ral Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 Legal Deposit - Second Quarter 2013 Biblioth?que et Archives nationales du Qu?bec Library and Archives Canada ISSN: 1925-6787 (print) ISSN: 1925-6795 (online) ISBN: 978-2-7647-1189-7 (print) ISBN: 978-2-7647-1193-4 (online) Please note that this English report has been translated from the original French version. In case of doubt or difference of interpretation, the French version shall prevail over the English. According to the Charter of the French Language and the Office qu?b?cois de la langue fran?aise, municipalities shall designate all official names, such as boroughs, departments, paramunicipal corporations as well as municipal and associated bodies by their French names alone, even in the English version. This report is available on our website at: ville.montreal.qc.ca/verificateur. May 23, 2013 Mr. Michael Applebaum Mayor of the Ville de Montr?al 275, rue Notre-Dame Est Suite 1.113 Montr?al, QC H2Y 1C6 Subject: Auditor general of the Ville de Montr?al's annual report for the year ended December 31, 2012 Dear Mr. Mayor, Please find enclosed the Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council for the year ended December 31, 2012 as per section 107.13 of the Cities and Towns Act (RSQ, chapter C-19) to be tabled at the next regular city council meeting on May 27, 2013 and the next urban agglomeration council meeting on May 30, 2013. Yours truly, Jacques Bergeron, CPA, CA, MBA, M. Sc. Auditor general 1550, rue Metcalfe, bureau 1201, Montr?al (Qu?bec) H3A 3P1 Table of Contents 1. Comments and Recommendations from the Auditor General .............. 7 2. Overview of the Bureau du v?rificateur g?n?ral ....................................... 27 2.1. Workforce Status ................................................................................................ 29 2.2. Performance Indicators....................................................................................... 33 3. Investigative and Forensic Accounting Report......................................... 43 4. Financial Statement Audits ............................................................................... 51 4.1. Consolidated Financial Statements of Ville de Montr?al..................................... 53 4.2. Financial Statements of Other Municipal Bodies ................................................ 57 5. Value-for-Money and Information Technology Audit .............................. 61 5.1. Follow-Ups to Recommendations from Previous Years ..................................... 63 5.2. Asset Disposal .................................................................................................... 69 5.3. Infrastructure Work Timeframes ......................................................................... 97 5.4. Issue of Building and Subdivision Permits - Contaminated Lands................... 127 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau)......................................................................... 163 5.6. Implementation Plan for Arterial Road System Infrastructure Work (Service des infrastructures, du transport et de l'environnement - Direction des infrastructures) ............................................................................ 229 5.7. Contract Awarding and Management Process (Soci?t? du parc JeanDrapeau) ........................................................................................................... 265 5.8. Residential Swimming Pool Safety Regulations ............................................... 323 5.9. First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al).......................................................................... 365 5.10. Subcontracted Construction Work .................................................................... 415 5.11. "Integrated Human Resources and Payroll Management System" Project............................................................................................................... 449 5.12. "Time Management" Subproject ....................................................................... 467 5.13. Protection of Personal Information.................................................................... 497 5.14. Penetration Tests.............................................................................................. 515 Auditor General of the Ville de Montr?al 2012 Annual Report 6. Appendices ........................................................................................................... 521 6.1. Appendix 1 - Excerpts from the Cities and Towns Act ..................................... 523 6.2. Appendix 2 - Employees of the Bureau du v?rificateur g?n?ral as of December 31, 2012 .......................................................................................... 535 6.3. Appendix 3 - Information Flow Charts - Value-for-Money and Information Technology Audit ........................................................................... 539 6.4. Appendix 4 - Accounts Statement of the Bureau du v?rificateur g?n?ral .............................................................................................................. 545 Auditor General of the Ville de Montr?al 2012 Annual Report Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 Comments and Recommendations from the Auditor General 1 Chapter 1 Comments and Recommendations from the Auditor General 1. Comments and Recommendations from the Auditor General This year, as in past years, I am including a few comments and recommendations for the municipal administration in my introduction. This year, I felt it was appropriate to address the following topics of interest: o Management of the ethics hotline and EPIM Squad (Escouade de protection de l'int?grit? municipale); o Commission of Inquiry on the Awarding and Management of Public Contracts in the Construction Industry (Charbonneau Commission); o Avenues to explore for improving the city's management processes: - Risk management, - Expected levels of service, - Infrastructure project planning, - Three-year capital expenditures program (TCEP), - Accountability mechanisms, - Follow-up of the auditor general's recommendations; o Follow-up on the distribution of contracts awarded by authorities to contractors; o Auditor general's budget; o Securing the Bureau du v?rificateur g?n?ral's email service and computer network. Management of the Ethics Hotline and the EPIM Squad On October 17, 2012, I asked city council members in a letter to reconsider their decision of December 2010 to transfer the responsibility of managing the ethics hotline from the Bureau du v?rificateur g?n?ral (BVG) to the Service du contr?leur g?n?ral. This request was justified by the current climate, in which Ville de Montr?al (the city) has been beset by allegations of collusion, corruption and misappropriation of funds since the opening of hearings of the Commission of Inquiry on the Awarding and Management of Public Contracts in the Construction Industry. I still believe that this decision would help raise the level of trust that city employees and the general public place in the municipal administration. My thinking was based on the following main arguments: o The particular status of the auditor general: as a city council officer, the auditor general is independent, neutral, impartial and free of ties to the administration. People who want to make a report would feel more confident about giving it to the BVG; Auditor General of the Ville de Montr?al 9 2012 Annual Report Chapter 1 Comments and Recommendations from the Auditor General o o o o The Service du contr?leur g?n?ral is subordinate to the authority of the city manager. This relationship could present a conflict of interest if allegations were levelled against the city management; Unlike the BVG, the Service du contr?leur g?n?ral does not have the power to audit or investigate bodies under the city's jurisdiction, such as the Soci?t? de transport de Montr?al (STM) or the Soci?t? d'habitation et de d?veloppement de Montr?al (SHDM), which are exempt from works conducted by the Service du contr?leur g?n?ral; The provisions of the Cities and Towns Act explicitly guarantee the confidentiality of any reporting to the BVG. Fraud or favouritism very often involves the participation of people on the inside and on the outside. The ethics hotline is designed specifically to enable employees, suppliers and citizens to report any wrongdoing in complete confidentiality; In other Canadian cities, an increasing number of reports are made to ethics hotlines every year, which is the opposite of what has happened in Montr?al since the line was transferred to the Service du contr?leur g?n?ral. For instance, in Toronto, the number of reports surged from 157 in 2002 (the first year in which the ethics hotline was used) to 822 in 2011, while in Ottawa the number of reports rose from 48 in 2005 to 182 in 2011. For Montr?al, in 2010 (the first year in which the auditor general was responsible for the ethics hotline, for a 13-month period), the BVG received 131 reports, whereas only 33 reports were received in 2011 (under the direction of the Service du contr?leur g?n?ral). We also want to stress that in all other Canadian cities with an ethics hotline, it is under the authority of the auditor general. Recommendation I remain convinced that it is in the city's interest to entrust the BVG with the responsibility of managing the ethics hotline. Accordingly, I reiterate the request I made to city council last October 17, which was that responsibility for the ethics hotline be transferred to the BVG. I was also informed of the creation of the EPIM Squad on January 11, 2013, which is mandated to protect the municipal integrity of the Ville de Montr?al, particularly with regard to the contract awarding process. The Service de police de la Ville de Montr?al (SPVM) was appointed to create and manage this new squad. A dedicated telephone line and an email account were set up for employees to pass on information to the EPIM in complete confidentiality. Since then, the BVG has remained in contact with the EPIM to coordinate our respective duties with a view to efficiency and effectiveness. Auditor General of the Ville de Montr?al 10 2012 Annual Report Chapter 1 Comments and Recommendations from the Auditor General Commission of Inquiry on the Awarding and Management of Public Contracts in the Construction Industry (Charbonneau Commission) The allegations made before the Commission of Inquiry on the Awarding and Management of Public Contracts in the Construction Industry (Charbonneau Commission) concerning activities involving collusion and corruption in the municipal administration of Montr?al are disquieting. According to allegations made at the Charbonneau Commission hearings, an organized system has been in place for several years to divert funds from the city's coffers to organized crime, political parties, suppliers, elected officials and employees. This system would include complex overbilling schemes, falsification of documents and tacit complicity. It appears that several people at different levels in the city's chain of command could be directly involved in the alleged schemes. For many years the BVG conducted audits for the purpose of detecting and preventing activities of collusion or corruption. I would like to present an overview of the audits of the contracting process that the BVG conducted over the past 15 years to show that the auditor general, within the scope of his responsibilities, repeatedly sounded the alarm. This is not an exercise in self-justification, but an account of the audits that the BVG conducted with a critical approach to the risks of collusion or corruption and with a desire to understand the particular context in which the alleged collusion and corruption became entrenched in the city. Auditors have a twofold responsibility with respect to fraud. First, they fulfil a preventive function by formulating appropriate recommendations following audits in an attempt to limit opportunities for fraud. Auditors also fulfil a function of fraud detection, insofar as possible, provided that evidence has come to light as a result of audits or whistleblowing. It is important to understand that collusion and corruption are among the most difficult types of fraud to detect. They result from secret agreements involving schemes that are often implemented at a point in the process that occurs ahead of the organization's internal control mechanisms. The allegations made before the Charbonneau Commission show the presence of all the elements of collusion and corruption in combination, which are extremely difficult to detect during a traditional audit. In fact, there are probably three types of schemes in use: o Internal collusion involving several people who are part of the chain of command; o Corruption of public officials and elected officials by organizations with interests in the city; Auditor General of the Ville de Montr?al 11 2012 Annual Report Chapter 1 Comments and Recommendations from the Auditor General o Outside collusion of various firms for the purpose of creating market-sharing cartels for contracts and price-fixing. Through its audit reports, the BVG has repeatedly had an impact over the past 15 years on controls used to prevent fraud by urging that the following measures be implemented: o Division of duties among key stakeholders; o Documentation of variances between cost estimates and the lowest bids; o The level of care the city must exercise in producing the most accurate estimates possible and in keeping them confidential; o Spot checks of supervision of projects entrusted to outside firms to ensure that the city pays only for work actually performed; o Follow-up on project implementation and control of contingencies (commonly called "extras"); o Cost comparisons with comparable work in previous years or similar work performed by other boroughs; o Production of relevant information so that authorities can make informed decisions. As early as 1997, the auditor general noted that a limited number of suppliers shared the Montr?al market (two annual reports--1997 and 2001--discuss this issue). In 2009, I also observed the troubling market share of construction firms within some boroughs. At that time, the BVG also perused a report released by the city's internal auditor in November 2006, which has since then been submitted to the Charbonneau Commission. This report concluded that the city was not operating in a completely competitive environment and proposed appropriate recommendations to mitigate the risk of fraud. It should be noted, however, that the conclusions of this report did not uncover any acts of collusion or corruption. In fact, according to this report: [TRANSLATION] "It should be noted that our observations were based on the results of tests conducted on a certain number of projects reviewed and would therefore not necessarily have led to the detection of fraud or any other illegal act if they ever existed." Moreover, the BVG agreed with all the recommendations of the 2006 internal audit report. In my opinion, the measures recommended were appropriate for preventing fraud in contracting. The city accepted these recommendations, and action plans were produced and assigned to individuals in positions of authority in the city. The BVG reviewed those action plans, which were ratified by city officials, to implement the proposed actions. In 2007, my predecessor noted that it would be unrealistic to believe that an auditor general could detect all fraudulent activity and recommended that the city set up an ethics hotline to Auditor General of the Ville de Montr?al 12 2012 Annual Report Chapter 1 Comments and Recommendations from the Auditor General receive reports in complete confidentiality on any wrongful act that could be connected with fraud or waste. The hotline, as is well-known, was introduced at the end of 2009 by the BVG but was withdraw in December 2010. As I mentioned above, it would seem that the perpetrators of the corruption and collusion brought to light by the Charbonneau Commission probably used schemes that were, according to the information disseminated, highly complex, carefully planned and skilfully orchestrated, making them almost undetectable in a traditional audit, even for the most experienced auditor. It should also be kept in mind that the scope of the BVG's audit and its findings is limited by four major factors: o The auditor general does not have the police authorities' power to investigate; o Unlike the Charbonneau Commission, which has the power to force members of various organizations to testify before it, the city's auditor general is empowered only to require that any employee of the city provide information necessary for the audit, and does not have the legal authority to demand information from elected officials or third parties or to force interviewees to explain themselves; o By their very nature, collusion and corruption impose limitations on an organization's internal control. As long as there are people who decide to conspire to commit corruption or collusion, it becomes difficult, even very difficult, for an auditor to detect schemes implemented for this purpose; o Funding provided to the auditor general is limited and set by law. The auditor general has a far-reaching mandate that involves not just the regulatory compliance audit, but extends to certification of the city's financial statements, as well as those of bodies under its jurisdiction, and the value-for-money audit in different areas, such as services provided to citizens, infrastructure longevity, information technologies and management of human, material and financial resources. In conclusion, I am convinced that an independent audit of the city's accounts and affairs, such as it is conducted by its auditor general, is still an important means of ensuring sound management of public funds. However, because of the abovementioned constraints, it would be unrealistic to expect the auditor general to be able to detect all fraudulent activity that could occur in a large city like Montr?al. I have neither the powers nor the resources of a commission of inquiry or a police force. Furthermore, fraud is usually detected on the strength of disclosures by individuals who witnessed it. To this end, my investigative and forensic accounting team always follows up on tips it receives and, if need be, performs the necessary investigative work. When necessary, the case is referred to the police (see Auditor General of the Ville de Montr?al 13 2012 Annual Report Chapter 1 Comments and Recommendations from the Auditor General chapter 3 of this report for more details). Also, in light of the revelations of the Charbonneau Commission, my office changed its auditing procedures to focus more on evidence that would help detect these offences. Auditor General of the Ville de Montr?al 14 2012 Annual Report Chapter 1 Comments and Recommendations from the Auditor General Avenues to Explore for Improving the City's Management Processes In this section, I want to call attention to some of the BVG's findings that are intended to tighten and improve city management. I believe these findings will provide grounds for the city to adopt a position and for the municipal administration to take concrete actions. Risk Management Several years ago the city developed a risk management grid that brought to light various risks that could jeopardize its activities. The municipal administration should promote the design of a new risk management grid that would take into account new parameters and certain key sectors (e.g., permits, professional fees, contracts) that are more at risk for embezzlement. Recommendation I recommend that city council ensure that the city produce a new risk management grid to determine key sectors and activities that are more exposed to misappropriation of funds in order to increase surveillance and strengthen its internal controls, thereby preventing financial losses. Expected Levels of Service Audits conducted by the BVG led me to the conclusion that city authorities have made no decisions about desired levels of service (desired condition) for buildings or infrastructures, whether they involve the road system, bridges and tunnels or water systems. These expected levels of service are of vital importance because they define the parameters of the corresponding levels of investment. They are based on knowledge of the present condition of infrastructures and buildings in different categories, and they clearly depend on choices that authorities must make in the areas of reliability and safety. Since the city has made no clear commitments regarding levels of service, and given the budget constraints and limited resources available, the investment deficit can only rise, and infrastructures and buildings will continue to deteriorate. Auditor General of the Ville de Montr?al 15 2012 Annual Report Chapter 1 Comments and Recommendations from the Auditor General Recommendation Given the present condition of the various asset categories for the city's buildings, road system, bridges, tunnels and waterworks systems, I recommend that city council take the necessary steps to ensure that: o Levels of service are established for these infrastructures and buildings; o These levels of service are submitted to and approved by authorities; o Levels of investment that correspond to the approved levels of service are allocated over a long-term period; o The city adopts a realistic funding method so that responses deemed necessary can be implemented. Infrastructure Project Planning Infrastructure projects are complex undertakings that must be planned over a timeline of several years so that all urgent or high-priority tasks can be covered adequately. This planning also gives an overview of what needs to be accomplished over the next few years. However, audits conducted by the BVG led me to the finding that the planning is currently done on an annual basis, since TCEP budgets are uncertain from one year to the next, which makes it difficult to execute some projects. Recommendation I recommend that city council take the necessary steps to ensure that infrastructure project planning is carried out over a period of several years in order to facilitate project implementation and help prioritize longer-term responses. Three-Year Capital Expenditures Program (TCEP) The TCEP helps determine the budget available for the various high-priority capital projects. The city's various departments and boroughs need to know the content of the TCEP so that they can plan and coordinate projects for which they are responsible. I have identified important issues in this area that deserve the administration's attention. First, I have noted that the TCEP is not approved early enough for business units (departments and boroughs) to be informed in a timely manner of budgets that will be allocated to them so that they can plan their projects accordingly. Business units are caught off guard and cannot begin necessary projects in time and take advantage of the lowest costs that might be offered by different contractors. In past years, authorities approved the TCEP in December. The TCEP for 2013 was approved in September 2012, which is an Auditor General of the Ville de Montr?al 16 2012 Annual Report Chapter 1 Comments and Recommendations from the Auditor General improvement. Nevertheless, this step in the process is still carried out too late for business units to be able to implement their projects at the best possible cost. Recommendation I recommend that city council take the necessary steps to ensure that adoption of the TCEP is scheduled earlier so that business units can plan their projects in a timely manner. Second, the municipal administration does not verify business units' capacity to perform the work demanded of them. Business units have limited financial and human resources. The fact that the administration does not take these limits into account in establishing the TCEP is essentially tantamount to an expression of wishful thinking. In fact, audits conducted by the BVG showed that: o The budget allocated to business units is not fully spent; o A large number of planned projects are not executed in the year in which they are scheduled. Many projects originally scheduled for one year are carried over to subsequent years, which forces the implementation rate up drastically and greatly exacerbates the deterioration of infrastructures. Recommendation I recommend that city council take the necessary steps to ensure that business units have the resources required to implement the projects provided for in the TCEP. Accountability Mechanisms Audits conducted by the BVG led me to the finding that the municipal administration does not always receive all the information it needs from business units to make informed decisions and discharge its governance responsibilities. For example, given the levels of investment allocated, the municipal administration does not receive periodic reporting on the condition of infrastructure and building assets. The municipal administration should make its accountability reporting needs clear to all business units. This would help establish high-priority or strategic issues on which it wishes to receive regular reporting. Auditor General of the Ville de Montr?al 17 2012 Annual Report Chapter 1 Comments and Recommendations from the Auditor General Recommendation I recommend that city council take the necessary steps to ensure that business units are clearly informed of the high-priority or strategic sectors on which they must regularly report so that the municipal administration will obtain the complete information it needs to make informed decisions and fulfill its governance responsibilities. Follow-Up of the Auditor General's Recommendations In 2012 I noted a certain laxness on the part of the municipal administration in implementing the recommendations outlined in the annual report for the year ended December 31, 2011. In fact, as the current report was going to press, a "completed" status applied to only 35% of the recommendations made in 2011, compared with 69% for recommendations made in 2010 at the same time last year. Recommendation I recommend that city council take the necessary steps to ensure that business units follow up more rigorously on the auditor general's recommendations. Auditor General of the Ville de Montr?al 18 2012 Annual Report Chapter 1 Comments and Recommendations from the Auditor General Follow-Up on the Distribution of Contracts Awarded by Authorities to Contractors Audits conducted in 2009 in connection with the production of my annual report raised questions for me about the distribution of contracts awarded to contractors for projects to develop, rehabilitate and replace facilities and infrastructures. At that time I established that a sizeable proportion of the contracts awarded from 2006 to 2009 by borough councils, the executive committee, city council and the urban agglomeration council went to a group of 21 firms. More specifically, I reviewed the distribution of contracts that each borough council awarded to this group of 21 firms. For this period, the results of this review led to the finding that some boroughs were awarding a high percentage of their contracts to the same contractors within this group of 21 targeted firms. Since 2009, the municipal administration has implemented an array of measures to tighten contracting rules, while, concurrently, new legislative provisions were introduced for the same purpose. As mentioned in my 2009 annual report, I had committed to following up in fiscal 2011. This follow-up was performed and presented in the 2011 annual report. The work consisted of a review to verify whether the trend noted in 2009 was still evident in 2011. I excluded 2010 data from my review because 2010 was the year in which the municipal administration introduced measures and some of the legislative provisions were enacted. The results of this follow-up, which focused on the year 2011, showed a trend towards a more equitable distribution of contracts among the 21 firms, which led me to believe that the measures implemented by the municipal administration and the provincial legislature were producing results. However, I had planned to continue my review for 2012, 2013 and 2014 so that I could assess its merits with greater assurance. Since January 1, 2012, new administrative and legislative measures have been in force. Some of them were adopted as a result of the revelations heard at the Charbonneau Commission, which opened on May 22, 2012. Here are a few of them: o 1 The requirement that any contractor wishing to enter into a construction contract of $25,000 or more with a municipal body hold a certificate of compliance from Revenu Qu?bec under the Regulation respecting construction contracts of municipal bodies, 1 which came into force on January 1, 2012. The certificate must confirm that the Gazette officielle du Qu?bec. Part 2, Vol. 143, No. 35, August 31, 2011, pp. 3899-3901. Auditor General of the Ville de Montr?al 19 2012 Annual Report Chapter 1 Comments and Recommendations from the Auditor General company filed the returns required under tax laws and has no overdue account payable to the Minister of Revenu Qu?bec. o The creation of the register of enterprises ineligible for public contracts (RENA), which has been in force since June 1, 2012. The RENA is one of the measures the provincial legislature has adopted to ensure the probity of companies wishing to conduct business with the government. The names of companies that have been convicted of an offence under a law or regulation listed in the Act respecting contracting by public bodies, 2 including the Criminal Code and the Income Tax Act, 3 are recorded in the RENA. Accordingly, companies registered in the RENA cannot be awarded a public contract or subcontract or carry out such a contract for a maximum period of five years. o The suspension of contracting for infrastructure work, which was ordered by the city's executive committee since October 3, 2012. This decision was motivated mainly by the new provincial government's desire to rapidly amend the scope of the Act to prevent, combat and punish certain fraudulent practices in the construction industry and make other amendments to the Building Act. 4 In this regard, my review of all the resolutions made by the executive committee, city council and the urban agglomeration council from that date until March 31, 2013, led me to conclude that no contract for infrastructure work was granted during this period. o The coming into force of the Integrity in Public Contracts Act, 5 assented to on December 7, 2012, which requires that companies wishing to contract with a public body or municipality obtain prior authorization from the Autorit? des march?s financiers (AMF) before entering into public contracts in order to safeguard transparency in contracting processes, the honest and fair treatment of tenderers, and the opportunity for qualified tenderers to participate in calls for tenders made by public bodies. The contracts to which this Act applies are construction, service and public-private partnership contracts and subcontracts that are executed as part of an infrastructure project equal to or greater than $40 million, as well as certain contracts of Ville de Montr?al, as determined by decree. To date, three decrees, each involving 25 bids, have come into force, one on January 15, one on February 13, and one on March 20, 2013. These are calls for tenders that the city wishes to pursue or issue and involve expenditures of less than $40 million. 2 3 4 5 RSQ, chapter C-65.1. RSC, 1985, chapter 1 (5th supp.). SQ, 2011, chapter 35. SQ, 2012, chapter 25. Auditor General of the Ville de Montr?al 20 2012 Annual Report Chapter 1 Comments and Recommendations from the Auditor General o An amendment to the contract management policy passed on December 17, 2012, which excludes companies that have admitted to taking part in collusion, corrupt practices or other offences of the same nature from the bidding process for a five-year period. Previously, such penalties could be imposed only when their liability was established under a final court decision. This amendment affects calls for tenders issued after that date. o The announcement by the municipal administration on January 11, 2013, of the creation of the EPIM, which reports to the SPVM. This squad will be mandated to protect the administrative integrity of Ville de Montr?al by handling all information, mostly crimerelated, that it receives, particularly with regard to the tendering process. It will also introduce prevention and detection mechanisms for schemes and ploys that sully the contract tendering process. This mandate must extend to all municipal departments. o The creation, on January 24, 2013, of the advisory committee on the granting of municipal contracts by the Ville de Montr?al, chaired by Mr. Jacques L?onard. On May 8, 2013, the Rapport du comit?-conseil sur l'octroi et la gestion des contrats municipaux ? la Ville de Montr?al has been tabled to the mayor. This report recommend notably measures for reviewing the rules for awarding contracts and their management, and ensure the effectiveness of internal and external controls as well as compliance with ethical standards. Under the circumstances, in view of the recent implementation of these new administrative and legislative measures, I think it would not be very useful to repeat the task of following up on contracts obtained by the 21 firms originally targeted. Auditor General of the Ville de Montr?al 21 2012 Annual Report Chapter 1 Comments and Recommendations from the Auditor General Auditor General's Budget In my last three reports, I discussed the problem of calculating credits granted to the auditor general under the Cities and Towns Act. I noted that calculation of my credits on the basis of the city's operating budget did not allow me to fully carry out my legal responsibilities, which include not only audits of the city's accounts and affairs, but also audits of bodies that are within the city's jurisdiction or are part of its reporting entity. The situation was even worse in 2012 because I had to assume the certification costs of several bodies that had been included in the city's reporting entity since January 1, 2011. The issue changed appreciably in 2012, when the Commission permanente sur les finances et l'administration (the Commission) heard some of my grievances. The Commission's report, entitled Rapport d'?tude publique et recommandations sur le Rapport annuel du v?rificateur g?n?ral de la Ville de Montr?al - Exercice 2011, was submitted to city council on August 20, 2012. In the report, the Commission presented several recommendations, three of which concerned the auditor general's operating budget. These recommendations are given on page 12 of the report. They essentially concern two points: o The audit fees charged to bodies that were already paying fees to have their financial statements certified by outside firms in 2011 and for which the BVG had to undertake certification activities in 2012. These bodies were: - Bixi Toronto Inc.; - Centres locaux de d?veloppement (CLDs); - Fiducie du Technoparc Montr?al; - Office municipal d'habitation de Montr?al (OMHM); - Soci?t? de jalonnement dynamique de Montr?al; - Soci?t? de v?lo en libre-service; - Soci?t? en commandite Stationnement de Montr?al; - Technoparc Montr?al. For these bodies, the Commission formulated Recommendation R-3, which reads as follows: [TRANSLATION] [The Commission recommends] that the bodies listed continue to pay audit fees out of their operating budgets. I estimated that these fees totalled $350,000, excluding CLDs. Auditor General of the Ville de Montr?al 22 2012 Annual Report Chapter 1 Comments and Recommendations from the Auditor General o The certification costs paid by the BVG for the financial statements of municipal bodies that were already under my jurisdiction before the amendment to the Cities and Towns Act came into force on January 1, 2011. These bodies are: - Anjou 80; - Conseil des arts de Montr?al; - Conseil interculturel de Montr?al; - Corporation d'habitation Jeanne-Mance; - Office de consultation publique de Montr?al; - Soci?t? de gestion Marie-Victorin; - Soci?t? de gestion du port de plaisance de Lachine 6 ; - Soci?t? d'habitation et de d?veloppement de Montr?al (SHDM); - Soci?t? du parc Jean-Drapeau. For these bodies, the Commission formulated Recommendation R-4, which reads as follows: [TRANSLATION] [The Commission recommends] that paramunicipal bodies, the Soci?t? d'habitation [et de d?veloppement] de Montr?al, the Soci?t? du parc Jean-Drapeau and the Commission des services ?lectriques de Montr?al, which form part of the city's reporting entity, pay all audit costs and fees out of their operating budget. Following analysis, I excluded the Commission des services ?lectriques de Montr?al from the list of bodies covered by this recommendation. I therefore estimated the total cost of the financial certification of these bodies in 2012 to be $260,100. Finally, the Commission formulated Recommendation R-5, which reads as follows: [TRANSLATION] [The Commission recommends] that in the event that recommendations R-3 and R-4 are not applicable, the Service des finances evaluate and consider the possibility of taking the funds to pay the auditor general's fees out of the budgets of the bodies concerned and placing them in the budget of the Bureau du v?rificateur g?n?ral. In September 2012 I approached the senior manager and treasurer of the city to enquire whether follow-up action was being taken on these recommendations. I indeed received credits of $610,100, which were added to my budget for fiscal 2013, which was approved by city council as part of the procedure for adoption of the city's budget. These credits are 6 New name since August 15, 2012. Formerly the Soci?t? de gestion NauBerges de Lachine. Auditor General of the Ville de Montr?al 23 2012 Annual Report Chapter 1 Comments and Recommendations from the Auditor General very useful for helping me fulfil my mandate properly. The senior manager and treasurer confirmed that these credits would be paid on a recurring basis. I want to stress that the additional amount allocated to the BVG covers only costs and expenses related to the certification of financial statements of bodies under the city's jurisdiction, but does not cover the other two components of the auditor general' mandate for those bodies: the value-for-money and the regulatory compliance audits. This is why I recommended in 2009 that the consolidated budget rather than the city's operating budget be used as the basis for calculating the auditor general's credits. I am nevertheless thrilled with the progress that has been made, as well as with the Commission's and city council's responsiveness. Auditor General of the Ville de Montr?al 24 2012 Annual Report Chapter 1 Comments and Recommendations from the Auditor General Securing the Bureau du v?rificateur g?n?ral's Email Service and Computer Network In 2012, the BVG carried out two major projects to make its communications and data more secure. The first project was to transfer all BVG email boxes from the Lotus Notes email system to the Google Apps email system, creating its own domain name (bvgmtl.ca) in the process. The BVG enlisted the services of a specialized firm to assist with this transfer process. Google's security measures were analyzed in depth and considered sufficient to meet the BVG's needs. The second project was to implement a new protected domain outside the city's domain so that it could assume full control of the BVG's network and server infrastructure. A connection with the city's internal network is still being maintained, but it is now controlled by the BVG. The BVG's network is now protected by a firewall under its control. Auditor General of the Ville de Montr?al 25 2012 Annual Report Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 Overview of the Bureau du v?rificateur g?n?ral 2 Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 Workforce Status 2.1 2.1. Workforce Status 2. Overview of the Bureau du v?rificateur g?n?ral 2.1. Workforce Status Over the years, we have witnessed a continuous and very worrisome erosion of our staff because of retirement and recruiting difficulties, especially with salaries that are not very competitive in our comparison market. Table 1 - Workforce Trends as of December 31 for 2002-2012 Total employeesa 41 36 38 37 35 33 28 26 30 33 30 Year 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 a The workforce excludes two audit professionals who were released full-time for union activities for the years 2008 to 2010 and one professional for 2011. Attracting and retaining qualified and experienced resources remains a major challenge for the Bureau, despite measures taken to address this problem. As this report goes to press, the Bureau now has 29 staff members following the retirement of a key employee. Five positions are currently vacant, including four senior auditor positions--two in the value-for-money audit, one in the information technology audit and one in financial statement certification--as well as an auditor's position in financial statement certification. Auditor General of the Ville de Montr?al 31 2012 Annual Report Chapter 2 Overview of the Bureau du v?rificateur g?n?ral These vacant positions, which represent 15% of our total resources, inevitably affect the Bureau's ability to fulfill its responsibilities. Furthermore, the shortage of resources has forced the Bureau to rely to a greater extent on the services of external firms in honouring its obligations to certify financial statements. It is clear that a greater effort must be made to solve the difficult issue of attracting and retaining human resources. Auditor General of the Ville de Montr?al 32 2012 Annual Report Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 Performance Indicators 2.2 2.2. Performance Indicators 2.2. Performance Indicators During our audits, we often observe a lack of appropriate performance indicators to allow thorough administrative follow-up and suitable reporting. We think we should continue to present the indicators that appear most relevant for the reader to form an opinion about the performance of the Montr?al Bureau du v?rificateur g?n?ral (the BVG). These indicators are: o Number of reports issued; o Recommendation implementation rate; o Time use; o Equal access to employment; o Financial results. Number of Reports Issued Table 1 shows the number of reports produced in recent years for the audit of financial statements as well as for the value-for-money and information technology (IT) audit. Table 1 - Number of Reports Issued from 2008 to 2012 Annual report reference year 2008 2009 2010 2011 2012 Current financial year 12 11 13 9 21 Financial audit Previous financial years 5 2 3 1 15 Total Value-for-money and IT audit 17 13 16 10 36 8 7 9 12 13 The significant increase in the number of audit reports for the 2012 financial year is primarily the result of two factors. Firstly, the authorities were late in appointing joint auditors for the certification of financial statements of the Ville de Montr?al and the Soci?t? de transport de Montr?al for the 2011 financial year; as a result, the independent auditor reports were not issued until the period covered by the current annual report. Secondly, the amendment to section 107.7 of the Cities and Towns Act, which came into effect on Auditor General of the Ville de Montr?al 35 2012 Annual Report Chapter 2 Overview of the Bureau du v?rificateur g?n?ral January 1, 2011, gave the auditor general additional responsibility for auditing the accounts and affairs of every legal person that is part of the reporting entity defined in the municipality's financial statements. Auditor General of the Ville de Montr?al 36 2012 Annual Report 2.2. Performance Indicators Recommendations Implementation Rate This indicator is particularly important because it allows the reader to assess how quickly and to what extent the administration takes appropriate measures to resolve the problems that trigger our recommendations. It should be pointed out that the figures shown in Figure 1 illustrate, for each of the years in which the recommendations were made, their implementation rate in the year following their publication. Figure 1 - Recommendation Implementation Rate for Value-for-Money and IT Audits, by Status 69% Completed 57% Under way Deferred 35% Cancelled 23% Others 5% 1% 5% 2% 0% 2011 3% 2010 Years The indicator reveals a slowdown in the implementation of our recommendations by business units. A significant decrease can be seen in the proportion of recommendations "completed" among those published in the 2011 annual report in comparison with those published in the 2010 annual report. Auditor General of the Ville de Montr?al 37 2012 Annual Report Chapter 2 Overview of the Bureau du v?rificateur g?n?ral Time Use Figures 2 and 3 show the breakdown of total and chargeable hours for members of the BVG over the last five years. Figure 2 - Breakdown of Total Hours 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Vacation time Sick days 2008 2009 2010 Chargeable 2011 2012 Figure 3 - Breakdown of Chargeable Hours 70% 60% 50% 40% 30% 20% 10% 0% Financial statements VMA and IT 2008 Forensic accounting 2009 2010 Methods 2011 Administration 2012 VMA and IT: Value-for-money and information technology audits. Methods: Includes accounting research, quality control and training. Auditor General of the Ville de Montr?al 38 2012 Annual Report 2.2. Performance Indicators The time use data show that the distribution of chargeable hours remained relatively stable from 2011 to 2012 apart from a significant increase for administration. This increase is related to the time dedicated to the project to secure the BVG's email service and computer network as described in Chapter 1 of this report. Regarding fluctuations in the distribution of chargeable hours for the audit of financial statements and of VMA and IT in years prior to 2010, the reader is referred to the explanations provided in the 2011 annual report. Other indicators pertaining to time use and the staff turnover rate are presented in Table 2. Table 2 - Other Indicators Regarding Time Use and Audit Staff Turnover Results 1. Audit staff turnover 2. Absenteeism (audit professionals) 3. Number of training hours 4. Average number of hours of training per employee 5. Ratio of training costs/payroll expenditures in accordance with Bill 90 (the objective for all city operations is 1%) 2012 10.0% 1.2% 969 32 2011 9.4% 1.3% 1,047 33 2.7% 2.9% It can be seen in Table 2 that the data for 2012 are comparable to the results for 2011. Auditor General of the Ville de Montr?al 39 2012 Annual Report Chapter 2 Overview of the Bureau du v?rificateur g?n?ral Equal Access to Employment Like the city, the BVG pays particular attention to issues of equal access to employment. As of December 31 of the last five years, the breakdown of the representation of groups targeted by the Act respecting equal access to employment in public bodies was as shown in Table 3. Table 3 - Representation of Targeted Groups Targeted group Men Women Targeted group First Nations Visible minorities Ethnic minorities Total 2012 46.7% 53.3% 2012 0.0% 6.7% 3.3% 10.0% 2011 59.4% 40.6% 2010 58.1% 41.9% 2009 64.3% 35.7% 2008 63.3% 36.7% 2011 0.0% 6.3% 3.1% 9.4% 2010 0.0% 3.2% 0.0% 3.2% 2009 0.0% 3.6% 0.0% 3.6% 2008 0.0% 3.3% 0.0% 3.3% It can be seen that the representation of women, as well as of visible and ethnic minorities, has improved significantly in recent years. Auditor General of the Ville de Montr?al 40 2012 Annual Report 2.2. Performance Indicators Financial Results The final indicator concerns the financial results for the BVG. Figure 4 shows these results for the last five years. Figure 4 - Budget and Financial Results (in millions of dollars) 7 6 5 4 3 2 1 0 2008 2009 2010 Budget 2011 2012 Results Nothing in particular needs to be pointed out about the financial results for 2012 in comparison with 2011. For further information about preceding years, the reader is referred to the annual reports for those years. Auditor General of the Ville de Montr?al 41 2012 Annual Report Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 Investigative and Forensic Accounting Report 3 Chapter 3 Investigative and Forensic Accounting Report 3. Investigative and Forensic Accounting Report Background Since 2010, the Bureau du v?rificateur g?n?ral (BVG) has had an investigative and forensic accounting team in place to deal with the allegations of wrongdoing it receives. This team also supports other BVG units with respect to evidence of irregularities or illegalities resulting from its audit work. Until March 30, 2011, most of these allegations were brought to the BVG's attention by way of the auditor general's ethics hotline. Since the responsibility for the management of the hotline was transferred to the Service du contr?leur g?n?ral, these allegations have been accepted by mail, email, telephone and in person. It is important to emphasize that the BVG has a highly secure email account set up specifically for this purpose. This account is hosted by an external provider and is not linked to the city's email system. Allegations can therefore be emailed to verificateurgeneral@bvgmtl.ca. Auditor General of the Ville de Montr?al 45 2012 Annual Report Chapter 3 Investigative and Forensic Accounting Report Processing Allegations Allegations received are handled by the investigative and forensic accounting team according to priority criteria. A thoroughly documented and secure file is created for every allegation processed by the BVG. A preliminary evaluation is not conducted if the subject of the allegation does not fall within the auditor general's purview or if the allegation is not substantial enough to warrant further investigation. Depending on the circumstances, such matters may be transferred to another city entity or closed without any further action undertaken. All other allegations received undergo a preliminary evaluation to establish the suitability of carrying out an investigation, based on the nature of the claim, the probative value of the information provided and the risks involved. The BVG has no choice but to be selective in the files it decides to investigate given its limited resources and the volume of allegations received every year. As such, certain files, such as those whose material impact is negligible, may not be followed up on. If an investigation is deemed relevant, it will be performed in accordance with recognized practices in investigative and forensic accounting. Once the investigation is completed, an investigative and forensic accounting report may be produced so that the necessary measures can be taken by the municipal administration. In cases where there is evidence of criminal wrongdoing, the results of the BVG's work are communicated to police authorities. In 2012, we analyzed 41 allegations (compared with 36 in 2011). Of this number, 16 are still pending. A few of these are similar allegations forwarded to us by different parties. We have therefore grouped them together, so that we currently have 14 files pending. These files are at the preliminary evaluation stage, under investigation or being studied for future audits. A total of 25 files were closed during the year for one of the following reasons: o investigation concluded; o insufficient evidence; o transfer to another body for an allegation outside of the auditor general's purview. Auditor General of the Ville de Montr?al 46 2012 Annual Report Chapter 3 Investigative and Forensic Accounting Report Sources of Allegations Received Figure 1 shows the sources of the allegations received, 58% of which (24 out of 41) were submitted by employees or citizens. Figure 1 - Overview of Allegations Received in 2012 By Source Other 22% Employees 24% Suppliers 15% Citizens 34% Elected officials 5% Among the allegations received, 26 out of 41 (63%) were channeled through the BVG's email account. The remainder were reported via telephone or mail. It is interesting to note that only 11 allegations were made anonymously. In other words, 30 complainants out of 41 (73%) identified themselves, up significantly from 53% in 2011. When complainants disclose their identity, it is easier for us to obtain the additional information necessary to pursue our corroboration and investigation. It is important to point out that we guarantee confidentiality to those who agree to disclose their identity. Furthermore, under section 107.16 of the Cities and Towns Act, the auditor general cannot be compelled to give testimony relating to any information obtained in the performance of his duties or to produce any document containing such information. Moreover, wherever possible, those who come forward are protected against any form of reprisal. Auditor General of the Ville de Montr?al 47 2012 Annual Report Chapter 3 Investigative and Forensic Accounting Report Categories and Examples of Allegations Received We have grouped together the allegations received by category: o Breach of ethics, conflict of interest, breach of trust, favouritism (7 allegations); o Violation of laws, regulations or policies (10 allegations); o Misappropriation or theft (time, money, assets), use of resources for personal reasons (9 allegations); o Price gouging or product substitution, secret commissions, bribes (10 allegations); o Other (5). Figure 2 - Overview of Allegations Received in 2012 By Type Price gouging or product substitution, secret commissions, bribes 24% Other 13% Violation of laws, regulations or policies 24% Misappropriation or theft (time, money, assets), use of resources for personal reasons 22% Auditor General of the Ville de Montr?al Breach of ethics, conflict of interest, breach of trust, favouritism 17% 48 2012 Annual Report Chapter 3 Investigative and Forensic Accounting Report Table 1 presents a few examples of allegations that were communicated to us in 2012. Note that these allegations have not necessarily been confirmed by our investigations at this point. Table 1 - Examples of Allegations Received Types of allegations Breach of ethics, conflict of interest, breach of trust, favouritism Violation of laws, regulations or policies Misappropriation or theft (time, money, assets), use of resources for personal reasons Examples of allegations o Allegation of favouritism in the staffing competition process and allegation of o o o o o o o o conflicts of interest. Allegation of waiving developers' fees. Allegation of zoning changes that favour developers. Allegation of wrongdoing in the awarding of a contract without a call for tenders. Allegation of irregularities in the awarding of a professional services contract. Allegation of extreme delays in paying a supplier. Allegation of non-compliant expenses claimed by a body with relation to service agreements and subsidies received by the city. Allegation of time theft by employees. Allegation of poorly synced invoicing and collection systems within a city business unit. o Allegation of gifts offered to city employees to favour certain suppliers. o Allegation of mistaken calculations in the quantities requested and listed in tables Price gouging or product substitution, secret commissions, bribes o o of bid prices, which could potentially lead to extra charges billed by the selected contractor. Allegation of non-compliance with the contract award process and employee corruption. Allegation of a city representative wasting public funds. Auditor General of the Ville de Montr?al 49 2012 Annual Report Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 Financial Statement Audits 4 Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 Consolidated Financial Statements of Ville de Montr?al 4.1 4.1. Consolidated Financial Statements of Ville de Montr?al 4. Financial Statement Audits 4.1. Consolidated Financial Statements of Ville de Montr?al The Charter of Ville de Montr?al and the Cities and Towns Act (CTA) both require the city to submit its financial statements to the city clerk's office by March 31 following the close of the preceding year and to the Minist?re des Affaires municipales, des R?gions et de l'Occupation du territoire (MAMROT), using the prescribed form, by April 30. Consequently, for the fiscal year ended in 2012, we have produced a total of five audit reports regarding the city's financial statements. In March 2013, the independent auditor's reports of the auditor general of the Ville de Montr?al expressing an unmodified opinion on the city's consolidated financial statements and on the breakdown of the city's mixed expenditures were issued. Both reports were included in the Annual Financial Report filed with the city clerk's office on March 28, 2013. Also, in April 2013, reports on the city's consolidated financial statements, on the breakdown of mixed expenditures and on the city's overall tax rate were produced. These three reports were recorded on the form required by the MAMROT. The audit work on the city's consolidated financial statements was conducted together with a joint auditor who also expressed an unmodified opinion on the city's financial statements. In accordance with the provisions of the CTA, the form required by the MAMROT, along with the three abovementioned auditor general's reports and the joint auditor's report on the consolidated financial statements were filed with the city council and the urban agglomeration council before being sent to the MAMROT. We would like to remind that, given the delay in the appointment of our joint auditor by authorities in 2012, our five reports regarding the fiscal year ended in 2011 were also delivered during the financial period covered by this annual report. Auditor General of the Ville de Montr?al 55 2012 Annual Report Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 Financial Statements of Other Municipal Bodies 4.2 4.2. Financial Statements of Other Municipal Bodies 4.2. Financial Statements of Other Municipal Bodies In accordance with the Cities and Towns Act (CTA) in effect on December 31, 2012, we must audit the bodies' financial statements of every legal person that meets one of the following criteria: o It is part of the reporting entity defined in the municipality's financial statements; o The municipality or its representative appoints more than 50% of the members of its board of directors; o The municipality or its representative holds more than 50% of its outstanding voting shares or units. Table 1 on the following page lists the other municipal bodies on whose financial statements we are required to produce an audit report and indicates the periods during which the audit reports were delivered for the fiscal years ended in 2010, 2011 and 2012. Auditor General of the Ville de Montr?al 59 2012 Annual Report Chapter 4 Financial Statement Audits Table 1 - Other Municipal Bodies Governed by the CTA and Audit Reports Produced on their Financial Statements as of April 30, 2013 Fiscal year ended in Other municipal bodies 2012 2011 2010 Anjou 80 Bixi Toronto Inc. Conseil des arts de Montr?al Conseil interculturel de Montr?al Corporation d'habitation Jeanne-Mance Fiducie du Technoparc Montr?al Office de consultation publique de Montr?al Office municipal d'habitation de Montr?al Soci?t? de gestion Marie-Victorin Soci?t? de gestion du port de plaisance de Lachinea Soci?t? de jalonnement dynamique de Montr?al Soci?t? de transport de Montr?al (2 reports) Soci?t? de v?lo en libre-service Soci?t? en commandite Stationnement de Montr?al Soci?t? d'habitation et de d?veloppement de Montr?al Soci?t? du parc Jean-Drapeau Technoparc Montr?al Trangesco S.E.C. a New name since August 15, 2012. Formerly the Soci?t? de gestion NauBerges de Lachine. Legend: Reports produced between May 1, 2012, and April 30, 2013. Reports produced before May 1, 2012. No report to produce; bodies added to our reporting entity at a later date. Auditor General of the Ville de Montr?al 60 2012 Annual Report Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 Value-for-Money and Information Technology Audit 5 Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 Follow-Ups to Recommendations from Previous Years 5.1 5.1. Follow-Ups to Recommendations from Previous Years 5. Value-for-Money and Information Technology Audit 5.1. Follow-Ups to Recommendations from Previous Years The percentage of the recommendations made by the Bureau du v?rificateur g?n?ral (BVG) that received concrete corrective measures is an essential indicator to ensure that departments and boroughs implement these recommendations promptly. The policy of the BVG is to initiate a follow-up to the recommendations the year after they appear in the annual report. The follow-up to the recommendations for a given year generally extends over a maximum period of three years, except in very specific circumstances when some recommendations are monitored for an additional year or two. The results of the follow-up to the recommendations made in the annual reports for years 2005 to 2011 are shown in Table 1, with the exception of follow-ups related to recommendations involving the Soci?t? de transport de Montr?al (STM), which we discuss separately at the end of this section. Table 1 - Results of the Follow-Ups to Recommendations Status of the recommendations Completed Under way Deferred Cancelled Not done No longer valid Othera Total number of recommendations made a Number of recommendations per year 2005 182 - - 6 9 3 - 2006 228 - - - 13 3 - 2007 162 3 - 2 9 6 - 2008 64 3 - 1 2 1 - 2009 231 13 2 19 8 13 - 2010 149 24 2 - - - - 2011 79 128 10 3 - - 4 Total 1,095 171 14 31 41 26 4 200 244 182 71 286 175 224 1,382 New recommendation. For recommendations made in 2011, 92% have been "completed" or are "under way" compared with the municipal administration's annual objective of 80%. Auditor General of the Ville de Montr?al 65 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit The follow-up done in March and April 2013 for recommendations made since 2005 showed that 1,095 were completed, including 170 that were completed during the past 12 months. On a cumulative basis (over the last seven years), therefore, 79% of recommendations were completed, whereas 12% are under way. Follow-Up to the 40 Major Recommendations Made between 2005 and 2010 for which the Status was "Not done", "No Longer Valid" or "Other" as of March 31, 2012 In our 2011 annual report, sixty-two recommendations were shown with a status of "not done", "no longer valid" or "other - new recommendation" for the years 2005 to 2010. Of these 62 recommendations, we identified 40 as being major (28 "not done" or "no longer valid", and 12 "other - new recommendation".) Following public review of our annual report by the Commission permanente sur les finances et l'administration in August 2012, and as we had recommended, the commission mandated the Direction g?n?rale to: o set priorities with the units involved so that satisfactory results could be achieved, within a year, in following up on the 40 recommendations of previous years that the Bureau du v?rificateur g?n?ral deemed to be important; o ensure that the action plans submitted by the various activity sectors of the city were updated. During our audit of the follow-up of the recommendations made between 2005 and 2011, we looked into the actions taken by the business units involved and the information collected for that purpose by the Direction g?n?rale in order to determine the degree to which the 40 major recommendations had been implemented. While we are aware that the business units have a fixed date in August 2013 to complete implementing the recommendations that concern them, the interim audit that we conducted in March and April 2013, revealed the results presented in Table 2. Auditor General of the Ville de Montr?al 66 2012 Annual Report 5.1. Follow-Ups to Recommendations from Previous Years Table 2 - Follow-Up to the 40 Major Recommendations 2011 situation Status Number Other - new recommendation 12 Updated situation - April 2013 Status Number Completed 7 28 Total 40 4 Deferred Not done or no longer valid Under way 1 Not done or no longer valid Completed Total 26 2 40 We intend to re-examine the follow-up of the implementation of these recommendations at a later time after the completion date set in August 2013 by the Commission permanente sur les finances et l'administration. Follow-Up to the Recommendations - Soci?t? de transport de Montr?al We also looked into the follow-up of the recommendations stemming from the audits conducted in 2010 involving the STM. It should be recalled that these audits dealt with tests of physical intrusion into its facilities and the integrated control system of the Montr?al metro. The results of this follow-up are presented separately since the STM is not subject to the accountability methods established by the municipal administration, but by those set by its board of directors. The results of follow-ups to the 17 recommendations issued as part of these audits are presented in Table 3. Table 3 - Results of the Follow-Up to the Recommendations in Audits Involving the STM Number of recommendations Status of recommendations Completed Under way Total number of recommendations made Auditor General of the Ville de Montr?al 67 2010 6 11 17 2012 Annual Report Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 Asset Disposal 5.2 5.2. Asset Disposal Table of Contents 1. Introduction ..................................................................................................................... 73 2. Audit Scope..................................................................................................................... 74 3. Findings and Recommendations .................................................................................... 75 3.1. Disposal of Computer Equipment ........................................................................... 75 3.2. Disposal of Vehicles and Rolling Stock ................................................................... 91 Auditor General of the Ville de Montr?al 71 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit List of Acronyms DMRA STI Division de la gestion du parc de v?hicules Service de police de la Ville de Montr?al Service des technologies de l'information Darik's Boot and Nuke DGPV ordinateurs pour les ?coles du Qu?bec Centre de services partag?s du Qu?bec DBAN not-for-profit organization SPVM business and recycling training centre CSPQ NPO OPEQ CFER Direction du mat?riel roulant et des ateliers municipaux Auditor General of the Ville de Montr?al 72 2012 Annual Report 5.2. Asset Disposal 5.2. Asset Disposal 1. Introduction The business units of the Ville de Montr?al (the city), i.e., the central departments and boroughs, use a variety of assets to carry out their activities and deliver their services. These assets include computer equipment and vehicles. The value of these assets depreciates over time as they are used. Once their useful life is over, the business units must dispose of them, primarily by auctioning them off, giving them away or discarding them. Under the Cities and Towns Act, 1 the city has the authority to dispose of assets that have become obsolete and cannot be repurposed. Moreover, in sections 34 and 35 of the Charter of Ville de Montr?al, 2 city council has delegated to the executive committee and borough councils the power to dispose of surplus movable property that is no longer required in the execution of their operations. The executive committee has in turn delegated these powers to central department officers and employees to make the asset disposal process more efficient. These responsibilities must also be implemented in compliance with the By-law concerning the delegation of powers to officers and employees (RCE 02-004) enacted on June 26, 2002, and with the administrative framework C-RM-SAC-D-09-001 entitled [TRANSLATION] "Disposal of movable assets, excluding vehicles, by the central departments," which came into force on June 23, 2009. This framework defined the scope of these responsibilities, i.e., all of the movable property available in the city's central departments, as well as the terms of disposal, i.e., whether or not a monetary consideration is involved. It also indicated that the responsibility for the application of the framework as it pertains to computer equipment fell to the Service des technologies de l'information (STI). Note that this framework excludes vehicles and does not extend to the boroughs. The new administrative framework (C-RM-SCARM-D-12-002) incorporates the principles of the waste management plan, which recommends that business units dispose of their movable property in a manner consistent with the following sequence: o attempt to repurpose the asset within another administrative or business unit; o sell the asset; o give the asset away; 1 2 RSQ, chapter C-19, section 28. RSQ, chapter C-11.4. Auditor General of the Ville de Montr?al 73 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit o o recycle the asset through recycling organizations to recover materials such as metal, wood and glass; discard the asset if all previous options have been exhausted. The framework also defines the central departments' responsibilities based on the method of disposal, i.e., sale (auction, private sale or public tender) or donation (when goods cannot be repurposed within another unit and their residual value is too low to warrant resale, they may be donated to a not-for-profit organization [NPO]). In either case, managers need to confirm obsolescence, initiate disposal procedures and estimate the fair value of any items to be sold. This represents a logical approach to optimizing financial resources in that the effective management of the asset disposal process can help obtain the best possible price for these surplus goods and the highest possible return for the city. Until December 31, 2008, when the municipal pound was closed, the central departments and boroughs disposed of their surplus goods in a centralized way, through public auctions at the rue Louvain site, under the supervision of the Direction de l'approvisionnement. At present, each business unit makes its own arrangements, based on established delegation rules. However, the responsibility for disposing of vehicles from the central departments falls to the Direction du mat?riel roulant et des ateliers municipaux (DMRA). The DMRA can also be specially tasked by certain other boroughs to dispose of their vehicles on their behalf. 2. Audit Scope The purpose of our audit was to ensure that the city receives fair value for its assets, that surplus goods can no longer be used in a cost-effective manner by the city and that disposals are carried out in accordance with applicable laws and by-laws. We also wished to ensure that the data contained on hard drives were securely destroyed before computers were disposed of. We therefore reviewed the processes involved in disposing of the following categories of assets: o computer equipment; o vehicles and rolling stock. Auditor General of the Ville de Montr?al 74 2012 Annual Report 5.2. Asset Disposal The audit work began in fall 2012 and focused primarily on the disposal of these two types of goods in 2011 and 2012 by business units selected in accordance with the following criteria: o Their level of involvement in the asset disposal process: - the STI (computer equipment); - the DMRA (vehicles), which comes under the Service de la concertation des arrondissements et des ressources mat?rielles; o The value of the goods under their responsibility and their status within the city (central departments, boroughs of the former Ville de Montr?al or boroughs created from the former suburban municipalities): - the Service de police de la Ville de Montr?al (SPVM); - Ville-Marie borough; - Saint-Laurent borough. For the disposal of computer equipment, and the destruction of hard drive data in particular, we decided to broaden our sample to include nine other boroughs created from the former suburban municipalities (in addition to Saint-Laurent borough) to ensure that: o all sensitive and confidential information is wiped from computers prior to disposal; o an effective data destruction procedure is in place and is adhered to. The former suburban municipalities are each responsible for these activities. In the case of the boroughs of the former Ville de Montr?al, however, data destruction is overseen by the STI. 3. Findings and Recommendations 3.1. Disposal of Computer Equipment The Cities and Towns Act, the Charter of Ville de Montr?al, the By-law concerning the delegation of powers to officers and employees and administrative framework C-RM-SACD-09-001, which came into force on June 23, 2009, and was replaced by framework C-RMSCARM-D-12-002 on October 22, 2012, are the main instruments governing asset disposal. Specifically as concerns IT assets, the STI also abides by an information security policy (C-OG-SSA-PA-06-001), dated June 22, 2006, and a procedure regarding data destruction technology (DSI-SecOp-0005), dated March 31, 2010. In addition, the STI is subject to a security directive entitled [TRANSLATION] "Directive concerning the destruction of information stored on magnetic or electronic media prior to disposal," which has been in effect since Auditor General of the Ville de Montr?al 75 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit November 1, 2000, and which requires data to be destroyed before any computer equipment is disposed of. We examined whether the computer equipment disposal process, as it is implemented by the STI and the business units (central departments and boroughs), was consistent with the regulatory framework and whether security rules related to the destruction of sensitive data were in place and strictly adhered to. 3.1.1. Role and Responsibilities of the Service des technologies de l'information 3.1.1.A. Background and Findings Since the municipal pound closed, the STI has been required to dispose of obsolete computer equipment (e.g., micro-computers, laptops, servers, printers, monitors) on a regular basis. In this context, the Direction g?n?rale approved the directive entitled [TRANSLATION] "Disposal of movable goods, excluding vehicles, by the central departments" (C-RM-SAC-D-09-001), making each department responsible for the disposal of these items. This directive applies to all central departments, although section 3 stipulates [TRANSLATION]: "Computer hardware is to be remitted to the Direction des syst?mes d'information [now the STI] for disposal." The responsibility for identifying the computer equipment to be disposed of (after five years) for all of the city's business units (central departments and 19 boroughs) has been delegated to the STI. Every year, the STI provides the business units with a list of obsolescent assets 3 indicating the workstations due for replacement. Moreover, the STI, by way of the Division des services aux utilisateurs, offers a data and sensitive information destruction service to erase the contents from hard drives, in accordance with the applicable procedure, before computers are donated to an NPO. This service is currently used by all of the city's central departments, with the exception of the SPVM and the following administrative units: the Direction associ?e--Biblioth?ques and the nature museum units (Botanical Garden, Biod?me, Insectarium and Planetarium) . As for the boroughs, only those of the former city and LaSalle use this service. The other boroughs created from the former suburban municipalities see to the destruction of their own data. 3 The STI obsolescence program works with a 20% annual computer replacement rate, based on a useful life cycle of five years. Auditor General of the Ville de Montr?al 76 2012 Annual Report 5.2. Asset Disposal Surplus computer equipment is generally donated to an NPO, i.e., the OPEQ 4 , pursuant to an agreement approved by the city on June 18, 2009 for a three-year period (2009-2012) and which was recently renewed for an additional three years. The basic process in place at the STI is: o the STI sends the list of obsolescent assets to the city's business units indicating the workstations due to be replaced and the computer equipment due to be decommissioned; o a decision-making summary is prepared describing each asset and listing the market value for each disposal (generally a zero value) and the identity of the recipient organization (OPEQ); o the Division des services aux utilisateurs forwards the hard drives recovered by an STI technician (this activity applies to the units that use the data destruction service offered by the STI). The Division des services aux utilisateurs proceeds with the destruction of the data on all the hard drives received before sending computers to the OPEQ for reuse. This is done in accordance with the procedure 5 adopted by the STI. DBAN 6 software is used to ensure information is erased in a secure manner. Unfortunately, once the data is wiped from the hard drive, DBAN does not generate any reports or formal documents certifying that the contents have indeed been completely destroyed, in accordance with the practices advocated by the National Institute of Standards and Technology in its Guidelines for Media Sanitization. 3.1.1.B. Recommendation We recommend that the Division des services aux utilisateurs of the Service des technologies de l'information produce or obtain a "secure data destruction certificate" that contains information about: o the erased hard drive (reference number and source); o the data destruction method (e.g., name of the software used); o the name and signature of the individual who destroyed the data and the date of the operation; o the name of the manager responsible for the operation; to formally document that the data have been properly destroyed and confirm that best practices have been followed. 4 5 6 Ordinateurs pour les ?coles du Qu?bec. [TRANSLATION] Procedure related to data destruction technology (DSI-SecOp-0005). Darik's Boot and Nuke. Auditor General of the Ville de Montr?al 77 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Business unit's response: [TRANSLATION] In order to ensure adherence to best practices, the current process will be enhanced as follows: o Both the computer and hard drive will be identified upon decommissioning (using existing tools (bar code scanning)). To date, only the computer had been identified in this manner; (Planned completion: September 2013) o The computer used to sanitize hard drives is not linked to the city network and cannot produce an electronic sanitization certificate. To make it possible to validate the source of the hard drives and associate them with a given computer, we will develop an application to associate the serial number of the hard drive (DSM source) with the computer in order to confirm destruction and subsequently produce a report; (Planned completion: September 2013) o The person who carried out the data destruction operation will put their name, signature and date on the resulting report; (Planned completion: September 2013) o The manager responsible for the operation will double-check the form and formally certify that the data have been properly destroyed. (Planned completion: September 2013) 3.1.2. Role and Responsibilities of the Audited Business Units For the boroughs of the former Ville de Montr?al and the central departments (except the SPVM), the STI technician assigned to the corresponding business unit is responsible for the physical disposal of computer equipment once the hard drives have been stripped. This process generally involves the following steps: o the list of obsolescent assets is checked and confirmed by the business unit and returned to the STI; o the decision-making summary is drafted by the corresponding business unit; o the STI technician recovers the decommissioned hard drive-less computers and stores them on the premises of the corresponding unit; o the inventory is updated; o the OPEQ is contacted to set a date for picking up the decommissioned computer equipment. 3.1.2.1. Service de police de la Ville de Montr?al 3.1.2.1.A. Background and Findings The physical disposal of the SPVM's computer equipment is carried out by the STI. In other words, the STI recovers the decommissioned computers and passes them along to the OPEQ. This operation is documented in a decision-making summary. Auditor General of the Ville de Montr?al 78 2012 Annual Report 5.2. Asset Disposal The SPVM's hard drives are destroyed mechanically by the SPVM, not the STI. However, based on the information we obtained, there are some shortcomings in the way the hard drives are handled, between the time of their removal from the computer case and their actual destruction: o the hard drives removed from the computer case are not identified, making it impossible to subsequently determine which computers they came from; o a lost hard drive would be impossible to detect. Currently, the SPVM's decommissioned hard drives are placed in a box by an STI technician and sent to the SPVM's Section des archives for destruction. The boxes are tracked, but their contents are not. If a hard drive were to be lost or stolen, it would go undetected. A review of the list of the obsolete equipment inventory in the SPVM's units shows a lack of information about the identification of the hard drives taken from the computers indicated on the list. 3.1.2.1.B. Recommendation We recommend that the Service de police de la Ville de Montr?al implement controls to ensure the secure handling of hard drives once they are removed from their respective computer cases and sent to the Section des archives for destruction. Business unit's response: [TRANSLATION] The Section des archives will add the following obligations to the hard drive destruction process: o complete form F 470-08, indicating the inventory of hard drives to be destroyed, the name of the computer each one is taken from and the serial number, along with the total number of hard drives to be destroyed; o send the hard drives, along with completed F 470-08 form, in a sealed box. A tracking number will be sent to the STI to follow up on the shipments sent for destruction. The Section des archives will then check the inventory prior to destruction and forward a copy of the destruction certificate to the STI, along with a list of the destroyed hard drives. The Section des archives will keep a copy of the hard drive inventory with the original destruction certificate, in accordance with records retention schedule rules. (Planned completion: September 2013) Auditor General of the Ville de Montr?al 79 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.1.2.1.C. Recommendation We recommend that the Service des technologies de l'information, in conjunction with the Service de police de la Ville de Montr?al, develop a hard drive identification mechanism to be able to track hard drives at any point in the destruction process. Business units' combined response: [TRANSLATION] The inventory of hard drives to be destroyed, the name of the computer each was taken from and the total number of hard drives to be destroyed will be indicated on form F 470-08, and a copy will be included with the hard drives in a sealed box. A second copy will be kept by the STI. The STI will receive a copy of the destruction certificate from the Section des archives once the operation is complete. This copy will be kept on file with the inventory form listing the destroyed hard drives, in accordance with records retention schedule rules. (Planned completion: September 2013) 3.1.2.2. Ville-Marie Borough 3.1.2.2.A. Background and Findings The Direction des services administratifs, du greffe et des relations avec les citoyens (Division des ressources financi?res et mat?rielles) is responsible for the physical disposal of computer equipment. All operations of this nature are first authorized by the borough council in a decision-making summary. However, these decision-making summaries do not contain a full, detailed list of the assets to be disposed of. The manager simply indicates the approximate number of computers to be decommissioned on an annual basis. 3.1.2.2.B. Recommendation We recommend that the Direction des services administratifs, du greffe et des relations avec les citoyens in Ville-Marie borough systematically attach a detailed list of the computer equipment due for decommissioning to the decision-making summary in order to comply with the administrative framework on asset disposal. Business unit's response: [TRANSLATION] As specified in contract 832034, covering January 1, 2013, to December 31, 2017, Ville-Marie borough will prepare a decision-making summary to approve the recycling of computers, accessories and peripherals by the OPEQ. Auditor General of the Ville de Montr?al 80 2012 Annual Report 5.2. Asset Disposal A list of assets, including a description, serial number and inventory number, will be attached to the decision-making summary. The first summary, bearing the number 1133200001, will be presented to the borough council with the requested modifications. (Completed, March 2013) 3.1.2.3. Saint-Laurent Borough 3.1.2.3.A. Background and Findings Saint-Laurent borough is not one of the boroughs that use the services of the STI's Division des services aux utilisateurs to erase the hard drives of the computers it donates to NPOs. The entire disposal process is carried out within the borough itself. The computers are donated to the OPEQ for refurbishing and distribution to schools. However, there are no decision-making summaries to authorize this donation, and there are no lists kept of the computers disposed of in this manner. Based on the information we obtained, computers are transferred to the OPEQ with their hard drives intact. There is no evidence that the data on these hard drives have been properly destroyed. Instead, the Direction des services administratifs et du greffe (Division des ressources informationnelles) entrusts the OPEQ with this operation, as the NPO indicates on its website that it guarantees data erasure and privacy, although it does not produce any formal documents certifying that the data have indeed been destroyed. 3.1.2.3.B. Recommendation We recommend that the Direction des services administratifs et du greffe in SaintLaurent borough keep records of all computer equipment donations in a decisionmaking summary that indicates which equipment has been given away and the name of the recipient organization. Business unit's response: [TRANSLATION] An email has been sent to all employees in Saint-Laurent borough to remind them that the disposal of surplus goods must be authorized by a resolution of the borough council and, consequently, absolutely must be covered in a decisionmaking summary. A decision-making summary will be issued for every shipment of donated computer equipment, including a detailed list of the assets involved and the recipient organization. (Completed, March 2013) Auditor General of the Ville de Montr?al 81 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.1.2.3.C. Recommendation We recommend that the Direction des services administratifs et du greffe in SaintLaurent borough proceed with the systematic, secure destruction of the data contained on the hard drives of computers donated to the OPEQ (Ordinateurs pour les ?coles du Qu?bec) in order to limit the risk of confidential borough information being leaked. Business unit's response: [TRANSLATION] The data contained on the hard drives of the computers donated to the OPEQ will be systematically and securely destroyed using the STI's data destruction services. (Completed, March 2013) 3.1.2.3.D. Recommendation We recommend that the Direction des services administratifs et du greffe in SaintLaurent borough consider the relevance of using the data destruction service offered by the Division des services aux utilisateurs of the Service des technologies de l'information. Business unit's response: [TRANSLATION] We have decided to make use of the optional data destruction service offered by the STI's Division des services aux utilisateurs. Based on the destruction procedure agreed upon with the STI, we will remove all hard drives from the computers slated for disposal and, once the borough council has given its approval, we will place them in a box and send them to the STI for destruction. (Completed, March 2013) 3.1.3. Role and Responsibilities of the Boroughs Created from the Former Suburban Municipalities 3.1.3.A. Background and Findings The boroughs created from the former suburban municipalities are in charge of disposing of their own surplus computer equipment. Their role involves validating the list of obsolescent assets issued by the STI. They must also prepare decision-making summaries to obtain the authorization to donate computers to the OPEQ or another NPO of their choosing. In addition, they are responsible for destroying the data contained on hard drives, physically disposing of the computer equipment and updating their inventory. Auditor General of the Ville de Montr?al 82 2012 Annual Report 5.2. Asset Disposal For each of these boroughs, we made sure that: o all donations to an NPO were duly authorized by the borough council by way of a decision-making summary, supported by a list of the corresponding assets and the name of the recipient NPO; o the assets were disposed of in a secure manner, i.e., the data found on the hard drives were properly destroyed. Table 1 presents an overview of the each borough's position in both respects. Auditor General of the Ville de Montr?al 83 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Table 1 - Disposal of Computer Equipment in the Nine Boroughs Created from the Former Suburban Municipalities Borough Authorizations obtained for the donation of assets to an NPO (decision-making summary, list of assets and identification of recipient organization) Hard drive data destroyed in a secure way using proven methods Verdun No decision-making summary No list of computer equipment Recipient NPO not identified Computers discarded directly in the trash in 2011 DBAN data destruction software used No data destruction certificate produced Anjou No decision-making summary No list of computer equipment Recipient NPOs: OPEQ and Ordi Occasion (Anjou-based) CopyWipe data destruction software used No data destruction certificate produced Decision-making summary produced List of computer equipment produced Recipient NPO: CFERa of Pierrefonds Data destroyed by CFER Certificate of destruction provided to the borough No decision-making summary No list of computer equipment Recipient NPO: none, as the OPEQ wanted to charge $45 because of the small number of computers to be recycled (6) Computers discarded directly in the trash in 2011 WipeDisk data destruction software used No data destruction certificate produced L'?le-Bizard- SainteGenevi?ve No decision-making summary No list of computer equipment Recipient NPO: OPEQ Data destruction entrusted to an outside firm No data destruction certificate produced Saint-L?onard No decision-making summary No list of computer equipment Recipient NPO: OPEQ Physical data destruction done by the borough No data destruction certificate produced PierrefondsRoxboro Outremont No decision-making summary, purchase order used Montr?al-Nord No list of computer equipment Recipient NPO: OPEQ DBAN data destruction software used No data destruction certificate produced LaSalle Data destroyed by the borough by reformatting hard drives No certificate of data destruction produced Opted in to the data destruction service offered by the Division des services aux utilisateurs as of 2013 Lachine a No decision-making summary No list of computer equipment attached to the decision-making summary Recipient NPO: OPEQ No decision-making summary No list of computer equipment Recipient NPO: OPEQ Hard drives destroyed by the borough Partition Manager data destruction software used No data destruction certificate produced Business and recycling training centre. In the nine boroughs we audited, we ascertained that: o seven boroughs, i.e., all except Pierrefonds-Roxboro and LaSalle boroughs, do not produce decision-making summaries authorizing the donation of computer equipment; o only Pierrefonds-Roxboro borough attaches a list of computer equipment slated for disposal in its decision-making summary; o seven boroughs (all but Verdun and Outremont boroughs) donate their computer equipment to an NPO recognized by the city or the borough; Auditor General of the Ville de Montr?al 84 2012 Annual Report 5.2. Asset Disposal o o four boroughs (Verdun, Anjou, Outremont and Montr?al-Nord) use secure software to erase data, but none of the programs used generates a secure data destruction certificate; two boroughs (Pierrefonds-Roxboro and L'?le-Bizard-Sainte-Genevi?ve) outsource data destruction. Of these, only Pierrefonds-Roxboro borough receives a formal data destruction certificate. Furthermore, the STI has introduced an integration plan to promote its data destruction service to the boroughs that do not yet prevail themselves of this option. LaSalle borough signed up for the service in 2013. 3.1.3.B. Recommendation We recommend that Verdun, Anjou, Outremont, L'?le-Bizard-Sainte-Genevi?ve, SaintL?onard, Montr?al-Nord, LaSalle and Lachine boroughs document all donations of computer equipment in a decision-making summary, which includes the list of donated assets and the name of the recipient organization. Business units' responses: VERDUN BOROUGH [TRANSLATION] The Section de l'informatique of Verdun borough will undertake the measures necessary to document the next disposal of surplus assets, which should occur in the summer, in a decision-making summary that will include the list of donated assets as well as the recipient organization. (Completed, March 2013) ANJOU BOROUGH [TRANSLATION] Before the auditor general's report, we had already planned to produce a decision-making summary for asset disposals in 2013 involving computer equipment decommissioned from the end of 2012 onward, including a list of assets to be donated as well as the name of the recipient organization. (Completed, January 2013) OUTREMONT BOROUGH [TRANSLATION] We discussed the matter with our IT technician, and we agreed that an Excel spreadsheet would be used to track all the computer equipment that is destroyed or donated. Moreover, a decision-making summary will be prepared annually and a copy of the list will be attached to it. (Completed, March 2013) Auditor General of the Ville de Montr?al 85 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit L'?LE-BIZARD-SAINTE-GENEVI?VE BOROUGH [TRANSLATION] A decision-making summary will be prepared for the borough council before any computer equipment is given away. (Completed, April 2013) SAINT-L?ONARD BOROUGH [TRANSLATION] From now on, when computer equipment is given away, a decisionmaking summary will be presented to the council, along with the list of assets to be donated and the name of the recipient organization. (Completed, April 2013) MONTR?AL-NORD BOROUGH [TRANSLATION] This information is already available, and we will include it in a decision-making summary from now on. (Planned completion: Before the next collection of IT equipment) LASALLE BOROUGH [TRANSLATION] We will include the list of equipment to be donated in the next decision-making summaries. (Completed, March 2013) LACHINE BOROUGH [TRANSLATION] A decision-making summary will be prepared when the computer equipment is disposed of, along with a list of assets and their serial numbers. This summary will also mention that the equipment will be donated to the recipient organization pursuant to agreement 832034 undertaken with the OPEQ. (Completed, March 2013) 3.1.3.C. Recommendation We recommend that L'?le-Bizard-Sainte-Genevi?ve and Pierrefonds-Roxboro boroughs proceed with the systematic, secure destruction of the data contained on decommissioned hard drives and, as appropriate, that this destruction be done by the borough itself to limit the risk of confidential information being leaked. Business units' responses: L'?LE-BIZARD-SAINTE-GENEVI?VE BOROUGH [TRANSLATION] We do not have the human resources to ensure the secure destruction of data on an in-house basis. The work will be carried out in our offices. Auditor General of the Ville de Montr?al 86 2012 Annual Report 5.2. Asset Disposal PIERREFONDS-ROXBORO BOROUGH [TRANSLATION] We will proceed with the systematic, secure destruction of hard drive data. A written procedure will be drafted. The procedure will cover the following: o each hard drive and the originating computer will be identified (labelled); o the data will be destroyed by an IT employee and entered on a sheet that will be checked and signed by a third party; o files will be kept for subsequent verification. (Planned completion: May 2013) 3.1.3.D. Recommendation We recommend that Verdun, Anjou, Outremont, L'?le-Bizard-Sainte-Genevi?ve, SaintL?onard, Montr?al-Nord, LaSalle and Lachine boroughs produce or obtain, as the case may be, a "secure data destruction certificate" containing information on: o the hard drive that was erased (reference number and source); o the data destruction method used (e.g., name of the software); o the name and signature of the person who destroyed the data and the date the operation was performed; o the name of the person in charge; to formally document data destruction operations and comply with best practices. Business units' responses: VERDUN BOROUGH [TRANSLATION] A hard drive decommissioning form will be produced for every drive removed from a computer earmarked for disposal. These forms will be completed by the person who destroys the residual hard drive data and will contain the following information: o the serial number of the hard drive and the computer; o the data destruction method; o the name and signature of the person who destroyed the data; o the date of destruction; o the name of the head of the Section de l'informatique. (Completed, March 2013) ANJOU BOROUGH [TRANSLATION] From now on, we will complete a form with: o the hard drive information (serial number, make, model, originating computer); o the name of the software used; o the dates the data were destroyed and checked; Auditor General of the Ville de Montr?al 87 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit o o o the name of the technician who erased the drive (with signature); the name of the technician who checked that the data was destroyed (with signature); the report generated by the software, enclosed in the file. (Completed, April 2013) OUTREMONT BOROUGH [TRANSLATION] Our Excel spreadsheet will contain a section with the required information: o the name of software used; o the name of the person who destroyed the data; o the date of destruction; o the reference number and source. (Completed, March 2013) L'?LE-BIZARD-SAINTE-GENEVI?VE BOROUGH [TRANSLATION] We will obtain a secure data destruction certificate from our service provider. (Completed, March 2013) SAINT-L?ONARD BOROUGH [TRANSLATION] For upcoming computer decommissioning operations involving a hard drive, we will produce a "secure data destruction certificate" containing the following information: o the reference number and source of the hard drive; o the description of the data destruction method, including the name of the software used; o the name and signature of the person who destroyed the data and the date; o the name and signature of the person responsible for IT asset disposal. The software to be used to destroy the data will comply with the best practices established by the National Institute of Standards and Technology. Data destruction certificates will be kept in a registry. (Completed, April 2013) MONTR?AL-NORD BOROUGH [TRANSLATION] We will produce a certificate listing the make, model, serial number, source, data destruction software, date of destruction and name of the person who destroyed the data. As a first step, a template will be sent out by email by the IT technicians. The email confirmation will serve as a substitute for an electronic signature. As a second step, we will integrate a data destruction form into our automated inventory system. This system will then flag any attempts to repurpose a computer for which the data destruction rules have not been heeded. (Completed, March 2013) Auditor General of the Ville de Montr?al 88 2012 Annual Report 5.2. Asset Disposal LASALLE BOROUGH [TRANSLATION] Now that we have decided to use the STI's data destruction service, the onus to produce a destruction certificate will be on the STI. We will still create a list that contains the following information for the hard drives sent to the STI for destruction: o the reference number and source of hard drive; o the date shipped to the STI; o the name of the person responsible for the shipment. (Completed, March 2013) LACHINE BOROUGH [TRANSLATION] Hard drives will be sent to the STI to have their data wiped, and the STI will issue the destruction certificate. We will list the serial numbers of the hard drives sent to the STI and will keep this list in our files. (Completed, March 2013) 3.1.3.E. Recommendation We recommend that the boroughs of Verdun, Anjou, Pierrefonds-Roxboro, Outremont, L'?le-Bizard-Sainte-Genevi?ve, Saint-L?onard, Montr?al-Nord and Lachine boroughs, whether or not they destroy their data themselves, consider the relevance of using the data destruction service offered by the Division des services aux utilisateurs of the Service des technologies de l'information to ensure data are erased in a secure manner. Business units' responses: VERDUN BOROUGH [TRANSLATION] The Section de l'informatique of Verdun borough could use the data destruction service of the STI's Division des services aux utilisateurs, based on the availability of the STI's internal resources. (Completed, March 2013) ANJOU BOROUGH [TRANSLATION] We will continue to ensure hard drive data are effectively erased within the borough, in compliance with the aforementioned recommendations. (Completed, March 2013) PIERREFONDS-ROXBORO BOROUGH [TRANSLATION] We will proceed with the systematic and secure destruction of hard drive data internally. A written procedure will be drafted. The procedure will cover the following: o each hard drive and the originating computer will be identified (labelled); Auditor General of the Ville de Montr?al 89 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit o o the data will be destroyed by an IT employee and entered on a sheet that will be checked and signed by a third party; files will be kept for subsequent verification. (Planned completion: May 2013) OUTREMONT BOROUGH [TRANSLATION] Our position is to involve as few people as possible and do the work within the borough to retain complete control. (Completed, March 2013) L'?LE-BIZARD-SAINTE-GENEVI?VE BOROUGH [TRANSLATION] We are looking into the relevance of using the STI's data destruction service. (Planned completion: May 2013) We are currently in talks with the STI to obtain a proposal. We met with one of their representatives, and we are still awaiting their response. (Planned completion: December 2013) SAINT-L?ONARD BOROUGH [TRANSLATION] Saint-L?onard borough will destroy the data on all computer media including hard drives, ROM memory devices and flash memory devices installed on its own computers earmarked for disposal, using a secure data destruction software program. A rigorous procedure will be implemented, in adherence with all of the aforementioned recommendations. (Completed, April 2013) MONTR?AL-NORD BOROUGH [TRANSLATION] We use the same software as the STI to destroy data, and the operation is not a complex one. The logistics involved in getting the STI to perform the task therefore seem unnecessary. LACHINE BOROUGH [TRANSLATION] Hard drives will be sent to the STI to be erased so the STI can issue a destruction certificate. We will list the serial numbers of the hard drives sent to the STI and keep a copy of this list in our files. (Completed, March 2013) Auditor General of the Ville de Montr?al 90 2012 Annual Report 5.2. Asset Disposal 3.2. Disposal of Vehicles and Rolling Stock The Cities and Towns Act 7 stipulates: Unless otherwise provided, no property of a municipality may be alienated otherwise than in return for valuable consideration. Each month the clerk of a municipality must publish a notice concerning the properties with a value greater than $10,000 that were alienated by the municipality otherwise than by auction or public tender. The notice must describe each property and indicate for each the price of alienation and the identity of the purchaser. Pursuant to a delegation by-law, namely the By-law concerning the delegation of powers to officers and employees (RCE 02-004), the DMRA is responsible for disposing of vehicles used by city departments. This by-law addresses the rules for delegating authority, including those that bestow the power to manage the central departments' fleet of vehicles upon the DMRA. The DMRA offers a road vehicle disposal service to all of the city's business units. The central departments are required to use this service; it is optional for the boroughs. That said, only the boroughs of Montr?al-Nord, Lachine, Villeray-Saint-Michel-Parc-Extension, Rosemont-La Petite-Patrie, Le Plateau-Mont-Royal, Rivi?re-des-Prairies-Pointe-auxTrembles and Mercier-Hochelaga-Maisonneuve have called upon the DMRA to dispose of all or some of their road vehicles. The revenue generated by the sale of these vehicles is remitted to their owners (central departments or boroughs, as applicable) on a regular basis. We reviewed the processes for disposing of vehicles used by the DMRA and the boroughs as well as the associated procedures and documentation in order to ensure that the city is decommissioning its vehicles in accordance with existing directives and administrative frameworks and that an evaluation or estimation is performed before any operation of this nature occurs in order to obtain fair value. 8 7 8 Section 28, subsection 1.0.1. Financial accounting standards define fair value as "the amount at which an asset could be bought or sold in a current transaction between willing parties." In this case, the fair value represents the value obtained at auction. Auditor General of the Ville de Montr?al 91 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.2.1. Role and Responsibilities of the Direction du mat?riel roulant et des ateliers municipaux 3.2.1.A. Background and Findings The DMRA's role as the body in charge of vehicle management for the central departments is clearly defined in the delegation by-law. The head of the Division de la gestion du parc de v?hicules (DGPV) of the DMRA has been designated as the person responsible for the process of disposing of vehicles and putting them up for auction. A mechanism is in place for producing a monthly status report on the DMRA's sales transactions. The DGPV is in charge of the vehicle disposal process and as such: o selects the disposal method; o updates the inventory of vehicles; o arranges for the vehicles to be towed from the business units to the auction site; o returns the license plates to the Soci?t? de l'assurance automobile du Qu?bec. The vehicle disposal process can take one of several forms. The DGPV can put the vehicle up for auction, using the services of auctioneers selected as a result of a market study conducted in 2009 when it took over from the municipal pound. The main auctioneers are Les Encans Ritchie Bros, Encan d'auto Impact and the Centre de services partag?s du Qu?bec (CSPQ). Table 2 summarizes all the disposals carried out by the DGPV on behalf of the central departments and boroughs involved since 2009. Table 2 - Vehicles Disposed of by the DGPV since 2009 Auctioneer 2009 Qty 2010 Sales Qty 2011 Sales 2012 Qty Sales Qty Sales CSPQ 15 $24,300 58 $132,800 76 $153,650 66 $127,475 Encan d'auto Impact 229 $294,050 67 $72,450 72 $93,725 9 $4,265 Les Encans Ritchie Bros. 69 $345,950 22 $78,450 11 $28,540 1 $1,300 Call for tenders 3 $293,500 - - 2 $3,840 10 $11,850 Private sales 1 $3,360 1 $1,960 - $ - 1 $5,500 Donations 7 - 6 - 4 $ - - - $279,755 87 $150,390 Subtotal: Gross sales 324 Auctioneers' commissions Other expenses Towing charges $961,160 154 $285,660 165 $87,967 $36,749 $32,766 $32,766 $9,235 $2,004 $1,240 $1,240 $24,238 $16,028 $16,665 $16,665 Subtotal: Expenses $121,440 $54,781 $50,671 $50,671 Net revenues $839,720 $230,879 $229,084 $99,719 Auditor General of the Ville de Montr?al 92 2012 Annual Report 5.2. Asset Disposal The DGPV also arranges private sales, calls for tenders and donations. Pursuant to the powers delegated to it, the DGPV does not prepare a decision-making summary in any of these instances, except when donations are involved. 9 In the case of donations, this delegation of authority is supposed to include the production of a delegated decisionmaking summary, as indicated in the decision-making record management system (delegated decisions). 3.2.1.B. Recommendation We recommend that the Division de la gestion du parc de v?hicules of the Direction du mat?riel roulant et des ateliers municipaux produce a delegated decision-making summary every time it exercises its authority under the delegation of authority bylaw, as specified in the decision-making record management system. Business unit's response: [TRANSLATION] The DMRA will produce a decision-making summary semi-annually, describing road vehicles and other equipment that have been sold via auction. (Planned completion: July 2013) 3.2.1.1. Administrative Framework 3.2.1.1.A. Background and Findings A pre-merger framework had been in place under the former structure, but it has not been updated since 2006, when the new structure was introduced. Consequently, there are no administrative frameworks in place. Moreover, there is no directive supported by an established procedure to follow when disposing of vehicles. The division head for this administrative unit nevertheless refers to the disposal policies of another municipality for guidance. According to the information we obtained, the city is currently without a vehicle disposal policy. Vehicles are decommissioned based on their age and projected life expectancy. The DGPV analyzes requests made by the respective departments to determine the reasons for the request (e.g., cost, vehicle condition) and prepares a report for approval by the head of the DGPV. 9 Generally speaking, vehicles are donated to auto mechanic schools. Auditor General of the Ville de Montr?al 93 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.2.1.1.B. Recommendation We recommend that the Direction du mat?riel roulant et des ateliers municipaux of the Service de la concertation des arrondissements et des ressources mat?rielles adopt an administrative framework for the central departments concerning the disposal of vehicles, which clearly establishes the scope, principles, roles, responsibilities, processes and documentation required to ensure vehicle disposal activities are carried out in compliance with all legislation and city by-laws in this regard. Business unit's response: [TRANSLATION] The DMRA will produce an administrative framework (DIRECTIVE) regarding the disposal of the central departments' road vehicles and other equipment. This framework will be submitted to the city manager for future reference. (Planned completion: May 2013) 3.2.2. Role and Responsibilities of the Audited Business Units As stated earlier, the DMRA offers a road vehicle disposal service, which is mandatory for the central departments (including the SPVM and the Service des infrastructures, du transport et de l'environnement). The same process therefore applies to all these departments. As concerns the boroughs, they oversee the vehicle disposal process themselves. They have the prerogative of whether to delegate all or part of this process to the DGPV. We therefore reviewed the processes and the supporting documentation in the boroughs of Ville-Marie and Saint-Laurent. 3.2.2.1. Ville-Marie Borough 3.2.2.1.A. Background and Findings An annual vehicle decommissioning plan is drawn up based on criteria such as vehicle age, use, maintenance cost, residual value and fuel consumption. A decision-making summary is produced prior to any disposal, indicating which vehicles are due to be decommissioned. Vehicle sales are entrusted to the CSPQ. The recommendation is then ratified by a borough council resolution. The actual disposal process is overseen by the borough's Division de la voirie (Direction des travaux publics) in conjunction with the DMRA. The respective responsibilities are divided up as follows: o the list of vehicles and rolling stock slated for disposal is prepared by the division head; Auditor General of the Ville de Montr?al 94 2012 Annual Report 5.2. Asset Disposal o o o the DGPV validates the list, removes the identified vehicles from the inventory and notifies the Soci?t? de l'assurance automobile du Qu?bec that the vehicles are in storage so the licence status can be updated accordingly; the vehicles are stored in a location designated by the borough until they are transferred to the auctioneer (CSPQ) for sale; the CSPQ sends sales reports to the borough. Some of the parts of these stored vehicles can still be reused. Based on the information we obtained, the warehouse facilities do not appear to be terribly well protected against equipment or component theft. Any employee can gain admittance to the facilities. 3.2.2.1.B. Recommendation We recommend that the Direction des travaux publics of Ville-Marie borough control and limit access to areas where vehicles and other equipment earmarked for disposal are being stored to prevent the possibility of equipment or component theft. Business unit's response: [TRANSLATION] The Direction des travaux publics of Ville-Marie borough has amended its practices and now uses a warehouse located on a lot on rue Mill, which is protected by a locked security fence. The building itself is also locked, and the Direction des travaux publics will use it from now on to store vehicles prior to auction. In addition, we plan to gradually increase the frequency of vehicle transfers to various auctions. Lastly, this problem was raised with the architects responsible for designing the upcoming facilities for the roadwork crews at the rue Bercy site to ensure a secure parking area would be developed for vehicles and equipment slated for disposal. (Completed, March 2013) 3.2.2.2. Saint-Laurent Borough 3.2.2.2.A. Background and Findings Saint-Laurent borough administers its own vehicle disposal process. It has an administrative framework in place on asset disposal, which takes into account a number of assessment criteria with regard to vehicles (useful life, number of years of service, odometer count and maintenance/repair costs). On this basis, the manager makes a Auditor General of the Ville de Montr?al 95 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit decision and submits it to the borough council for approval, following which the vehicles are decommissioned. Auctions are held every two years. The borough council authorizes them after reviewing the list of assets due for disposal and selecting an auctioneering firm. The entire process is covered in a decision-making summary. The borough deals with an auctioneer who handles all the aspects of the sale, in exchange for administrative fees. The company then produces an official report with the auction results. The proceeds from the disposal are recorded in a Saint-Laurent borough revenue account. The asset that has been disposed of is then removed from the respective inventories. Our audit did not reveal any discrepancies in the vehicle disposal process in Saint-Laurent borough. Auditor General of the Ville de Montr?al 96 2012 Annual Report Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 Infrastructure Work Timeframes 5.3 5.3. Infrastructure Work Timeframes Table of Contents 1. Introduction ................................................................................................................... 101 2. Audit Scope................................................................................................................... 102 3. Findings and Recommendations .................................................................................. 102 3.1. Projects Completed Within the Initial Timeframe .................................................. 103 3.2. Projects Not Completed Within the Initial Timeframe............................................ 105 4. Appendix ....................................................................................................................... 125 4.1. List of the 17 Projects Covered by Our Audit ........................................................ 125 Auditor General of the Ville de Montr?al 99 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit List of Acronyms CSEM Commission des services ?lectriques de Montr?al SITE Service des infrastructures, du transport et de l'environnement Auditor General of the Ville de Montr?al TCEP 100 three-year capital expenditures program 2012 Annual Report 5.3. Infrastructure Work Timeframes 5.3. Infrastructure Work Timeframes 1. Introduction To achieve municipal infrastructure improvement objectives that require major investments, the central departments and boroughs of the Ville de Montr?al (the city) plan collective infrastructure projects to determine and prioritize renewal work to improve the quality of the city's infrastructure and extend its useful life. On a yearly basis, the city adopts a three-year capital expenditures program (TCEP) covering the infrastructure that is managed by its central departments, such as the Service de l'eau and the Service des infrastructures, du transport et de l'environnement (SITE), as well as the boroughs' own road infrastructure (roads, sidewalks) and water and sewer system rehabilitation programs. The city hires contractors to carry out the construction work planned under the TCEP. In the case of work estimated at over $100,000, the business units are required to issue a public call for tenders, in accordance with the Cities and Towns Act. The major steps involved in a public call for tenders process consist of preparing the call for tenders, calling upon the market, analyzing bids received, awarding contracts and managing them thereafter. Infrastructure work contracts are awarded to the lowest compliant bidder, and the bid price is the determining factor. The documents making up the call for tenders 1 are used to document the project's technical aspects, and they enable contractors to prepare their bid within the legal, administrative and operational frameworks that govern project completion. The presentation of a bid is equivalent to the bidder certifying that he obtained satisfactory knowledge of the documents, understood the documents and prepared his bid after having carefully evaluated the conditions that will apply to the work carried out. The bidder also certifies that he read and understood the supplementary clauses and understood the specifics regarding delays and applicable penalties. 1 The call for tender documents are accompanied by bidder instructions and appendices that form integral parts of the call for tenders, i.e., the notice to bidders, tender forms, price schedule, special instructions, standardized instructions (which include the general contract provisions) and various technical fascicles as well as plans. Auditor General of the Ville de Montr?al 101 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit The administrative requirements are described in the call for tender documents in accordance with the city's needs. The general (2009 version) and supplementary contract provisions are integral parts of the call for tender documents that the contractor is required to sign. These administrative provisions enable both the city's project manager and the contractor to whom a contract was awarded to efficiently complete the project and stipulate the timeframes, costs, change authorizations, penalties for lateness and non-compliance as well as acceptance of work. 2. Audit Scope The objective of this audit was to verify that infrastructure work contracts signed with contractors stipulated completion timeframes, that the stipulated timeframes were met and that measures were taken when contracts were not completed on time. Our audit covers infrastructure work carried out during the 2011 and 2012 fiscal years. To this effect, we examined 17 projects (see Appendix 4.1) carried out by the following business units: o Ville-Marie borough (2 projects); o Villeray-Saint-Michel-Parc-Extension borough (2 projects); o Saint-Laurent borough (2 projects); o Rivi?re-des-Prairies-Pointe-aux-Trembles borough (2 projects); o Service de l'eau: - Direction de l'eau potable (3 projects), - Direction de la gestion strat?gique des r?seaux d'eau (3 projects); o SITE: - Direction des infrastructures (3 projects). 3. Findings and Recommendations A book of specifications is a contract document that describes the project owner's expectations with respect to the contractor. It therefore consists of a document that describes--in the most specific and simple terms possible--the needs, requirements and obligations that the contractor is required to meet. The book of specifications sets out the general and supplementary contract provisions. Auditor General of the Ville de Montr?al 102 2012 Annual Report 5.3. Infrastructure Work Timeframes The general provisions are the contract's general administrative and technical conditions. The administrative conditions set out the administrative provisions that apply to a category of goods and services (e.g., the work schedule, the work start date, penalties for delays) whereas the technical conditions set out the technical provisions that apply to the performance of work of identical nature (e.g., the completion of rehabilitation work or lining work on the city's territory). The supplementary provisions consist of special administrative and technical conditions. The administrative conditions set out each contract's administrative provisions whereas the technical conditions set out the technical provisions applying to the completion of all work deliverable under a contract. The call for tender documents are comprised of bid forms, plans and specifications, addendums and bidder instructions that set out all of the conditions that bids must meet. These documents also include the general and supplementary contract provisions. Awarded contracts are made up of the call for tender documents and the winning bid. During our audit of the 17 selected projects, we examined the clauses stipulated in the call for tender documents in order to gather information on the work completion timeframes and applicable administrative clauses. The work completion timeframes stipulated in the documents make up the essence of the contract, and the contractor may be fined a penalty by the manager 2 simply for failing to complete the work on time. 3.1. Projects Completed Within the Initial Timeframe 3.1.A. Background and Findings The conditions described in the call for tender documents reflect the needs, requirements and obligations that contractors must meet when preparing their bid and completing the work. The business units are responsible for following up on contracts to ensure that these conditions are met and that the information obtained is compliant. Also, corrective action must be imposed when these conditions are not met. 2 The term "manager" refers to the person in charge who was appointed by the business unit manager and usually acts as the project manager. Auditor General of the Ville de Montr?al 103 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit The work completion timelines provided in the documents making up the winning bid become the timelines of the contract that the contractor agrees to meet. Thereafter, the manager may impose a penalty on the contractor simply for failing to meet these timelines. The following documents are used to determine the number of days stipulated in the contract as well as the number of days actually taken to complete the work up to the provisional acceptance stage: o the comprehensive schedule provided by the contractor and accepted by the project manager or his representative (this document becomes the contract schedule on the basis of which the start and completion of work are taken into consideration to impose penalties for failing to complete the work on time); o the project manager's letter establishing the official start date for the work; o the letters specifying when work will be interrupted and when it will resume should completion timelines overlap a winter period (from December to April); o the letter confirming provisional acceptance of the work that the project manager sends to the contractor. The work start and completion dates are based respectively on the work start order and the provisional acceptance of the work following a site inspection. In cases where it is not possible to determine an official work completion date from the provisional acceptance, this date is established on the basis of the work progress report (work completion schedule), the project engineer's report or the project manager's weekly report. To this effect, we examined all of the documents enabling us to retrace the start and end dates for the work specified in the contract, the follow-up of work completion and any indication as to delays in completion. Where the contractor failed to meet the contract timelines, we also examined whether or not penalties were imposed. A site start-up meeting is held once the contract has been awarded. The purpose of this meeting is to review the contract stipulations, timelines and clauses setting the penalties for delays. Thereafter, the city or the boroughs issue the order to start the work. Concurrently with our analyses, we interviewed the managers responsible for these projects in order to have them validate the information that we had collected. Where infrastructure work projects had not been completed according to the schedule, as specified in the decision-making summary or the comprehensive schedule provided by the contractor in his bid or as provided during the start-up meeting, we asked for an explanation of the delays and the reasons justifying whether penalties were imposed or not. Auditor General of the Ville de Montr?al 104 2012 Annual Report 5.3. Infrastructure Work Timeframes In the case of eight of the 17 projects that we chose to audit, the work was completed within the initial timeframe, as stipulated in the infrastructure contracts signed with the contractors. Consequently, the city did not impose any penalties to the contractors who had worked on these projects. The latter are presented in Table 1 below. Table 1 - Projects Completed Within the Initial Timeframe Start and end dates Timeframe Project no. Contract award Initially planned a Comments b Actual Villeray-Saint-Michel-Parc-Extension borough PRR-11-03 $671,823.45 July 25 to August 18, 2011 25 calendar days July 25 to August 15, 2011 22 calendar days PRR-12-05 $1,051,369.75 September 19 to November 2, 2012 45 calendar days C12-036 $3,503,621.14 Fall 2012 40 working days per phase, i.e., 120 working days for the 3 phases ING12-05 $442,906.70 July 23 to September 30, 2012 70 calendar days Completed on time September 19 to October 24, 2012 Completed on time 36 calendar days Saint-Laurent borough September 5 to November 26, 2012 Completed on time 59 working days for the 3 phases Rivi?re-des-Prairies-Pointe-aux-Trembles borough July 23 to September 14, 2012 54 calendar days Completed on time Service de l'eau - Direction de l'eau potable 9953 (A-323-1) $1,744,928.98 Completed on time August 8 to December 5, 2011 No penalty 120 calendar days One year late because Work completed within the contract the first contractor timelines by the new contractor declared bankruptcy June 25 to December 21, 2010 155 calendar days Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau 9991 $11,935,721.68 May 1 to November 30, 2011 214 calendar days April 25 to November 17, 2011 206 calendar days Completed on time 10033 East: $13,062,136.26 West: $11,818,424.01 April 23 to November 16, 2012 208 calendar days April 18 to November 16, 2012 c 213 calendar days Completed on time October 28 to December 16, 2011 May 14 to November 30, 2012 251 calendar days Completed on time SITE 1180 $4,912,027.84 a b c September 2011 to December 2012 270 calendar days The scheduled work start and completion dates were provided in the infrastructure contracts signed with the contractors. As for the timeframe, it is specified in the call for tender documents. The actual work start and completion dates are established respectively from the order to start the work and the provisional acceptance of the work following a site inspection. The actual timeframe is determined from this information. An order to start the work was issued on April 18, 2012. 3.2. Projects Not Completed Within the Initial Timeframe Although completion timeframes were stipulated in the infrastructure work contracts signed with the contractors and approved by the manager, our analysis indicates that eight projects were not completed within the initial timeframe and that one project will be completed in Auditor General of the Ville de Montr?al 105 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 2013. To this effect, we obtained the necessary documentary evidence indicating whether or not penalties had been imposed under the contract. 3.2.1. Ville-Marie Borough 3.2.1.A. Background and Findings Table 2 presents key information on the number of calendar days scheduled in the contract documents binding the contractor to the borough as well as the number of calendar days actually required to complete the work. To this effect, neither of the projects that we examined were completed within the initial timeframe and penalties were not imposed in both cases. Table 2 - Project Completion (Initially Planned and Actual) Ville-Marie Borough Start and end dates Timeframe Project no. Contract award Comments Initially planned Actual VMP-11-016 $1,038,247.88 October 25 to December 9, 2011 45 calendar days October 18 to December 19, 2011 April 19 to May 18, 2012 93 calendar days 48 calendar days late Not completed on time No penalty VMP-11-018 $1,488,556.00 October 1 to December 16, 2011 70 calendar days May 14 to July 26, 2012 August 6 to 13, 2012 82 calendar days 12 calendar days late Not completed on time No penalty With respect to these two projects, we attempted--in vain--to obtain from the Direction des travaux publics the minutes of the administrative meetings that were held after the start-up meeting in order to evaluate if the work had progressed as planned and if the timeframes had been met as well as to consult the recommended documents in the general contract provisions (letters issued to interrupt and resume the work, changes authorized by the city). According to the Direction des travaux publics, these difficulties in obtaining the documentary evidence are due to the fact that the work was supervised by a city employee who is currently on paternity leave. Furthermore, the engineer responsible for the project retired from the city in August 2012. We were surprised to learn that the contract documents supporting project completion and follow-up are not available in the project records. Auditor General of the Ville de Montr?al 106 2012 Annual Report 5.3. Infrastructure Work Timeframes Project VMP-11-016 - Reconstruction of a Sewer and a Secondary Water Conduit ($1,038,247.88) The tender form dated July 27, 2011 stipulates that the work must be completed within 45 calendar days after the manager has authorized the start of the work in writing. On September 27, 2011, the city council authorized an expense of $1,038,247.88 (including taxes as well as contingency and incidental fees) to complete this project. The timeframe was initially set from October 25 to December 9, 2011. In accordance with clause 5.1.6 of the general contract provisions, the manager ordered the start of the work on October 18, 2011. On October 13, 2011, the contractor proposed a work schedule over a two-year period, i.e.: o In 2011: from October 18 to November 29 (43 calendar days); o In 2012: from May 21 to June 6 (17 calendar days). Under this schedule, the work would be completed in a total of 60 calendar days. Consequently, the timeframe proposed by the contractor did not comply with the required 45 calendar days under the contract. According to clause 5.1.5.2 of the general contract provisions, the contractor could not amend the work schedule without the manager's written authorization, specifically at the time of signing a modification instruction. However, we were unable to retrace any document authorizing the revised timeframe of 60 calendar days. Our analysis of the daily worksite reports confirmed that the work had started on October 18, 2011 and was interrupted on December 19, 2011 for the winter season (63 calendar days). Thereafter, infrastructure work resumed on April 19, 2012 and was completed on May 18, 2012 (30 calendar days). A total of 93 calendar days were required to complete the infrastructure work, i.e., 48 days more than stipulated in the contract (45 calendar days). Under clauses 5.1.8 ("Suspension of the work") and 5.1.6 ("Start of the work") of the general contract provisions, we were unable to retrace the order issued to interrupt the work on December 19, 2011 or the order issued on April 19, 2012 to resume the work. By virtue of clause 5.1.10 of the general contract provisions, establishing the applicable penalties for failing to complete the work on time, the manager should have imposed the applicable penalties given that the work had not been completed within the initial timeframe. However, he did not deem it appropriate to impose the applicable penalties. Auditor General of the Ville de Montr?al 107 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit The head of the Division des ?tudes techniques explains the project completion delays in this case as follows: o overtime hours were required to install a temporary aqueduct and the project was hindered by poor weather conditions; o masses that were not indicated in the call for tenders were found under the sidewalk. This project was completed within budget. Here is the situation with respect to the aforementioned project (VMP-11-016): o it was not completed within the contract's timeframe; o the borough manager supervising the completion of the work did not deem it appropriate to impose the applicable penalties and we were unable to obtain a formal document motivating his decision; o a certain number of formal documents provided under the general contract provisions (letters ordering the interruption and resumption of the work, changes ordered by the city) were not produced and are not included in the project's permanent records. Project VMP-11-018 - Reconstruction of a Combined Sewer and a Secondary Water Conduit ($1,488,556.00) The tender form dated August 12, 2011 stipulates that the work must be completed within 70 calendar days after the manager has authorized the commencement of the work in writing. On October 24, 2011, the city council authorized an expense of $1,488,556.00 (including taxes as well as contingency and incidental fees) to complete this project. The timeframe was initially set from October 1 to December 16, 2011. However, given that the awarding of the contract by the city council was delayed (the contract was finally awarded on October 24, 2011), the head of the borough's Division des ?tudes techniques pointed out to us that the work was postponed until 2012 because of the poor weather at that time of the year. However, we were unable to retrace any communication between the Ville-Marie borough and the contractor that officially postponed the work. In accordance with clause 5.1.6 of the general contract provisions, the manager ordered the start of the work on May 14, 2012. Despite the requirement for the contractor to file a work schedule before starting the work, we were unable to retrace any evidence to the effect that the contractor had prepared a Auditor General of the Ville de Montr?al 108 2012 Annual Report 5.3. Infrastructure Work Timeframes new work schedule. Consequently, we cannot validate the required timeframe of 70 calendar days provided in the contract. Furthermore, we were unable to obtain from the manager any evidence confirming that the contractor had provided a new schedule. Our analysis of the daily worksite reports confirmed that the work started on May 14, 2012 and was interrupted on July 26, 2012 for the construction holiday (74 calendar days). Thereafter, work resumed on August 6, 2012 and was completed on August 13, 2012 (8 calendar days). Accordingly, a total of 82 calendar days were required to complete the infrastructure work, i.e., 12 days more than stipulated in the contract (70 calendar days). The manager should have imposed the applicable penalties given that the work had not been completed within the initial timeframe. However, the head of the Division des ?tudes techniques explained that unscheduled work entrusted to the contractor required additional days to complete the project. The unscheduled work consisted of removing a larger quantity of contaminated soil than expected and carrying out additional deviation and hookup work. The laboratory analysis report indicates that the work was completed on August 13, 2012. At the time we produced our audit report, provisional acceptance of the work had not yet been issued. This project was completed within budget. The VMP-11-018 project took longer than expected to complete after the borough had approved requests for changes. 3.2.1.B. Recommendation We recommend that the Direction des travaux publics of the Ville-Marie borough instigate a documentation procedure and keep records of important information to support the decisions made and show compliance with the contract provisions. Business unit's response: [TRANSLATION] The Direction des travaux publics has already begun to restructure its procedures. A work group is currently developing a project classification system that reflects project know-how and tracking at all stages. Furthermore, a check list of important project items will be added to the new management procedure. Auditor General of the Ville de Montr?al 109 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Project managers are responsible for the classification of their projects. However, management will now appoint a member of the project team to be in charge of the file (another person will be appointed as an alternate) to ensure that project documents are properly managed and classified. A central classification system will be implemented to facilitate the archiving of all projects of the Division des ?tudes techniques. (Planned completion: July 2013) 3.2.1.C. Recommendation We recommend that the Direction des travaux publics of the Ville-Marie borough, given the lack of documentary evidence supporting the completion and follow-up of these projects, take the necessary measures to ensure that: o the manager approves the infrastructure work completion schedule prepared by the contractor; o the manager approves the issuance of the order to interrupt the work as well as the order to start the work, when required; o the project manager imposes the applicable penalties when the work is not completed as initially planned or in accordance with contract provisions; to ensure compliance with the administrative provisions and provide justification of the decisions made. Business unit's response: [TRANSLATION] The following improvements will be made to the project management procedure used by the Direction des travaux publics of the Ville-Marie borough: o The borough's project manager will ask the contractor or service provider to prepare a comprehensive work schedule including a Gantt chart mapping out the project. On a monthly basis, the contractor will be required to provide the project manager with an updated schedule. These new requirements will be added to the Ville-Marie borough's specifications; o Work start-up, interruption and provisional acceptance forms will be developed and provided to contractors or service providers to enable them to adequately document these project stages; o A follow-up of the calendar will be done and recorded during each worksite meeting to ensure the project is completed as per the contract timelines. Should a delay be justified, the contractor will be required to request an extension. This request will be officially prepared by the project manager using a modification instruction or a calendar extension form. (Planned completion: July 2013) Auditor General of the Ville de Montr?al 110 2012 Annual Report 5.3. Infrastructure Work Timeframes 3.2.1.D. Recommendation We also recommend that the Direction des travaux publics of the Ville-Marie borough: o obtain documentary evidence concerning projects that were not completed within the initial timeframe and for which no penalty was imposed in order to validate the reasons for the project manager's proposal; o periodically report to the borough director to inform him of the projects for which penalties were not imposed although they were subject to penalties. Business unit's response: [TRANSLATION] With the purpose of informing the borough director about late projects for which penalties were not imposed or for which penalties could be imposed, the Division des ?tudes techniques will provide the borough manager, three times a year, with a table--yet to be developed--as well as all existing project documents, including the decisions made and the reasons for these decisions. All of these documents will be approved by the division head. (Planned completion: July 2013) 3.2.2. Saint-Laurent Borough 3.2.2.A. Background and Findings Table 3 presents key information on the number of calendar days scheduled in the contract documents binding the contractor to the borough and on the number of calendar days actually required to complete the work. One of the two projects we audited was not completed as initially planned but no penalty was imposed. Table 3 - Project Completion (Initially Planned and Actual) Saint-Laurent Borough Start and end dates Timeframe Project no. Contract award Comments Initially planned C11-042 $3,157,579.49 Actual September to December 2011 May to June 2012 58 working days October 3 to November 29, 2011 April 10 to June 26, 2012 98 working days Auditor General of the Ville de Montr?al 111 40 working days late Not completed on time No penalty 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Project C11-042 - Reconstruction of Water and Sewer Systems ($3,157,579.49) The tender form dated July 25, 2011 stipulates that the work must be completed within 58 working days--which includes 8 days to apply the second layer of asphalt in 2012--after the manager has authorized the start of the work in writing. An expense of $3,157,579.49 (including taxes and incidental fees) was authorized to complete this project by the Saint-Laurent borough council on August 2, 2011 and by the city council on September 27, 2011 (the project was financed by both authorities). As indicated in the decision-making summary, the timeframe was initially set from September to December 2011 and from May to June 2012. However, given that the city council awarded the contract at the end of September, borough management agreed during the start-up meeting to have the contractor work fewer days in 2011 and more days in 2012. The work nevertheless had to be completed within 58 working days. Our analysis of the tracking documents (purchase orders issued to the contractors, timelines filed on different dates, minutes of worksite meetings, partial provisional acceptance of the work completed) confirmed that work had started on October 3, 2011 and was interrupted on November 29, 2011 (42 working days). Thereafter, work resumed on April 10, 2012 and was completed on June 26, 2012 (56 calendar days). Consequently, a total of 98 calendar days were needed to complete the infrastructure work, i.e., 40 more than stipulated in the contract (58 contract working days). The manager should have imposed the applicable penalties given that the work had not been completed within the initial timeframe. However, he did not deem it appropriate to do so for the following reasons supported by documentary evidence: o the work started only in October. As a result, the work was carried out in winter conditions and the contractor was obliged to modify his work methods. The contractor had planned that two teams would work on the three streets concurrently. However, because of the winter conditions and the presence of gas lines on rue Cardinal, the Saint-Laurent borough decided to delay the work to avoid keeping only one street open during the winter, which would have inconvenienced citizens (delay of 27 days); o the excavation of bedrock on a main road delayed the completion of the work by approximately nine days; o it rained for four days during which no work could be completed on the site. Auditor General of the Ville de Montr?al 112 2012 Annual Report 5.3. Infrastructure Work Timeframes The work was completed on June 26, 2012. At the time we produced our audit report, the provisional acceptance of the work had not yet been issued. This project was completed within budget. 3.2.2.B. Recommendation We recommend that the Direction des travaux publics of the Saint-Laurent borough: o obtain documentary evidence concerning projects that were not completed within the initial timeframe and for which no penalty was imposed in order to validate the reasons for the project manager's proposal; o periodically report to the borough director to inform her of the projects for which penalties were not imposed although they were subject to penalties. Business unit's response: [TRANSLATION] Projects that were not completed on schedule are normally identified during worksite meetings, usually held every two weeks. All delays are documented and explained in the worksite report as are the suggested corrective action and penalties imposed. Controls are implemented to ensure that the public works manager is informed of the delay, reason for the delay, corrective action and penalty. (Completed, March 2013) A quarterly report must be provided to the borough director to inform him of projects that are running late. (Planned completion: June 2013) 3.2.3. Rivi?re-des-Prairies-Pointe-aux-Trembles Borough 3.2.3.A. Background and Findings Table 4 presents key information on the number of calendar days scheduled in the contract documents binding the contractor to the borough and on the number of calendar days actually required to complete the work. One of the two audited projects was not completed as initially planned but no penalty was imposed. Auditor General of the Ville de Montr?al 113 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Table 4 - Project Completion (Initially Planned and Actual) Rivi?re-des-Prairies-Pointe-aux-Trembles Borough Start and end dates Timeframe Project no. Contract award Comments Initially planned ING11-07 $629,727.84 Actual October 17 to November 30, 2011 45 calendar days October 17 to December 9, 2011 54 calendar days 9 calendar days late Not completed on time No penalty Project ING11-07 - Reconstruction of a Combined Sewer and a Secondary Water System ($629,727.84) The call for tenders stipulates that the work must be completed within 45 calendar days once the manager has ordered the start of the work. On September 27, 2011, the city council authorized an expense of $629,727.84 (including taxes and contingency fees) to complete this project. The timeframe was initially set from October 17 to November 30, 2011. In accordance with clause 4.1.6 of the borough's general administrative provisions, the manager ordered the start of the work on October 17, 2011. On October 6, 2011, the contractor provided a work schedule specifying that work would be completed from October 17 to December 2, 2011, for a total of 47 calendar days (the two additional days corresponding to Thursday and Friday). Our analysis of the daily worksite reports confirmed that the work started on October 17, 2011 and was completed on December 9, 2011 (54 calendar days), i.e., nine days more than the contract stipulated (45 contract calendar days). The project manager should have imposed the applicable penalties given that the work was not completed within the initial timeframe. However, he did not deem it appropriate to do so and specified that the initial project design had been modified to accommodate another related project. Furthermore, we counted a total of 20 requests for changes that were authorized. The provisional work acceptance certificate was issued on February 1, 2012. This project was completed within budget. Auditor General of the Ville de Montr?al 114 2012 Annual Report 5.3. Infrastructure Work Timeframes 3.2.3.B. Recommendation We recommend that the Direction des travaux publics of the Rivi?re-des-Prairies- Pointe-aux-Trembles borough: o obtain documentary evidence concerning projects that were not completed within the initial timeframe and for which no penalty was imposed in order to validate the reasons for the project manager's proposal; o periodically report to the borough director to inform him of the projects for which penalties were not imposed although they were subject to penalties. Business unit's response: [TRANSLATION] Formalize the criteria used to determine whether or not the penalty clauses should be applied in the case of contracts that are not completed as initially planned: o have these criteria validated by Legal Services to ensure their compatibility with the legal management of contracts that are behind schedule; o have these criteria approved by management (the Direction des travaux publics and the borough management). (Planned completion: June 2013) Implement a formal tracking and reporting process for contracts that are not completed as initially planned: o require the contractor to provide a comprehensive work schedule including all critical milestone dates. This schedule will be used in reference to the contract's timelines and make it easier to track differences; o formally document and quantify all delays as the contract is completed that could result in the work not being completed according to schedule; o determine whether the city or the contractor is responsible for the delays and explain why; o formally notify the contractor during the completion of the contract of the delays for which he is deemed responsible and inform him that these delays could cause the city to impose penalties as provided under the contract; o following a joint analysis by the project manager and manager, render a decision on whether or not to impose penalties in accordance with the criteria approved by management. The project manager motivates the decision, which must then be approved by the manager for each contract behind schedule for which no penalty was imposed; o provide the public works manager and the borough director with a summary of all contracts behind schedule and for which the penalty clauses were not applied as well as related documentary evidence. This summary will be provided on a quarterly basis. (Planned completion: September 2013) Auditor General of the Ville de Montr?al 115 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.2.4. Service de l'eau 3.2.4.1. Direction de l'eau potable 3.2.4.1.A. Background and Findings Table 5 presents key information on the number of calendar days scheduled in the contract documents binding the contractor to the city and on the number of calendar days actually required to complete the work. Two of the three audited projects were not completed as initially planned and a penalty was imposed in the case of one project. Table 5 - Project Completion (Initially Planned and Actual) Direction de l'eau potable Start and end dates Timeframe Project no. Contract award Comments Initially planned Actual 9925 (A-327-3) $5,286,906.98 October 29, 2009 to November 19, 2010 240 calendar days May 3 to December 13, 2010 July 4 to September 14, 2011 298 calendar days 9954 (A-329) $2,062,966.99 August 12 to November 20, 2010 100 calendar days 58 calendar days late Not completed on time No penalty 27 calendar days late October 25, 2010 to March 25, 2011 Not completed on time 127 calendar days Penalty imposed Project 9925 (A-327-3) - Construction of Main and Secondary Water Conduits and Various Work on 600-mm Conduits ($5,286,906.98) The tender form dated August 3, 2009 stipulates that the work must be completed within 240 calendar days after the manager has authorized the start of the work in writing. On September 21, 2009, the city council authorized an expense of $5,286,908.98 (including taxes) to complete this project. The schedule included in the decision-making summary indicates that the work would be completed over a two-year period (from 2009 to 2010) and that work would be interrupted during the winter season. The initial project schedule, once approved, covered the period between October 20, 2009 and November 19, 2010, for a total of 240 calendar days. The work was subdivided into five phases. The contractor was informed in writing that the start of the work had been ordered for May 3, 2010. The letter also reiterated that 240 calendar days were scheduled to complete the work (from May 3, 2010 to December 29, 2010). Auditor General of the Ville de Montr?al 116 2012 Annual Report 5.3. Infrastructure Work Timeframes The minutes of the project start-up administrative meetings which were held on March 25, 2010 and April 21, 2010 indicate that the city and the contractor reached an agreement on a schedule and the date of May 3, 2010 for the start of the work. The contractor carried out the work in accordance with the city's requirements and the five phases stipulated in the specifications, i.e.: o Completion of phases 1 to 4 in 2010: Work started on May 3, 2010 and was completed on December 13, 2010, for a total of 225 calendar days; o Completion of phase 5 in 2011: Work resumed on July 4, 2011 and was completed on September 14, 2011, for a total of 73 calendar days. The contractor required a total of 298 calendar days to complete the project, i.e., 58 calendar days more than the initial contract schedule provided in the specifications (240 calendar days). On July 8, 2011, the contractor requested an extension of the contract delay to complete additional work. The Direction de l'eau potable indicated to us that this extension resulted in additional work days which had not been scheduled in the initial contract, because of the following situations: o Blocks of concrete belonging to Bell and the Commission des services ?lectriques de Montr?al (CSEM) were discovered near the site planned for a new valve house. Consequently, the planned site was modified and moved down the road to enable the construction of the valve house. This modification also resulted in changes being made to the valve house's water outlet lines; o Shop drawings needed to be drafted and water line parts needed to be fabricated; o A new request for authorization was filed with the Minist?re de la Culture, des Communications et de la Condition f?minine to modify the electrical panels in accordance with the CSEM regulations, initially unknown. The Direction de l'eau potable admits the overruns but also confirms that they are in no way imputable to the contractor. Moreover, no penalty was imposed. Provisional acceptance was issued on November 2, 2011, and this project was completed within budget. Auditor General of the Ville de Montr?al 117 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Project 9954 (A-329) - Construction of a Main Water Conduit ($2,062,966.99) The tender form dated April 21, 2010 stipulates that the work must be completed within 100 calendar days after the manager has authorized the start of the work. On June 14, 2010, the city council authorized an expense of $2,062,966.99 (including taxes and incidental fees) to complete this project. The schedule included in the decision-making summary indicates that the work was to be completed between August 12 and November 20, 2010. However, prior approval of the Minist?re du D?veloppement durable, de l'Environnement et des Parcs was required before work could start and the Direction de l'eau potable needed to obtain this approval by September 1, 2010. Consequently, the manager's order to start work was delayed. During the start-up meeting held on September 9, 2010, the contractor presented a preliminary work schedule based on a work start date of September 20, 2010 and a work completion date of December 16, 2010. These dates implied that the work could be completed within 88 days whereas the contract stipulated a timeframe of 100 calendar days. The contractor was later notified by letter that the start of the work had been set for October 25, 2010. The work started on October 25, 2010 and was interrupted for 24 days during the holiday period. The contractor revised the schedule several times and filed requests for additional work up to the completion of the work, on March 25, 2011. Under the circumstances, the work was completed 27 calendar days late. The city agreed that 18 of these 27 calendar days were imputable to the requests for additional work that it had approved and that the remaining nine days were imputable to the contractor. These nine days should have resulted in a penalty being levied against the contractor for delays in completing the work. However, no penalty was imposed seeing as the contractor did not bill for the additional work. On September 26, 2011, the city issued the provisional acceptance of the work, completed since March 25, 2011. This project was completed within budget. Auditor General of the Ville de Montr?al 118 2012 Annual Report 5.3. Infrastructure Work Timeframes 3.2.4.1.B. Recommendation We recommend that the head of the Division des projets majeurs of the Direction de l'eau potable: o obtain documentary evidence concerning projects that were not completed within the initial timeframe and for which no penalty was imposed in order to validate the reasons for the project manager's proposal; o periodically report to the manager of the Direction de l'eau potable to inform him of the projects for which penalties were not imposed although they were subject to penalties. Business unit's response: [TRANSLATION] Implement an instruction requiring project managers to have all orientations concerning project contract schedule overruns approved. (Planned completion: May 2013) Include, in a monthly report, the actual number of days required to complete the work in comparison to the duration of each contract as well as documentary evidence of the differences. (Planned completion: June 2013) 3.2.4.2. Direction de la gestion strat?gique des r?seaux d'eau 3.2.4.2.A. Background and Findings Table 6 presents key information on the number of calendar days scheduled in the contract documents binding the contractor to the city and on the number of calendar days actually required to complete the work. One of the three audited projects was not completed as initially scheduled, but no penalty was imposed. Table 6 - Project Completion (Initially Planned and Actual) Direction de la gestion strat?gique des r?seaux d'eau Project no. Contract award Start and end dates Timeframe Initially planned 9979 $12,049,926.67 April 1 to November 30, 2011 244 calendar days Auditor General of the Ville de Montr?al Comments Actual April 18 to December 9, 2011 15 calendar days January 30 to February 21, 2012 Not completed on time 259 calendar days No penalty 119 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Project 9979 - Installation of Linings to Rehabilitate Sewers on Various Streets of the Ville de Montr?al ($12,049,926.67) According to the information contained in the call for tender documents, the initial project completion schedule covered the period between April 1 and November 30, 2011. However, the city reserved the right to delay a portion of the work at its discretion at no additional cost. The project's timeframe was not provided in terms of calendar days, so we therefore deducted that the work had to be completed within 244 calendar days. On March 21, 2011, the city council authorized an expense of $12,049,926.67 (including taxes and incidental fees) to complete this project. In accordance with clause 5.1.6 of the general administrative provisions, the contractor was notified by letter that the work was to start on April 18, 2011. This letter reiterated that, in accordance with clause 3.0 of the supplementary administrative provisions, the work was to be completed by November 30, 2011 at the latest. During the September 16, 2011 administrative meeting, the Direction de strat?gique des r?seaux d'eau expressed concern about the completion of all of planned under the contract by November 30, 2011. On October 26, 2011, manager of the Service de l'eau authorized the contractor to continue the December 16, 2011. la gestion the streets the senior work until On January 23, 2012, the senior manager of the Service de l'eau authorized the contractor to continue the work from January 30 to March 30, 2012. The work was completed on February 21. However, a part of the work was cancelled because work other than lining sewers was being completed by other contractors at the same time and on the same street (e.g., sidewalks). On October 31, 2012, the city issued the provisional acceptance of the work. Under the contract, the work was to be completed within about 244 calendar days. However, the work actually took 259 days (15 days more than scheduled) to complete. The Direction de la gestion strat?gique des r?seaux d'eau approved an extension of the work period to make up for the delay. No penalty was imposed seeing as the additional work completed by the contractor had been approved by the city. This decision is supported by documentary evidence. The project was completed within budget. Auditor General of the Ville de Montr?al 120 2012 Annual Report 5.3. Infrastructure Work Timeframes 3.2.4.2.B. Recommendation We recommend that the Division de la gestion strat?gique des r?seaux d'eau - Unit? Nord of the Direction de la gestion strat?gique des r?seaux d'eau: o obtain documentary evidence concerning projects that were not completed within the initial timeframe and for which no penalty was imposed in order to validate the reasons for the project manager's proposal; o periodically report to the manager of the Direction de la gestion strat?gique des r?seaux d'eau to inform her of the projects for which penalties were not imposed although they were subject to penalties. Business unit's response: [TRANSLATION] The Direction de la gestion strat?gique des r?seaux d'eau will make sure it obtains documentary evidence concerning the projects that were not completed on time and the head of the Division de la gestion strat?gique des r?seaux d'eau - Unit? Nord will periodically report to the manager of the Direction de la gestion strat?gique des r?seaux d'eau to inform her of projects for which penalties were not imposed although they were subject to penalties. (Completed, April 2013) 3.2.5. Service des infrastructures, du transport et de l'environnement 3.2.5.A. Background and Findings Table 7 presents key information on the number of calendar days scheduled in the contract documents binding the contractor to the city and on the number of calendar days actually required to complete the work. One of the three audited projects was not completed as initially scheduled and a penalty may be levied against the contractor depending on the outcome of the investigation currently under way. One project is under way and will be completed in 2013. Auditor General of the Ville de Montr?al 121 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Table 7 - Project Completion (Initially Planned and Actual) SITE Start and end dates Timeframe Project no. Contract award Comments Initially planned Actual 1109 $2,472,000.49 March 9, 2011 to March 9, 2012 90 calendar days October 11 to December 17, 2011 April 30 to October 1, 2012 223 calendar days 162301 $1,198,325.60 September to November 2012 90 calendar days September 10 to October 7, 2012 28 calendar days in 2012 Work resumed in the spring of 2013 133 calendar days late Not Completed on time Possible penalty On schedule to date Work will be completed in 2013 Project 1109 - Reconstruction of a Combined Sewer and a Secondary Water Conduit ($2,472,000.49) The tender form stipulates that the work must be completed within 90 calendar days after the manager has authorized the start of the work. It provides that the work may begin in the spring of 2011. On April 14, 2011, the city council authorized an expense of $2,472,000.49 (including taxes and incidental fees) to complete this project. The schedule included in the decision-making summary indicates that the work was to be completed in 2011. However, the decision-making summary also includes an initial project completion schedule covering the period between March 9, 2011 and March 9, 2012. A start-up meeting was held on October 3, 2011. During this meeting, the contractor was reminded of the work completion timeframe of 90 consecutive calendar days. The contractor was sent a letter notifying him that the work was scheduled to start on October 11, 2011. An order interrupting the work was issued on December 17, 2011 during a worksite meeting. However, the contractor was required to return to the worksite during the winter to repair temporary paving that had been applied in poor weather conditions beyond his control. The work to reconstruct the sewer and drinking water conduits resumed on April 30, 2012 following a communication from the engineer responsible for the project. The work continued through the summer and was completed on October 1, 2012. Auditor General of the Ville de Montr?al 122 2012 Annual Report 5.3. Infrastructure Work Timeframes The SITE issued the provisional acceptance of the work on October 1, 2012. The work was completed in 223 calendar days, i.e., 133 days more than the 90 calendar days initially planned. The head of the Division de r?alisation des travaux informed us that the case is currently being investigated and that the contractor was asked to explain the delays in completing the work. This project was completed within budget. Project 162301 - Reconstruction of Sidewalks on Various Streets of the Ville de Montr?al ($1,198,325.60) The tender form stipulates that the work must be completed within 90 calendar days during the fall of 2012. However, the contractor is informed that the work could be completed in part or in whole in the spring of 2013. On August 21, 2012, the city council authorized an expense of $1,198,325.60 (including taxes and incidental fees) to complete this project. The initial project schedule set out that the work would be completed from September to November 2012. The management of the SITE ordered the start of the work on September 10, 2012. The minutes of the August 29, 2012 start-up meeting indicate that the Division de la r?alisation des travaux reminded the contractor that he had a maximum of 60 calendar days to complete all of the work. Accordingly, all of the work had to be completed by November 9, 2012 at the latest. A difference of 30 calendar days was noted between the timeframe provided in the tender form (90 days) binding the contractor to the city and the minutes of the August 29, 2012 project start-up meeting. Management informed us that it was unable to explain this difference. Management of the SITE ordered the contractor to interrupt the work on October 7, 2012 because of the weather conditions that made it impossible reconstruct the sidewalks. The work will be completed in 2013. We have noted that one of the planned project phases has not yet started. Auditor General of the Ville de Montr?al 123 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.2.5.B. Recommendation We recommend that the Division de la r?alisation des travaux of the Direction des infrastructures: o obtain documentary evidence concerning projects that were not completed within the initial timeframe and for which no penalty was imposed in order to validate the reasons for the project manager's proposal; o periodically report to the manager of the Direction des infrastructures to inform her of the projects for which penalties were not imposed although they were subject to penalties. Business unit's response: [TRANSLATION] In cases where additional delays are granted to contractors, the documentary evidence is systematically attached to the final count in the work completion form that summarizes the actual timeline. (Completed, March 2013) Have work completion approved by the manager of the Direction des infrastructures in the case of contracts for which penalties were not imposed although they were subject to penalties. (Planned completion: May 2013) Auditor General of the Ville de Montr?al 124 2012 Annual Report 5.3. Infrastructure Work Timeframes 4. Appendix 4.1. List of the 17 Projects Covered by Our Audit Table A - Scope and Value of the Audited Contracts Project no. Scope of work Contract value Ville-Marie borough VMP-11-016 Reconstruction of a sewer and a secondary water conduit $1,038,247.88 VMP-11-018 Reconstruction of a combined sewer and a secondary water conduit $1,488,556.00 Villeray-Saint-Michel-Parc-Extension borough PRR-11-03 Reconstruction of sidewalks along with geometric revision work at various locations in the borough $671,823.45 PRR-12-05 Leveling and paving of the roadway and reconstruction of sidewalks and borders $1,051,369.75 Saint-Laurent borough C11-042 Reconstruction of water and sewer systems $3,157,579.49 C12-036 Pavement rehabilitation work - 2012 program $3,503,621.14 ING11-07 Reconstruction of a combined sewer and a secondary water system $629,727.84 ING12-05 Reconstruction of a combined sewer, drinking water conduits, paving and sidewalks $442,906.70 Rivi?re-des-Prairies-Pointe-aux-Trembles borough Service de l'eau - Direction de l'eau potable 9925 (A-327-3) Construction of main and secondary water conduits and various work on 600-mm conduits $5,286,906.98 9953 (A-323-1) Installation of a structural lining to rehabilitate the 500-mm gray cast iron conduit and construction of a 200-mm secondary water conduit $1,744,928.98 Construction of a main water conduit $2,062,966.99 9954 (A-329) Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau 9991 Installation of linings to rehabilitate secondary drinking water conduits on various streets of the Ville de Montr?al 10033 Installation of linings to rehabilitate drinking water conduits on various streets of the Ville de Montr?al (East and West sectors) 9979 Installation of linings to rehabilitate sewers on various streets of the Ville de Montr?al 1109 Reconstruction of a combined sewer and a secondary water conduit $2,472,000.49 1180 Construction and reconstruction of a combined sewer, a secondary water conduit, flexible pavement and sidewalks and development of a cycling path $4,912,027.84 Reconstruction of sidewalks, where necessary, on various streets of the Ville de Montr?al $1,198,325.60 $11,935,721.68 East: $13,062,136.26 West: $11,818,424.01 $12,049,926.67 SITE 162301 Auditor General of the Ville de Montr?al 125 2012 Annual Report Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 Issue of Building and Subdivision Permits - Contaminated Lands 5.4 5.4. Issue of Building and Subdivision Permits - Contaminated Lands Table of Contents 1. Introduction ................................................................................................................... 131 2. Audit Scope................................................................................................................... 132 3. Findings and Recommendations .................................................................................. 132 3.1. Audit of Permit Applications and Compliance with the Applicable Legislative Provisions............................................................................................ 134 3.2. Establishment and Collection of Fees for the Issue of Permits ............................. 150 4. Appendix ....................................................................................................................... 162 4.1. Decision Tree - Processing Building and Subdivision Permit Application ............ 162 Auditor General of the Ville de Montr?al 129 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit List of Acronyms Act Act respecting land use planning and development EQA Environment Quality Act Auditor General of the Ville de Montr?al MDDEFP 130 Minist?re du D?veloppement durable, de l'Environnement, de la Faune et des Parcs 2012 Annual Report 5.4. Issue of Building and Subdivision Permits - Contaminated Lands 5.4. Issue of Building and Subdivision Permits - Contaminated Lands 1. Introduction Soil contamination in Montr?al stems from the industrial history of the Ville de Montr?al (the city). Contaminated lands are concentrated along the city's riverbanks, watercourses and railways as well as former landfill sites throughout the city's territory that were covered over and redeveloped over the years. Contaminated soil management is essentially under the jurisdiction of the Minist?re du D?veloppement durable, de l'Environnement, de la Faune et des Parcs (MDDEFP). Under Division IV.2.1, Chapter I of the Environment Quality Act 1 (EQA), as well as related regulations, policies and guides, the MDDEFP has made rules that apply to the protection and rehabilitation of contaminated lands. In 2003, the legislative provisions of Bill 72 2 , which replaced Division IV.2.1 of Chapter I of the EQA, came into effect and established new rules regarding the protection of land and the rehabilitation of contaminated land. Qu?bec municipalities were directly impacted by these legislative amendments. More specifically, these amendments: o require them to draw up a list of all contaminated lands situated in their territory (section 31.68 of the EQA); o prevent them from issuing building or subdivision permits that concern land that is entered on their list of contaminated lands and the subject of a rehabilitation plan approved by the MDDEFP before obtaining the attestation of an expert establishing that the project for which the permit application is made is consistent with the provisions of the rehabilitation plan (section 31.68 of the EQA, which refers to sections 120 and 121 of the Act respecting land use planning and development [the Act 3 ]). The Direction du greffe of the Ville de Montr?al is the business unit responsible for preparing and maintaining the list of contaminated lands on the basis of the notices registered in the land register (e.g., contamination notices, restriction notices, decontamination notices). 1 2 3 RSQ, chapter Q-2. Act to amend the Environment Quality Act and other legislative provisions with regard to land protection and rehabilitation, SQ, 2002, chapter 11 (assent received on June 8, 2002). RSQ, chapter A-19.1. Auditor General of the Ville de Montr?al 131 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit The city's borough shares the responsibility for issuing permits. In this regard, sections 120 and 121 of the Act set out the rules that apply when issuing building or subdivision permits that concern contaminated lands. 2. Audit Scope Our audit's main objective was to verify whether or not the boroughs comply with the applicable legislative provisions when issuing building and subdivision permits that concern contaminated lands. We also verified whether or not applicants pay the established fees for obtaining their permits. Concurrently, we evaluated whether or not the duties of the delegated officers in charge of the permit issuing process were adequately segregated. More specifically, our audit work focussed on building and subdivision permits issued in 2011 and 2012 by the following business units: o Le Sud-Ouest borough - Direction de l'am?nagement urbain et des services aux entreprises: - Division des permis et inspections; o Mercier-Hochelaga-Maisonneuve borough - Direction de l'am?nagement urbain et des services aux entreprises: - Division des permis et inspections; o Verdun borough - Direction du d?veloppement du territoire, des ?tudes techniques et des services aux entreprises: - Division des permis et des inspections, - Division de l'ing?nierie; o Lachine borough - Direction de l'am?nagement urbain et des services aux entreprises: - Division de l'urbanisme, des permis et inspections. 3. Findings and Recommendations In 2003, following the legislative amendments made by the MDDEFP regarding contaminated soil management, the city's executive committee adopted a document titled "Lignes directrices ? l'intention des services municipaux de la Ville de Montr?al et de ses arrondissements" (the guidelines). These guidelines were intended to assist municipal stakeholders within the central departments and boroughs in the routine management of their operations and ensure that the city met its legal obligations regarding contaminated lands. Since 2003, these guidelines have been updated in accordance with new regulatory provisions and clarifications issued by the MDDEFP. The city's executive committee adopted the revised document on March 7, 2012. The guidelines as well as the list of Auditor General of the Ville de Montr?al 132 2012 Annual Report 5.4. Issue of Building and Subdivision Permits - Contaminated Lands contaminated lands drawn up by the Direction du greffe are available on the city's official portal. The guidelines specifically cover the verifications that boroughs are required to make as well as the measures they must take before issuing building or subdivision permits. Moreover, they define "building permit" and "subdivision permit" in the following terms: Building permit Under the EQA and in accordance with the terminology used in the Act respecting land use planning and development (the Act) (section 119, RSQ chapter A-19.1), a building permit is an authorization issued by a city borough [...] for a building construction, alteration, enlargement or extension project on a site. Subdivision permit Under the EQA, a subdivision permit is an authorization issued by a city borough approving a cadastral operation project. In the audited boroughs, the process for issuing building or subdivision permits is generally triggered when a permit issuing officer 4 receives a written application along with various required documents such as plans and specifications, an evaluation of the project's cost and the location certificate. Upon receipt of a permit application, the officer must: o obtain all documents required to examine the application and make the necessary verifications; o establish the permit fees in accordance with the applicable tariffs; o issue the permit once all application requirements have been met, the applicable fees have been paid and the prior authorizations have been obtained from a senior stakeholder, if applicable. The following sections of this audit report present the results of our audit, based on our examination of 10 building or subdivision permits selected in each of the audited boroughs, with certain exceptions in the following boroughs: o Le Sud-Ouest borough: our audit focussed on eight permits given that the borough eventually cancelled two of the ten initially selected permits that concerned land entered on the list of contaminated lands drawn up by the Direction du greffe; o Mercier-Hochelaga-Maisonneuve borough: our audit dealt with eight permits given that two of the ten initially selected permits that concerned land entered on the list of contaminated lands drawn up by the Direction du greffe had been issued to transform 4 For the purpose of this audit report, we use the expression "permit issuing officer" despite the fact that permits may be issued by permit issuing officers or building inspectors in certain boroughs. Auditor General of the Ville de Montr?al 133 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit rehabilitated lots into subdivisions for condominiums. We noted that a subdivision and building permit had been issued for the lands in question in 2009. Consequently, seeing as our audit focussed on 2011 and 2012, we did not examine these two permits. We examined a total of 36 building and subdivision permits. To the extent possible, we selected permits that concerned lands entered on the list of contaminated lands made available by the Direction du greffe and attempted to include permits spread out evenly between 2011 and 2012 in our selection. However, we also examined permits that concerned lands not entered on the list in order to more extensively evaluate to which extent the verifications required under the EQA had been made. 3.1. Audit of Permit Applications and Compliance with the Applicable Legislative Provisions The Act sets out the conditions that allow a designated municipal officer to issue a building permit (section 120) or a subdivision permit (section 121). Specifically, section 120 sets out the following (section 121 is similarly worded): [...] where the land in respect of which the building permit application is made is entered on the list of contaminated lands drawn up by the municipality pursuant to section 31.68 of the Environment Quality Act (chapter Q-2) and is the subject of a rehabilitation plan approved by the Minister of Sustainable Development, Environment and Parks under Division IV.2.1 of Chapter I of that Act, the permit shall be issued only if the application is accompanied with the attestation of an expert referred to in section 31.65 of that Act establishing that the project for which the permit application is made is consistent with the provisions of the rehabilitation plan. To this effect, the aforementioned city guidelines set out the city's legal obligations when issuing building and subdivision permits. Moreover, the required verifications to ensure that permits that concern contaminated lands are issued in compliance with the regulations are clearly set out in the form of a decision tree titled "Decision Tree - Processing Building and Subdivision Permit Applications" (see Appendix 4.1 of this audit report). Before issuing a permit, and in accordance with the legislative provisions as well the city's guidelines, the permit issuing officer is therefore required to follow the procedure described below. 1. Verify if the land for which the permit application is made has been entered on the list of contaminated lands drawn up by the Direction du greffe (on the basis of the notices registered in the land register). Auditor General of the Ville de Montr?al 134 2012 Annual Report 5.4. Issue of Building and Subdivision Permits - Contaminated Lands To this effect, section 31.51 of the EQA stipulates that a person who permanently ceases an industrial or commercial activity of a category designated by regulation of the government is required to perform a characterization study of the land. The study must be transmitted to the MDDEFP (contamination notice). If the characterization study reveals the presence of contaminants in a concentration exceeding the regulatory limit values, a contamination notice must be registered in the land register. 2. If the land has been entered on the list, the officer must consult the public register of the MDDEFP (in which approved rehabilitation plans are registered) to verify if the MDDEFP approved an environmental rehabilitation plan for the land in respect of which the permit application is made. The obligation to submit a rehabilitation plan to the minister is stipulated in section 31.43 (Division IV.2.1) of the EQA. This rehabilitation plan must set out the measures that will be implemented to protect human beings, the other living species and the environment in general, together with an implementation schedule of the proposed rehabilitation measures. 3. If the land has been entered on the list of contaminated lands and a rehabilitation plan has been approved, under the Act (sections 120 and 121), the permit application must be accompanied by the attestation of an expert establishing that the project for which the permit application is made is consistent with the provisions of the rehabilitation plan. To this effect, in accordance with section 31.65 of the EQA, the MDDEFP draws up and maintains a list of experts authorized to furnish the certificate required under the provisions of the Act. In the case where the land for which a permit application has been made is entered on the list of contaminated lands and in the absence of an approved rehabilitation plan, the permit issuing officer can follow the regular procedure for issuing permits. However, under these circumstances, it must be noted that the applicant has the obligation to decontaminate the land to make it consistent with the requirements of its intended use even though an attestation is not required. 4. Finally, the permit issuing officer must validate the attestation issued by the applicant's expert. Specifically, the officer verifies that: o the land's coordinates are accurate; o the rehabilitation plan identified in the attestation is the one that the MDDEFP approved; Auditor General of the Ville de Montr?al 135 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit o o the person who signed the attestation is an expert authorized by the MDDEFP. To this effect, the MDDEFP posts the list of its authorized experts on its website; the expert duly signed and dated the attestation. If the officer is able to validate all of the above, the permit issuing procedure may continue, namely with respect to establishing the permit fees, having all inherent fees paid by the applicant and obtaining all necessary approvals prior to issuing the permit. We also note that the Act (sections 227, 227.1 and 228) sets outs possible sanctions and recourses against violators. Indeed, it specifically stipulates that, at the request of the Attorney General, the municipality or any interested stakeholder, the Superior Court may order the cessation of a use of land or a structure incompatible with the provisions of a land rehabilitation plan approved by the MDDEFP. Under the circumstances, it is that much more important that boroughs follow a rigorous procedure for issuing building and subdivision permits such as to provide assurance that the city has fulfilled its legal obligations in this regard. We therefore present on the following pages the results of our audit work, which focussed on evaluating the measures taken in each of the audited boroughs and confirming that building and subdivision permits are issued in accordance with the applicable legislative provisions, as per the four-step verification process described above. 3.1.1. Le Sud-Ouest Borough 3.1.1.A. Background and Findings First, we would like to point out that the permit issuing officers of Le Sud-Ouest borough report to the Division des permis et inspections under the Direction de l'am?nagement urbain et des services aux entreprises. Table 1 below describes the characteristics of the eight permits included in our audit sample. Auditor General of the Ville de Montr?al 136 2012 Annual Report 5.4. Issue of Building and Subdivision Permits - Contaminated Lands Table 1 - Distribution of Permit Applications and Entry on the Official List of Contaminated Lands Le Sud-Ouest Borough Type Year Lands entered on the list Lands not entered on the list Total Building permit 2011 2 1 3 2012 2 - 2 Subdivision permit 2011 - 1 1 2 - 2 6 2 8 2012 Total Thus, for these eight applications, the first step of the permit issuing process for the permit issuing officer consisted of verifying if the land in respect of which the permit application had been made was entered on the list of contaminated lands drawn up by the Direction du greffe. It should be noted that, in the case where land for which a permit application is made is not entered on this list, the regular permit issuing process applies. Our audit work revealed the following facts: o We examined the files prepared for the issue of the eight permits making up our sample and found that only four of them (4/8 - 50% of cases) provided evidence that the permit issuing officer had verified if the land was entered on the list of contaminated lands. Although we do not exclude the possibility that the officer had made the necessary verifications, the four other files making up our sample nevertheless provided no such evidence. o Our own verifications confirmed that, among the eight lands for which permit applications had been made, six were entered on the list of contaminated lands drawn up by the Direction du greffe. Regarding these six applications, the permit issuing officer was then required to consult the public register of the MDDEFP to verify if a rehabilitation plan had been approved for the land. In light of the information collected from the persons we interviewed, it appears that permit issuing officers do not systematically make this verification. Moreover, our study of the files allowed us to validate this fact, since no evidence was provided that this verification had been made in these cases. Our own verifications revealed that the MDDEFP had approved rehabilitation plans for four of the six contaminated lands. o When the MDDEFP has approved a rehabilitation plan, the permit application must be accompanied by the attestation of an authorized expert establishing that the project for which the permit application is made is consistent with the provisions of the rehabilitation plan. Conversely, in the absence of an approved rehabilitation plan, the regular permit issuing procedure applies (see Appendix 4.1) without however releasing applicants from their obligation to decontaminate the land to make it consistent with its Auditor General of the Ville de Montr?al 137 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit o intended use. Among the four contaminated lands for which the MDDEFP had approved a rehabilitation plan, our verification revealed that only one file provided evidence that an attestation had been obtained from an expert as required. With respect to the other three lands, although we do not exclude the possibility that an attestation had been obtained and validated, there were no such traces in the files and no mention whatsoever to this effect could be retraced with respect to the permits issued. The permit issuing officer must ultimately validate the attestation provided by the applicant's expert, specifically that the land's coordinates are accurate, that the rehabilitation plan identified in the attestation is the one that the MDDEFP approved, that the person who signed the attestation is an expert authorized by the MDDEFP and that the expert duly signed and dated the attestation (see Appendix 4.1). We noted that none of the four files for which an expert attestation was required provided evidence that the attestation had been validated. This includes the file for which we were able to retrace the expert's attestation. Under the circumstances and given the fact that the permit files provide very little evidence of the work carried out and the supporting documents obtained, we have reason to suspect that the building or subdivision permits that concerned contaminated lands were issued without first making the required verifications or obtaining the necessary attestations. In our opinion, this situation could unnecessarily expose the city to potential lawsuits should it be established that certain persons were prejudicially affected by the borough's negligence in fulfilling its permit issuing obligations. For this reason, we consider it imperative that corrective action be taken as soon as possible to ensure that the borough meets all of its legal obligations regarding the issue of permits for building or subdivision projects that concern contaminated lands. Moreover, based on the information we collected from the employees we interviewed, we concluded that employees were not familiar with certain aspects of the legal obligations regarding the issue of permits of this type (e.g., the obligation to verify if an approved rehabilitation plan exists and to validate the attestation provided by an applicant's expert). Furthermore, although the employees we interviewed told us that they were aware of the city's applicable guidelines, they nevertheless admitted that they did not necessarily refer to these guidelines in their routine operations. Likewise, we noted that the borough had no control mechanism in place to provide a high level of assurance that applications for this type of permit, especially with respect to lands entered on the list of contaminated lands, are processed in accordance with the applicable legislative provisions. In our opinion, a checklist of the various verifications that must be made before issuing a permit in accordance with the applicable legislative provisions and the city's guidelines (in other words, a list of checkpoints) could serve this purpose. It is our Auditor General of the Ville de Montr?al 138 2012 Annual Report 5.4. Issue of Building and Subdivision Permits - Contaminated Lands opinion that all permit issuing officers should systematically use such checklists and keep them on file. This checklist could specifically include: o information on the identity of the applicant, the nature and date of the application as well as the land for which a permit application is made; o a list of the documents obtained from the applicant in accordance with those that applicants are required to provide and that are to be kept on file; o a list of the verifications made in accordance with the law or the city's guidelines (e.g., verification if the land for which a permit application is made is entered on the list of contaminated lands maintained by the Direction du greffe, verification of the existence of a rehabilitation plan approved by the MDDEFP, issue of an attestation by an expert, validation of the attestation issued by the expert). In our opinion, such a mechanism would provide a greater level of assurance to the borough that it has fulfilled its legal obligations regarding contaminated lands and would facilitate the file review process, if applicable. 3.1.1.B. Recommendation We recommend that the Direction de l'am?nagement urbain et des services aux entreprises of Le Sud-Ouest borough take the necessary measures as soon as possible to remind all employees involved in the issue of building and subdivision permits of the specific legislative provisions that apply as well as the existence of the guidelines regarding contaminated lands adopted by the city. Employees should be made aware of the importance of exercising rigorous control to: o avoid issuing permits for projects that concern contaminated lands without having first obtained the required attestations; o provide assurance that the borough has fulfilled all of its legal obligations in this regard. Business unit's response: [TRANSLATION] In collaboration with the Direction de l'environnement (Service des infrastructures, du transport et de l'environnement) of the central city, organize a training session for permit issuing officers on the applicable legislative provisions as well as the guidelines regarding contaminated lands adopted by the city. (Planned completion: June 2013) Auditor General of the Ville de Montr?al 139 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.1.1.C. Recommendation We recommend that the Direction de l'am?nagement urbain et des services aux entreprises of Le Sud-Ouest borough take the necessary measures to: o develop an analysis form listing the various checkpoints that must be verified before issuing permits, given the applicable legislative provisions and the guidelines regarding contaminated lands adopted by the city, as well as the required documents obtained from applicants and kept on file; o have all employees involved in the permit issuing process use this analysis form and keep it systematically on file; in order to provide a greater level of assurance to the borough that it has fulfilled its legal obligations and would ultimately facilitate the file review process. Business unit's response: [TRANSLATION] Incorporate in project analysis forms a section on contaminated lands listing the various verifications that must be made before issuing permits. Prepare and issue a directive on the use of project analysis forms (including the aforementioned section on contaminated lands) to ensure that it is systematically used and kept on file in permit applications. (Planned completion: October 2013) 3.1.2. Mercier-Hochelaga-Maisonneuve Borough 3.1.2.A. Background and Findings In the Mercier-Hochelaga-Maisonneuve borough, the permit issuing officers report to the Division des permis et inspections under the Direction de l'am?nagement urbain et des services aux entreprises. Table 2 below describes the characteristics of the eight permits included in our sample. Auditor General of the Ville de Montr?al 140 2012 Annual Report 5.4. Issue of Building and Subdivision Permits - Contaminated Lands Table 2 - Distribution of Permit Applications and Entry on the Official List of Contaminated Lands Mercier-Hochelaga-Maisonneuve Borough Type Year Lands entered on the list Lands not entered on the list Total Building permit 2011 1 2 3 2012 1 1 2 Subdivision permit 2011 - 1 1 2012 - 2 2 2 6 8 Total As shown in Table 2, despite our attempt to select permits that concerned lands entered on the list of contaminated lands maintained by the Direction du greffe, we only managed to find two 5 permits issued by this borough that met this selection criterion. The remainder of our sample therefore consists of building and subdivision permits that concerned lands not entered on the list of contaminated lands. Nevertheless, for these permits, a verification of this aspect was required under the applicable legislative provisions at the outset of the permit issuing process. Moreover, where the land for which a permit application had been made was not entered on this list, it should be noted that the regular permit issuing process does not require any further verification in particular (see Appendix 4.1). Our audit work thus revealed the following facts: o All of the application files that we examined (8/8 - 100% of cases) provided evidence, i.e., comments recorded in the file, that the permit issuing officer had verified from the outset of the process whether or not the land for which a permit application had been made was entered on the list of contaminated lands drawn up by the Direction du greffe. o For the two lands among the eight that were entered on the list of contaminated lands, the permit issuing officer was then required to consult the public register of the MDDEFP to verify if a rehabilitation plan had been approved. On reading the comments recorded in the file, we concluded that this verification had indeed been made (2/2 - 100% of cases). Furthermore, the rehabilitation plans had been obtained and recorded in the files. o For these same two lands, if the MDDEFP had approved a rehabilitation plan, the officer was required to verify if the applicant had provided the attestation of an authorized expert establishing that the project for which the permit application was made was consistent with the provisions of the rehabilitation plan. After auditing the 5 It must be noted that four permits that concerned lands entered on the list of contaminated lands maintained by the Direction du greffe had been selected initially. However, we excluded two of these permits from our selection for the reasons provided in the preamble of section 3 of this audit report. Auditor General of the Ville de Montr?al 141 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit o application files, we concluded that attestations had been obtained from experts and recorded in the files (2/2 - 100% of cases). Ultimately, the permit issuing officer was required to validate the attestation provided by the applicant's expert. Specifically, the officer was required to verify that the land's coordinates were accurate, that the rehabilitation plan identified was the one that the MDDEFP had approved, that the person who had signed the attestation was an expert authorized by the MDDEFP and that the expert had duly signed and dated the attestation (see Appendix 4.1). Two of audited permit applications provided evidence that these verifications had been made. In light of these results and given the information collected from the persons we interviewed, we conclude that the applicable rules were known and applied. Moreover, these same individuals informed us that they referred to the city's guidelines in their work and that the employees concerned within the borough had received relevant training. 3.1.3. Verdun Borough 3.1.3.A. Background and Findings From the outset, it must be noted that the Verdun borough's organizational structure provides that permit applications may be processed by either the Division des permis et des inspections (in the case of building permits) or the Division de l'ing?nierie (in the case of subdivision permits). Both divisions report to the Direction du d?veloppement du territoire, des ?tudes techniques et des services aux entreprises. Table 3 below describes the characteristics of the ten permits included in our sample. Table 3 - Distribution of Permit Applications and Entry on the Official List of Contaminated Lands Verdun Borough Type Year Lands entered on the list Lands not entered on the list Total Building permit 2011 1 2 3 2012 1 2 3 Subdivision permit 2011 - 1 1 2012 Total 2 1 3 4 6 10 As part of our audit, based on the information gathered from the employees and managers we interviewed, we concluded that they were not all necessarily familiar with the legislative Auditor General of the Ville de Montr?al 142 2012 Annual Report 5.4. Issue of Building and Subdivision Permits - Contaminated Lands provisions applying to the issue of building and subdivision permits that concerned contaminated lands. Indeed, the persons we interviewed were unfamiliar with both the applicable legislative provisions and the guidelines adopted by the city in this regard. Furthermore, it appears that the employees directly responsible for the process had not been trained on the verifications they were required to make or the attestations that needed to be obtained before approving applications and issuing permits of this type. We were able to confirm this affirmation based on our study of the permit applications making up our audit sample. Indeed, our audit revealed the following facts: o Only four applications out of ten, which concerned subdivision permit applications processed by the Division de l'ing?nierie, provided evidence that a verification had been made to determine if the land for which the permit application had been made was entered on the list of contaminated lands maintained by the Direction du greffe (4/10 - 40% of cases). o In fact, of the ten permits included in our sample, four concerned lands that were entered on the list of contaminated lands of the Direction du greffe (2 building permits and 2 subdivision permits). For these applications, the officer was therefore required to establish whether or not the MDDEFP had approved a rehabilitation plan. None of the four permit application files provided evidence that any such verification had been made (4/4 - 100% of cases). Our own verifications revealed that rehabilitation plans had been approved for these four lands. o Under the circumstances, in the case of these four permit applications, the permit issuing officer was required to obtain the attestation of an authorized expert establishing that the project was consistent with the land's condition. We were unable in any of the four files to find evidence that the attestation had been obtained or that it had been validated (4/4 - 100% of cases). Under the circumstances and given the fact that the application files provide very little evidence of the work carried out and the supporting documents obtained, we have reason to suspect the possibility that the building or subdivision permits were issued in respect of contaminated lands without first making the required verifications or obtaining the necessary attestations. In our opinion, this situation could unnecessarily expose the city to potential lawsuits should it be established that certain persons were prejudicially affected by the borough's negligence in fulfilling its permit issuing obligations. For this reason, we consider it imperative that corrective action be taken as soon as possible to provide assurance that the borough has fulfilled all of its obligations regarding the issue of permits for projects that concern contaminated lands. Auditor General of the Ville de Montr?al 143 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit With respect to the corrective action that the borough should implement, we believe that it would benefit from the introduction of a control mechanism providing greater assurance that all building and subdivision permit applications--especially in respect of lands that are entered on the list of contaminated lands--are processed in accordance with the applicable legislative provisions. In our opinion, a checklist of the various verifications that must be made before issuing a permit in accordance with the applicable legislative provisions and the city's guidelines (in other words, a list of checkpoints) could serve this purpose. It is our opinion that all permit issuing officers should systematically use such checklists and keep them on file. It could specifically include: o information on the identity of the applicant, the nature and date of the application as well as the land in respect of which a permit application is made; o a list of the documents obtained from the applicant in accordance with those that applicants are required to provide and that are to be kept on file; o a list of the verifications made in accordance with the law or in the city's guidelines (e.g., verification if the land for which a permit application is made has been entered on the list of contaminated lands maintained by the Direction du greffe, verification of the existence of a rehabilitation plan approved by the MDDEFP, issue of an attestation by an expert, validation of the attestation issued by the expert). In our opinion, such a mechanism would provide a greater level of assurance to the borough that it has fulfilled its legal obligations regarding contaminated lands and would facilitate the file review process, if applicable. 3.1.3.B. Recommendation We recommend that the Direction du d?veloppement du territoire, des ?tudes techniques et des services aux entreprises of the Verdun borough take the necessary measures as soon as possible to remind all employees involved in the issue of building and subdivision permits of the specific legislative provisions that apply as well as the existence of the guidelines regarding contaminated lands adopted by the city. Employees should be made aware of the importance of exercising rigorous control to: o avoid issuing permits for projects that concern contaminated lands without having first obtained the required attestations; o provide assurance that the borough has fulfilled all of its legal obligations in this regard. Auditor General of the Ville de Montr?al 144 2012 Annual Report 5.4. Issue of Building and Subdivision Permits - Contaminated Lands Business unit's response: [TRANSLATION] Subdivision permit issues will be temporarily moved to the 1st floor with the Division de l'urbanisme to regroup in a single location all activities regarding the study of building permit applications and the issue of building permits. We will implement a form, presented as a list of checkpoints, to ensure the follow-up of subdivision permits, including those involving projects that concern contaminated lands. This form will accompany the file and be kept in the file for archiving purposes. At a later stage, the Oracle Permits system will be used for follow-up purposes to ensure that all steps are carried out in compliance with the applicable standards and regulations and that all documents and attestations are received before permits are issued. (Planned completion: September 2013) 3.1.3.C. Recommendation We recommend that the Direction du d?veloppement du territoire, des ?tudes techniques et des services aux entreprises of the Verdun borough take the necessary measures to: o develop an analysis form listing the various checkpoints that must be verified before issuing permits, given the applicable legislative provisions and the guidelines regarding contaminated lands adopted by the city, as well as the required documents obtained from applicants and kept on file; o have this checklist used by all employees involved in the permit issuing process and systematically recorded in permit application files; in order to provide a greater level of assurance to the borough that it has fulfilled its legal obligations and would ultimately facilitate the file review process. Business unit's response: [TRANSLATION] Subdivision permit issues will be temporarily moved to the 1st floor with the Division de l'urbanisme to regroup in a single location all activities regarding the study of building permit applications and the issue of building permits. We will implement a form, presented as a list of checkpoints, to ensure the follow-up of subdivision permits, including those involving projects that concern contaminated lands. This form will accompany the file and be kept in the file for archiving purposes. At a later stage, the Oracle Permits system will be used for follow-up purposes to ensure that all steps are carried out in compliance with the applicable standards and regulations and that all documents and attestations are received before permits are issued. (Planned completion: September 2013) Auditor General of the Ville de Montr?al 145 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.1.4. Lachine Borough 3.1.4.A. Background and Findings The permit issuing officers of the Lachine borough report to the Division de l'urbanisme, des permis et inspections under the Direction de l'am?nagement urbain et des services aux entreprises. The ten permits making up our audit sample have the characteristics described in Table 4 below. Table 4 - Distribution of the Permit Applications and Entry on the Official List of Contaminated Lands Lachine Borough Type Year Lands entered on the list Lands not entered on the list Total Building permits 2011 2 - 2 2012 2 2 4 Subdivision permits 2011 1 2 3 2012 - 1 1 5 5 10 Total For these ten permit applications, the first step of the issuing process consisted of having the permit issuing officer verify if the land in respect of which the permit application was made had been entered on the list of contaminated lands drawn up by the Direction du greffe. In this regard, the division head we interviewed informed us that the borough had used an IT application named "Gestionnaire municipal" to create a virtual map of the borough's territory on which contaminated lands were identified by a contrasting colour. According to this division head, particularly with respect to the contaminated lands, the map is populated with the information provided by the Direction du greffe (email received) as well as historic knowledge of potential risks of contamination on the territory. The permit issuing officers therefore rely on this virtual map to locate contaminated lands. To obtain a reasonable assurance that the map developed by the borough was up to date with respect to contaminated lands, we located on the map five lands that we had identified in the official list of contaminated lands prepared by the Direction du greffe. We were able to locate all of the contaminated lands that we had selected on the map. Therefore, although this method requires the borough to demonstrate a certain level of rigour to ensure that it has access to Auditor General of the Ville de Montr?al 146 2012 Annual Report 5.4. Issue of Building and Subdivision Permits - Contaminated Lands continuously updated information, our audit results tend to demonstrate the borough's databank is reliable. Thus, our audit work led us to make the following findings: o We examined the application files for the ten permits making up our sample but we were unable to retrace any evidence that the permit issuing officer had verified that the lands concerned had been identified on the virtual map. Although we do not exclude the possibility that the permit issuing officer had made the necessary verifications, we found no trace of such verifications in any of the 10 files that we audited. At this stage, it must be noted that, when the land for which a permit application is not identified as contaminated land on the virtual map, the officer may proceed with the permit issue procedure (see Appendix 4.1). o Our own verifications confirmed that of the ten lands for which permit applications had been made, five were entered on the list of contaminated lands drawn up by the Direction du greffe and were identified on the aforementioned virtual map. In the case of these five lands, the permit issuing officer was then required to consult the public register of the MDDEFP to verify if a rehabilitation plan had been approved. In light of the information obtained from the individuals we interviewed, it appears that the borough requires that applicants who want to proceed with land rehabilitation (decontamination) work must first obtain a special authorization certificate, issued by the borough, as well as the building permit. Based on this same information, if such an authorization certificate has been issued, the officer is presumed to have verified the existence of a rehabilitation plan approved by the MDDEFP. However, even though we found evidence of the issue of a special authorization certificate for the five contaminated lands, we were unable to retrace any evidence of the verifications made to confirm the existence of a rehabilitation plan approved by the MDDEFP (0/5 cases). o When a rehabilitation plan has been approved by the MDDEFP, the permit application must be accompanied by the attestation of an expert authorized by the MDDEFP, which establishes that the project is consistent with the condition of the land. Our verifications made in the public register of the MDDEFP revealed the existence of approved rehabilitation plans for the five contaminated lands. However, we found evidence of an expert attestation, as required under the Act, in only one of the five cases (1/5 - 20% of cases). Although we do not exclude the possibility that such an attestation had been obtained, it had not been recorded in the application file and there was no evidence that it had been verified and validated before issuing the permit. o Ultimately, the permit issuing officer must validate the attestation provided by the applicant's expert, namely by verifying that the land's coordinates are accurate, that the rehabilitation plan identified corresponds to the plan approved by the MDDEFP, that the person who signed the attestation is an expert authorized by the MDDEFP and that the Auditor General of the Ville de Montr?al 147 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit expert has duly signed and dated the attestation (see Appendix 4.1). In none of the audited permit applications for which an expert attestation was required were we able to find evidence that the attestations had been validated. This includes the file in which we were able to identify the expert's attestation. In light of these results and given the fact that the application files contain very little evidence of the work carried out and the supporting documents obtained, we have doubts as to the completeness of the verifications actually made by permit issuing officers to avoid issuing permits for projects that concern contaminated lands without having first obtained all of the required attestations or making sure that the borough has fulfilled all of its applicable legal obligations. Moreover, the information collected from the employees we interviewed allowed us to conclude that they were not familiar with some of the legal obligations regarding the issue of this type of permit (e.g., the obligation to validate the attestation provided by an applicant's expert). Furthermore, the employees interviewed told us that they were unaware of the city's applicable guidelines and did not use them as a reference tool in their routine operations. Likewise, we noted that the borough had no control mechanism in place to provide a higher level of assurance that applications for this type of permit, especially with respect to lands entered on the list of contaminated lands, are processed in accordance with the applicable legislative provisions. In our opinion, a checklist of the various verifications that must be made before issuing a permit in accordance with the applicable legislative provisions and the city's guidelines (in other words, a list of checkpoints) could serve this purpose. It is our opinion that all permit issuing officers should systematically use such checklists and keep them on file. It could specifically include: o information on the identity of the applicant, the nature and date of the application as well as the land for which a permit application is made; o a list of the documents obtained from the applicant in accordance with those that applicants are required to provide and that are to be kept on file; o a list of the verifications made in accordance with the law or in the city's guidelines (e.g., verification if the land for which a permit application is made has been entered on the list of contaminated lands maintained by the Direction du greffe, verification of the existence of a rehabilitation plan approved by the MDDEFP, issue of an attestation by an expert, validation of the attestation issued by the expert). In our opinion, such a mechanism would provide a greater level of assurance to the borough that it has fulfilled its legal obligations regarding contaminated lands and would facilitate the file review process, if applicable. Auditor General of the Ville de Montr?al 148 2012 Annual Report 5.4. Issue of Building and Subdivision Permits - Contaminated Lands 3.1.4.B. Recommendation We recommend that the Direction de l'am?nagement urbain et des services aux entreprises of the Lachine borough take the necessary measures to remind all employees involved in the issue of building and subdivision permits of the specific legislative provisions that apply as well as the existence of the guidelines regarding contaminated lands adopted by the city. Employees should be made aware of the importance of exercising rigorous control to: o avoid issuing permits for projects that concern contaminated lands without having first obtained the required attestations; o provide assurance that the borough has fulfilled all of its legal obligations in this regard. Business unit's response: [TRANSLATION] Issue a directive to inform permit issuing officers that they are required to use the analysis form, sign it once they have completed their analysis and record it in the relevant civic file once the application has been processed. (Planned completion: May 2013) Modify our existing analysis form to add the following control points to it: o Verify if the land for which the permit application was made is entered on the list of contaminated lands drawn up by the Direction du greffe; o If so, consult the public register of the MDDEFP to verify if an environmental rehabilitation plan has been approved by the MDDEFP for the land for which the permit application was made; o If so, require the attestation of an authorized expert establishing that the project is consistent with the land's condition; o When an attestation is required, verify that the following requirements are met: - the land's coordinates are accurate, - the rehabilitation plan identified in the attestation is the one that the MDDEFP approved, - the person who signed the attestation is an expert authorized by the MDDEFP, - the attestation has been duly signed and dated by the expert. (Planned completion: May 2013) Issue a directive to inform permit issuing officers that all information obtained to this effect must be recorded in the relevant civic file. (Planned completion: May 2013) Auditor General of the Ville de Montr?al 149 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.1.4.C. Recommendation We recommend that the Direction de l'am?nagement urbain et des services aux entreprises of the Lachine borough take the necessary measures to: o develop an analysis form listing the various checkpoints that must be verified before issuing permits, given the applicable legislative provisions and the guidelines regarding contaminated lands adopted by the city, as well as the required documents obtained from applicants and kept on file; o have this checklist used by all employees involved in the permit issuing process and systematically recorded in permit application files; in order to provide a greater level of assurance to the borough that it has fulfilled its legal obligations and would ultimately facilitate the file review process. Business unit's response: [TRANSLATION] Refer to the corrective action described in the business unit's previous recommendation. 3.2. Establishment and Collection of Fees for the Issue of Permits Adequate controls must be in place for the collection of fees for the issue of permits to ensure that applicants pay a fair price for their permit in accordance with the applicable tariffs and that the city collects the totality of the sums to which it is entitled. With respect to the establishment of fees, all boroughs manage the issue of building permits in accordance with the city's regulation on tariffs. This regulation is revised and adopted annually by the city council. Under this regulation, the cost of a building permit is calculated on the basis of the estimated value of the work, according to the type of building, or on the basis of a minimum lump sum, whichever is higher. Thus, in 2012, the cost of a permit was set at $8.90 per $1,000 of work ($8.70 in 2011) and the lump sum (variable) did not exceed $800 ($780 in 2011). With respect to subdivision permits, fees vary according to the applicable regulations adopted by individual boroughs. At this stage, our audit work consisted of verifying, for the permits making up our sample, if the detailed calculation of the building or subdivision permit fees or evidence that these fees had been established was provided in the application file. We also verified if the borough had duly received the amounts that had been established in accordance with the applicable tariffs prior to issuing the permits. Auditor General of the Ville de Montr?al 150 2012 Annual Report 5.4. Issue of Building and Subdivision Permits - Contaminated Lands 3.2.1. Le Sud-Ouest Borough 3.2.1.A. Background and Findings From the outset, it should be noted that the permit issuing officers of Le Sud-Ouest borough are responsible for establishing permit fees in accordance with the applicable tariffs and for invoicing these fees to applicants. Our audit work thus revealed that, for all of the permits making up our sample (8/8 - 100% of cases), the fees were established and invoiced in accordance with the applicable regulation on tariffs and that applicants duly paid these fees before their permits were issued. Also, for four of the eight permits selected (50% of cases), the application files provided information that enabled us to calculate the established fees (e.g., detailed calculations), whereas for the remaining four (50% of cases), the files provided no such information. We therefore had to redo the calculations to verify if the applicable fees had been invoiced. In our opinion, to render the decision-making process more transparent and facilitate the decision review process, permit issuing officers should keep on file all information regarding the permits issued, including information on the establishment of the permit fees. To this effect, the Direction de l'am?nagement urbain et des services aux entreprises would benefit from issuing formal directives in this regard. Moreover, we noted that each permit issued bore the electronic signature of the manager of the Direction de l'am?nagement urbain et des services aux entreprises attesting the approval of its issue. However, our examination of the permit application files making up our sample revealed that none provided evidence that a senior official had reviewed the permit issuing officers' work prior to the permits being issued. To the contrary, information collected from the individuals we interviewed confirmed that the permit issuing officers' work was not reviewed. In our opinion, corrective action should be taken by implementing a survey-based file review process. To this effect, we believe that the control mechanism (analysis form) that we described at the beginning of this audit report and that the borough could adopt should also provide a section where the designated officer could appose his/her signature to attest to the compliance of the permit issuing process, including with respect to the establishment and collection of permit fees. Finally, according to the information collected, to obtain their permits, applicants must first present the invoice issued by the permit issuing officer at the counter of the borough office and pay the applicable fees. A clerk then processes the payment. For the eight cases Auditor General of the Ville de Montr?al 151 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit included in our audit, we were able to validate that the payment had been processed and found evidence that the fees had been paid. In our opinion, the tasks are adequately segregated in that different stakeholders are entrusted with establishing the permit fees and subsequently receiving the payment of these fees. This is in line with sound management practices and contributes to the implementation of the necessary internal controls to prevent or limit risks of errors, irregularities or misconduct. The same can be said of the levels of review, at higher hierarchical levels, that the borough should implement before issuing permits. 3.2.1.B. Recommendation We recommend that the Direction de l'am?nagement urbain et des services aux entreprises of Le Sud-Ouest borough issue a formal directive to all employees concerned ordering them to keep on file all information regarding permits issued, including information on the establishment of permit fees, to render the decisionmaking process more transparent and facilitate the review process. Business unit's response: [TRANSLATION] Issue and enforce a directive on the information that must be entered in the Oracle Permits system for all permit applications, including information concerning the establishment of the permit fees. (Planned completion: October 2013) 3.2.1.C. Recommendation We recommend that the Direction de l'am?nagement urbain et des services aux entreprises of Le Sud-Ouest borough, for the purpose of providing assurance as to the compliance of the entire process leading to the issue of permits, including the establishment and receipt of the permit fees, take the necessary measures to ensure that: o a senior official is appointed to review the permit issuing officers' files before permits are issued; o evidence of this review and subsequent approval is provided on the analysis form (control mechanism) to be completed and kept on file. Business unit's response: [TRANSLATION] Implement a file review procedure prior to the issue of all building or subdivision permits. (Planned completion: December 2013) Auditor General of the Ville de Montr?al 152 2012 Annual Report 5.4. Issue of Building and Subdivision Permits - Contaminated Lands 3.2.2. Mercier-Hochelaga-Maisonneuve Borough 3.2.2.A. Background and Findings In the Mercier-Hochelaga-Maisonneuve borough, the permit issuing officers are responsible for establishing permit fees in accordance with the applicable tariffs and for invoicing applicants for these fees. Our audit work thus revealed that, for all of the permits making up our sample (8/8 - 100% of cases), the fees were established and invoiced in accordance with the applicable regulation on tariffs and that applicants duly paid these fees before their permits were issued. Moreover, for seven of the eight permits making up our sample (88% of cases), the information on file enabled us to calculate the permits fees that had been established (e.g., detailed calculations). No such information was available in the eighth file. We therefore had to redo the calculations to verify if the applicable fees had been invoiced. In our opinion, to render the decision-making process more transparent and facilitate the decision review process, permit issuing officers should keep on file all information regarding the permits issued, including information on the establishment of the permit fees. To this effect, the Direction de l'am?nagement urbain et des services aux entreprises should remind all of its employees concerned of this requirement. Furthermore, we noted that each permit issued bore the electronic signature of the manager of the Direction de l'am?nagement urbain et des services aux entreprises attesting the approval of its issue. Also, the individuals we interviewed mentioned to us that building permit applications were systematically reviewed by the senior permit issuing officer whereas subdivision permit applications were reviewed by the division head before permits were issued. Our examination of the permit applications making up our sample revealed that a senior official had reviewed the building permit applications. Traces of these reviews are recorded directly in the computerized application ("Gestion du territoire - Permis") used by the borough to process permit applications. With respect to subdivision permits, although the division head claims that the decisions endorsed by the decisionmaking record management system (GDD) is evidence of the verifications he made, we have yet to find evidence that the required legislative verifications were made where contaminated land was concerned. In this regard, it is our opinion that a trace of the verifications made with respect to this type of permit should also be kept on file using the Gestion du territoire - Permis computer application. This would provide evidence of the compliance of the permit issuing process, including with respect to the establishment and collection of fees, as is the case with building permits. Providing evidence of the reviews Auditor General of the Ville de Montr?al 153 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit and approvals obtained appears that much more relevant given that the borough is planning to stop using the GDD application in the preparation and endorsement of its decisions. Finally, according to the information collected, to obtain their permits, applicants must first present the invoice issued by the permit issuing officer at the counter of the borough office and pay the applicable fees. A clerk then processes the payment. For the eight cases included in our audit, we were able to validate that the payment had been processed and found evidence that the fees had been paid. In our opinion, the tasks are adequately segregated in that different stakeholders are entrusted with establishing the permit fees and subsequently receiving the payment of these fees. This is in line with sound management practices and contributes to the implementation of the necessary internal controls to prevent or limit risks of errors, irregularities or misconduct. The same can be said of the levels of review, at higher hierarchical levels, that the borough should implement before issuing permits. 3.2.2.B. Recommendation We recommend that the Direction de l'am?nagement urbain et des services aux entreprises of the Mercier-Hochelaga-Maisonneuve borough reiterate to all employees concerned the importance of keeping on file all information regarding permits issued, including information on the establishment of permit fees, to render the decision-making process more transparent and facilitate the review process. Business unit's response: [TRANSLATION] Distribute an internal directive on the methods used to establish the permit fees stipulating that the worksheets must be kept on file for building permit applications. (Planned completion: June 2013) 3.2.2.C. Recommendation We recommend that the Direction de l'am?nagement urbain et des services aux entreprises of the Mercier-Hochelaga-Maisonneuve borough take the necessary measures to have a designated senior official keep evidence in all subdivision permit application files of the verifications made and the approvals obtained in the computerized application used by permit issuing officers to process these permit applications, in order to attest to the compliance of the entire process leading to the issue of permits, including the establishment and receipt of the permit fees. Auditor General of the Ville de Montr?al 154 2012 Annual Report 5.4. Issue of Building and Subdivision Permits - Contaminated Lands Business unit's response: [TRANSLATION] Modify the operational procedure by adding an automatic line in the permit system to prevent files from being closed before a verification is made. (Planned completion: June 2013) 3.2.3. Verdun Borough 3.2.3.A. Background and Findings In the Verdun borough, when an application concerns a subdivision permit issued by the Division de l'ing?nierie, a municipal engineering technician is responsible for establishing the permit fees in accordance with the applicable tariffs and for invoicing the permit. However, when an applicant files for a building permit issued by the Division des permis et des inspections, although the permit issuing officers examine the application, it is an information clerk who establishes the permit fees and invoices the permit. Thus, in the audit work we carried out to verify if the fees had been established in accordance with the applicable tariffs and duly paid by the applicants before their permits were issued, we made the following findings: o For three of the ten permits making up the audit sample (30% of cases), the fees had not been correctly established. In these three cases, the errors disadvantaged the borough (see Table 5). Indeed, in two of these cases (no. 1 and 2 in Table 5), we noted that the correct tariff had been applied but that the minimum (lump sum) amount per dwelling unit had been used whereas the permit fees would have been higher if they had been established on the basis of the estimated value of the work. In the third case (no. 3 in Table 5), the fees that should have been established for the creation of a new civic number were never collected. Table 5 - Incorrect Amounts Invoiced Verdun Borough Actual amount invoiced Amount that should have been invoiced Shortfall for the borough No. Type of permit Year issued 1 Building 2011 $9,360 $10,494 $1,134 2 Building 2011 $99,840 $130,554 $30,714 3 Building 2012 $7,770 Auditor General of the Ville de Montr?al 155 $7,826 $56 $116,970 Total $148,874 $31,904 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit According to the information obtained from the division head responsible for issuing building permits under the Division des permis et des inspections, similar errors would have been detected following a verification made by the city's comptroller in the summer of 2012. She informed us that on August 9, 2012, an internal directive was issued to order the review of all building permit application files and their approval by the division head prior to the issue of the permits. She provided us with a copy of this directive. o For two of the ten cases making up our sample (20% of cases), the permits were issued before the applicants had paid the fees to obtain them. In one case, the building permit was issued in 2011; in the other case, the subdivision permit was issued in 2012. Moreover, for six of the ten selected permits (60% of cases), we were able to calculate the permit fees that had been established (e.g., detailed calculations) from the information kept on file. For the remaining four files (40% of cases), no such information was available. In our opinion, to render the decision-making process more transparent and facilitate the decision review process, the municipal engineering technician (or the information clerk, as the case may be) should keep on file all information regarding the permits issued, including information on the establishment of the permit fees. To this effect, the Direction du d?veloppement du territoire, des ?tudes techniques et des services aux entreprises would benefit from issuing formal directives to this effect. Furthermore, we noted that each permit issued bore the electronic signature of the manager of the Direction du d?veloppement du territoire, des ?tudes techniques et des services aux entreprises attesting the approval of its issue. However, although the individuals we interviewed indicated to us that, since August 2012, applications were reviewed prior to the issue of permits, our examination of the selected files revealed that none of them provided evidence of any review carried out by a superior of the person designated to issue permits (10/10 - 100% of cases). Among these ten permits, three had been issued after the new directive concerning file reviews came into effect on August 9, 2012. Furthermore, one of the calculation errors that we detected during our work (no.3 in Table 5, shortfall of $56)--although minor--concerned one of these three permits issued on September 10, 2012. In this regard, to better appreciate the benefits of the new directive issued, we selected and examined an additional sample of five permits, which had all been evaluated and issued after August 9, 2012. On the one hand, we determined that the fees for obtaining the permits had been correctly established. On the other hand, we found evidence that the competent division head had reviewed all of these permit applications. Auditor General of the Ville de Montr?al 156 2012 Annual Report 5.4. Issue of Building and Subdivision Permits - Contaminated Lands To this effect, it is our opinion that measures should be taken to correct the root cause of the problem in order to provide assurance to the borough that it is collecting all of the fees to which it is entitled, on the one hand, and to more efficiently review permit applications, on the other hand. To this effect, we believe that the control mechanism (analysis form) that we described at the beginning of this audit report and that the borough could adopt should also provide a section where the borough's designated officer could appose his/her signature to attest to the compliance of the permit issuing process, including with respect to the establishment and collection of permit fees. Finally, according to the information collected, to obtain their permits, applicants must first present the invoice issued by the permit issuing officer at the counter of the borough office and pay the applicable fees. The clerk then processes the payment. For the ten cases included in our audit, we were able to validate that the payment had been processed and found evidence that the fees had been paid. In our opinion, the tasks are adequately segregated in that different stakeholders are entrusted with establishing the permit fees and subsequently receiving the payment of these fees. This is in line with sound management practices and contributes to the implementation of the necessary internal controls to prevent or limit risks of errors, irregularities or misconduct. The same can be said of the levels of review, at higher hierarchical levels, that are not always systematically implemented by the borough before permits are issued. 3.2.3.B. Recommendation We recommend that the Direction du d?veloppement du territoire, des ?tudes techniques et des services aux entreprises of the Verdun borough take the measures it deems appropriate to provide the assurance that it is collecting all of the fees to which it is entitled and to prevent permits from being issued before applicants have paid the fees established in accordance with the applicable legislative provisions. Business unit's response: [TRANSLATION] Currently, work covered by the examination fees is carried out by the information clerk. However, we plan to add a permit issuing officer to the organizational structure. One of this person's roles will be to ensure compliance and follow-up of permit applications such that they reflect the nature of costs of the work described in the application. (Planned completion: December 2013) Auditor General of the Ville de Montr?al 157 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.2.3.C. Recommendation We recommend that the Direction du d?veloppement du territoire, des ?tudes techniques et des services aux entreprises of the Verdun borough issue a formal directive to all employees concerned ordering them to keep on file all information regarding permits issued, including information on the establishment of permit fees, to render the decision-making process more transparent and facilitate the review process. Business unit's response: [TRANSLATION] Calculations will be made in an Excel spreadsheet that will be kept on file with the permit application. (Planned completion: June 2013) 3.2.3.D. Recommendation We recommend that the Direction du d?veloppement du territoire, des ?tudes techniques et des services aux entreprises of the Verdun borough take the necessary measures to: o more efficiently review applications before issuing permits; o ensure that evidence of the reviews conducted and approvals obtained is provided on the analysis form (control mechanism) that is the completed and kept on file; in order to attest to the compliance of the entire permit issuing process, including the establishment and collection of permit fees. Business unit's response: [TRANSLATION] Payment documents such as the invoice and proof of payment will be kept on file with permit applications. (Planned completion: May 2013) 3.2.4. Lachine Borough 3.2.4.A. Background and Findings In the Lachine borough, permit issuing officers establish permit fees in accordance with the applicable tariffs and issue the invoices to permit applicants. We thus made the following findings in the audit work we carried out to verify if the fees had been established in accordance with the applicable tariffs and duly paid by the applicants before their permits were issued: Auditor General of the Ville de Montr?al 158 2012 Annual Report 5.4. Issue of Building and Subdivision Permits - Contaminated Lands o For one of the ten permits making up the audit sample (10% of cases), the fees were not correctly established. In this case, the error disadvantaged the borough (see Table 6). Indeed, we noted that the correct tariff had been applied but that the amount had been calculated on the basis of the estimated value of the work. Had the minimum (lump sum) amount per dwelling unit been used, the permit fees would have been higher. Table 6 - Incorrect Amount Invoiced Lachine Borough No. Type of permit Year issued 1 Building 2012 Total o Actual amount invoiced Amount that should have been invoiced Shortfall for the borough $87,700 $95,940 $8,240 $87,700 $95,940 $8,240 For two of the ten cases making up our sample (20% of cases), the permits were issued before the applicants had paid the fees to obtain them. In both cases, building permits were involved; one was issued in 2011 and the other was issued in 2012. In none of the selected permit applications (10/10 - 100% of cases) were we able to find any information on file enabling us to recalculate the permit fees that had been established (e.g., detailed calculations). In our opinion, to render the decision-making process more transparent and facilitate the decision review process, permit issuing officers should keep on file all information regarding the permits issued, including information on the establishment of the permit fees. To this effect, the Direction de l'am?nagement urbain et des services aux entreprises should issue formal directives to this effect. Moreover, according to the information collected, to obtain their permits, applicants must first present the invoice issued by the permit issuing officer at the counter of the borough office and pay the applicable fees. A clerk then processes the payment. For the ten cases included in our audit, we were able to validate that the payment had been processed and found evidence that the fees had been paid. At this stage of the process, the tasks are adequately segregated. However, we noted that the same permit issuing officer who had examined the applications and established the permit fees signed the permits to approve their issue. We are aware that this practice is authorized under the borough's "R?glement de permis et certificats" 6 . Indeed, section 1.2.4 under "Terminologie" stipulates the following [TRANSLATION]: 6 Borough council, by-law 2528, April 22, 1991. Auditor General of the Ville de Montr?al 159 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit "Designated municipal officer: Means any inspector 7 of the Ville de Lachine as well as the manager of the Direction du service de l'am?nagement". Notwithstanding this authorization, it is our opinion that this is contrary to sound management practices and that the necessary internal controls should be implemented to prevent or limit risks of errors, irregularities or misconduct. Under the circumstances, the tasks that permit issuing officers carry out are inconsistent in that they both examine and approve the permit applications. We believe that measures should be taken to have permits approved by a person other than the permit issuing officer. Furthermore, we noted that the files kept on issued permits provided no evidence of any review by a hierarchical superior of the permit issuing officers. In this regard, the information obtained from the authorized division head effectively confirms that files are not reviewed. In our opinion, this situation should be corrected by the implementation of a survey-based file review process, even more so since the verifications made on the ten selected permits revealed shortcomings. To this effect, we believe that the control mechanism (analysis form) that we described at the beginning of this audit report and that the borough could adopt should also provide a section where the borough's designated officer could appose his/her signature to attest to the compliance of the permit issuing process, including with respect to the establishment and collection of permit fees. 3.2.4.B. Recommendation We recommend that the Direction de l'am?nagement urbain et des services aux entreprises of the Lachine borough issue a formal directive to all employees concerned ordering them to keep on file all information regarding permits issued, including information on the establishment of permit fees, to render the decisionmaking process more transparent and facilitate the review process. Business unit's response: [TRANSLATION] Issue a directive to all employees concerned ordering them to keep on file all information regarding the permits issued, including information on the establishment of the permit fees. (Planned completion: May 2013) 7 It should be noted that, for the purpose of this audit report, we use the expression "permit issuing officer" regardless of the fact that, in the Lachine borough, permits are issued by building inspectors. Auditor General of the Ville de Montr?al 160 2012 Annual Report 5.4. Issue of Building and Subdivision Permits - Contaminated Lands 3.2.4.C. Recommendation We recommend that the Direction de l'am?nagement urbain et des services aux entreprises of the Lachine borough take the necessary measures to: o have issued permits signed to confirm their approval by a person other than the permit issuing officer who examined the application and established the permit fees; o designate a senior official to review application files before permits are issued; o ensure that evidence of the reviews conducted and approvals obtained is provided on the analysis form (control mechanism) that is completed and kept on file; in order to provide assurance to the borough that its entire process is compliant, that it collects the totality of the sums to which it is entitled and that permits are not issued before applicants pay the fees established in accordance with the applicable legislative provisions. Business unit's response: [TRANSLATION] Issue a directive to permit issuing officers to inform them of the implementation of a survey-based file review process applying to building and subdivision permits. (Planned completion: June 2013) As is the practice in the other boroughs, on top of the handwritten signature of the permit issuing officer who examined the application and established the applicable fees, add the electronic signature of the manager of the Direction de l'am?nagement urbain et des services aux entreprises on the sign and permit application. (Planned completion: June 2013) Designate the head of the Division de l'urbanisme, des permis et inspections as the person responsible for reviewing survey-based files prior to the issue of subdivision permits. (Planned completion: June 2013) Designate the senior permit issuing officer as the person responsible for reviewing survey-based files prior to the issue of building permits. (Planned completion: June 2013) On our analysis form, add a space reserved for the person responsible for reviewing survey-based files in which he/she can indicate and sign off on the items that were counterchecked. (Planned completion: June 2013) Auditor General of the Ville de Montr?al 161 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 4. Appendix 4.1. Decision Tree - Processing Building and Subdivision Permit Application Figure A - Decision Tree Is the land entered on the official list of contaminated lands? no yes Did the MDDEFP approve the rehabilitation plan? no yes The permit issuance procedure can continue yes (Note 1) Does the application include an expert's certificate? no Inform the applicant that an expert's certificate is required before a permit can be issued yes Expert certificate checklist yes no Are the site's coordinates accurate? Does the rehabilitation plan identified in the certificate correspond to the document approved by the MDDEFP? Is the person who signed the certificate a MDDEFP-accredited expert? Did the expert duly date and sign the certificate? yes Is the certificate valid? All validation criteria must be met no Inform the applicant where in the checklist his or her application does not meet the validation criteria and that he/she must submit a revised certificate. Note 1: Although an attestation is not required, the applicant nevertheless has the legal obligation to decontaminate the land to make it consistent with its intended use. Source: Lignes directrices ? l'intention des services municipaux de la Ville de Montr?al et de ses arrondissements. Document prepared by the Division de la planification et du suivi environnemental of the Direction de l'environnement, Service des infrastructures, du transport et de l'environnement, February 2012. Auditor General of the Ville de Montr?al 162 2012 Annual Report Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) 5.5 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) Table of Contents 1. Introduction ................................................................................................................... 167 2. Audit Scope................................................................................................................... 169 3. Findings and Recommendations .................................................................................. 170 3.1. Physical Inventory Data ........................................................................................ 173 3.2. Infrastructure Condition Data ................................................................................ 178 3.3. Comprehensive Response Strategy ..................................................................... 190 3.4. Level of Service..................................................................................................... 194 3.5. Priority Planning .................................................................................................... 202 3.6. Planning Investment Projects and Maintenance Expenses .................................. 210 3.7. Accountability Reporting ....................................................................................... 225 3.8. Overall Conclusion ................................................................................................ 226 Auditor General of the Ville de Montr?al 165 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit List of Acronyms AB as built DEEU Direction de l'?puration des eaux us?es DEP Direction de l'eau potable DGSRE Direction de la gestion strat?gique des r?seaux d'eau DI Direction des infrastructures DTP Direction des travaux publics IDAS integrated decision aid system IRP integrated response plan MAMR MAMROT Minist?re des Affaires municipales, des R?gions et de l'Occupation du territoire Minist?re des Affaires municipales et des R?gions PRP RP Auditor General of the Ville de Montr?al response plan SIS structural integrity score SITE Service des infrastructures, du transport et de l'environnement TCEP 166 partial response plan three-year capital expenditures program 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) 1. Introduction The Ville de Montr?al (the city) has an impressive water system that supplies drinking water to a population of 1.9 million 1 and an equally impressive sewer system that captures and recovers wastewater. Besides drinking water treatment plants and the wastewater treatment plant, system assets consist of fire hydrants, valves, service connections, manholes, sumps, pumping stations and several thousand kilometres of underground water mains and sewer lines that criss-cross the subsoil. The replacement value of all the water assets is $40 billion, which includes an estimated $24.2 billion for secondary system assets. 2 Jurisdiction over water management is shared among the urban agglomeration council, the city council and the borough councils. The urban agglomeration council is responsible for water supply and wastewater treatment infrastructure and equipment, except for local lines. City council is responsible for management of local water main and sewer line systems, except for the downtown region, which falls under the purview of the urban agglomeration council. Maintenance of the lines falls under the responsibility of each of the boroughs concerned. Depending on their locations, filtration plants draw the water they treat from Lac Saint-Louis, from Rivi?re des Prairies or from the St. Lawrence River. The water is then transported to the city's water system, which consists of 4,250 km of lines. On the Island of Montr?al, 681 km of primary lines carry water to 3,572 km of secondary system lines on city property. Drinking water is distributed through secondary lines to residences, industries, businesses and institutions. As for the sewer system, it consists of nearly 4,234 km of secondary lines into which millions of litres of wastewater flow every day before converging towards the 689 km of sewer mains leading to interceptors, which in turn lead to the wastewater treatment plant. 1 2 Population in 2011 for the Island of Montr?al according to the 2006 and 2011 Statistics Canada population censuses. Plan strat?gique 2010-2014, Direction de l'eau of the Service des infrastructures, transport et environnement, November 17, 2009 (amounts based on market costs for the city in 2008). Auditor General of the Ville de Montr?al 167 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Since 2010, water management-related activities have been under the authority of the Service de l'eau, whereas before, they were under the responsibility of the Service des infrastructures, du transport et de l'environnement (SITE). Responsibility for managing all water assets falls to three operational directorates of the Service de l'eau. Drinking water production plants and primary water lines fall under the jurisdiction of the Direction de l'eau potable (DEP); the sewer main system, interceptors and the wastewater treatment plant come within the remit of the Direction de l'?puration des eaux us?es (DEEU); and secondary water and sewer system assets come under the purview of the Direction de la gestion strat?gique des r?seaux d'eau (DGSRE). Each of these directorates is responsible for investing in renewal and development and for water and sewer system maintenance planning. Each directorate is also in charge of conducting tests and inspections on water and sewer systems based on its assigned responsibilities. Studies conducted in 2002 at the time of the municipal reorganization revealed major deficiencies in the management and funding of water services. They showed that the water and sewer systems were in a state of deterioration. The underground infrastructure had large investment and maintenance deficits due to the paltry budgets allocated to maintenance and upkeep of assets for many years. Given the condition of the lines at that time, future prospects were troubling, and major investments in asset maintenance and renewal were needed to make up for this shortfall and prevent widespread deterioration of the systems in the long term. According to sound management practices, a rate of annual asset renewal should be maintained in order to ensure infrastructure longevity. With respect to water and sewer systems, in November 2002, the Qu?bec government adopted a water policy that set this rate at 1% of the replacement value per year. One of the commitments of this policy was to encourage municipalities to achieve a renewal rate of 0.8% per year in 2007 and 1% per year in 2012 for their systems. Since this commitment was made, municipalities could be eligible for government grants only if their responses were justified by a prioritization set out in a response plan (RP). It was against this backdrop that the SITE undertook, in 2005, to develop an RP for the water and sewer systems throughout the territory of the Island of Montr?al (before the reconstituted municipalities were created). The purpose of this RP was to gain a thorough knowledge of the infrastructure inventory and to produce a diagnosis of the condition of the lines. It also helped establish a process for analyzing and planning responses supporting decision-making. Auditor General of the Ville de Montr?al 168 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) As a result of five years of multidisciplinary collaboration among stakeholders from both inside and outside the city, RPs were produced from November 2009 to May 2010 for the primary water system, for the sewer main system and for secondary water and sewer systems. To accomplish this, given the scope of the assets, the SITE, after engaging a consortium of consulting engineering firms in 2008, used an integrated decision aid system (IDAS) to model the response strategy. This modeling helps optimize responses, taking into account impacts that systems have on each other. The IDAS also incorporates a financial component to produce medium- and long-term investment scenarios. In 2002, the city embarked on a major shift towards responsible asset management, in particular by reviewing several studies on the topic and by producing RPs. Because of the extent and deteriorated condition of the assets, combined with the underinvestment of the past few years, one of the major challenges is to determine as efficiently as possible what work needs to be carried out on underground systems. This means determining the most appropriate responses and the optimal time for implementing them, based on infrastructure life cycles, within a limited budget and with a view to offering the targeted level of service. This challenge has become pressing since June 2012, when the urban agglomeration council approved a comprehensive 10-year vision of the strategic objectives for Montr?al's water strategy. 3 It also adopted the objective of implementing the proposed funding plan, which provides for full self-financing of the city's water management. This ambitious project, which will extend over several years, will require a thorough knowledge of the systems and the implementation of timely, appropriate responses to ensure longevity of assets. 2. Audit Scope The main objective of our audit of the infrastructure project implementation plan was to ensure that responses deployed on the city's infrastructures will be based on the establishment of priorities. Since considerable investments will be required in coming years, we targeted secondary water and sewer system infrastructures. Accordingly, this audit report focuses on the city's responses for maintenance and renewal of the secondary water and sewer lines for which it is responsible. 3 Rapport du Comit? de suivi du projet d'optimisation du r?seau d'eau potable - Version int?grale, September 2011. Auditor General of the Ville de Montr?al 169 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Management of the water and sewer systems is a public health and safety issue for the city. We analyzed the processes and tools used in planning infrastructure projects implemented by the DGSRE of the Service de l'eau to ensure that priorities were well identified. The sample we used for our audit consisted of two boroughs: Le Plateau-Mont-Royal and VilleMarie. We want to point out that the reason for this choice was that these appeared to be the boroughs with the oldest lines and for which the most substantial investments would be required in coming years. Our audit focused mainly on investment project planning in 2010 and 2011, but we also took into account information from previous years and, when the situation required it, from the year 2012. 3. Findings and Recommendations To facilitate comprehension, we have included here the following paragraphs outlining the major steps to be taken prior to RP development. First, in the fall of 2003, the executive committee adopted a resolution to ratify the procedure for creating a water fund. Later, a master water plan introduced in 2004 included a major scheme to implement an RP in order to ensure the longevity of the city's water and sewer systems. The RP had to serve as a baseline for managers for planning short- and medium-term investments. To implement this scheme, the urban agglomeration council awarded a 10-year contract of $30.7 million (taxes included) in January 2005 to a consortium of consulting engineering firms. Under the contract, RPs had to be produced for the secondary water and sewer systems, the primary water system and the sewer main system. Beginning in 2005, SITE resources (the "RP project team" 4 and the Division de la g?omatique) and the consortium of consulting engineering firms were mobilized to produce response plans for water systems. The procedure adopted by the RP project team at that time was based on the one proposed in the Guide d'?laboration d'un plan d'intervention pour le renouvellement des conduites d'eau potable et d'?gout, published by the MAMR. 5 To facilitate the work for municipalities, the MAMR proposed the procedure outlined in Figure 1. The management framework used 4 5 Since its creation in 2005, the administrative unit responsible for developing the RP for water and sewer systems has undergone several name changes. For the purposes of this report, this unit will be called the "RP project team." Minist?re des Affaires municipales et des R?gions, which in 2009 became the Minist?re des Affaires municipales, des R?gions et de l'Occupation du territoire (MAMROT). Auditor General of the Ville de Montr?al 170 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) was minimal, but this procedure was not supposed to replace the procedures already put in place by municipalities. Figure 1 - Procedure for Developing a RP Compiling data - Compiling available data; - Determining critical sectors; - Establishing preliminary diagnosis; - Assessing data needs. Investigating (testing) the condition of the lines - Preparing the investigation program; - Investigating the lines. Updating data Assessing (diagnosing) the condition of the lines - Establishing indicator scores for each segment; - Assessing the overall condition of lines. Investigating and assessing road conditions Establishing priorities for renewal - Classifying segments in decreasing order of priority based on results obtained in previous step; - Preparing summary tables. Developing a reponse plan - Analysing possible responses; - Determining response strategies; - Developing response strategies. Implementing the response plan Source: MAMR. Guide d'?laboration d'un plan d'intervention pour le renouvellement des conduites d'eau potable et d'?gout, October 2005. For the MAMR, the RP had to provide a systematic procedure for establishing response prioritization so that responses would be implemented at the best possible locations and in the best possible ways, thereby optimizing investments. The MAMR also considered the RP to be a planning tool that was part of a comprehensive municipal infrastructure management plan used by municipal decision-makers. In 2006, the MAMR offered municipalities the opportunity to receive financial assistance under the programme de la taxe sur l'essence et de la contribution du Qu?bec (TECQ) (gas tax and Qu?bec contribution program), provided that they submit an RP no later than December 31, 2007. Since this deadline was too short for the city to produce the RP Auditor General of the Ville de Montr?al 171 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit requested, the MAMR agreed to the production of a partial response plan (PRP) for the related municipalities of the agglomeration. This led to the production of a PRP in July 2007. In August 2008, the urban agglomeration council approved an additional budget of $10.1 million (taxes included) for the initial contract awarded to the consortium of consulting engineering firms, both to help the city meet certain governmental requirements, including those of the MAMR, and to integrate road infrastructure data involving integrated response planning. The additional expenditure then had to be spread over a four-year period (from 2008 to 2011). Later, in May 2010, the RP project team produced a final RP, respectively, for 1) the primary water line system, 2) the sewer main system and 3) the secondary water and sewer systems. In the case of the secondary water and sewer systems, one specific RP was also produced in December 2009 for each of the 19 boroughs, and in November 2009 for 10 related municipalities that were originally part of the territory covered by the project. (The RP was initiated before the municipal demerger in 2006). To date, the only report that the urban agglomeration council has approved is the Plan d'intervention des r?seaux d'eau secondaires de la Ville de Montr?al; this was in March 2011. Under this approval, this report was to be submitted to the MAMROT for obtaining financial assistance under various grant programs. To date, the city has complied with the procedure proposed by the MAMR, except as it pertains to integration of surface infrastructures, including pavement. Given the importance of taking surface infrastructures into account in determining the future needs of underground systems, the city has been developing an integrated response plan (IRP) since 2010. According to the information obtained, a specific IRP was issued to each borough at the end of 2012. An IRP report should be submitted to authorities some time in 2013. Throughout the RP development project, several managers and several DGSRE resources were invited to participate on various working committees. Resources from each borough were also called upon to validate the information appearing in the RP. During our audit, we took into account the RPs and the degree to which human and financial resources were utilized in assessing the extent to which the responses deployed on the city's infrastructures were based on the priorities established. To do this, we first considered the inventories of both physical data and status data in the planning process. We then examined the comprehensive response strategy, the targeted level of service that was decided upon and the planning priorities that were determined for 2010 and 2011. Auditor General of the Ville de Montr?al 172 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) Lastly, we analyzed the annual response plan, in terms of investments and maintenance of secondary systems, and accountability reporting. 3.1. Physical Inventory Data 3.1.A. Background and Findings The project planning process must be based primarily on a complete inventory of all secondary water and sewer system assets. We wanted to assess the extent to which the DGSRE had such a complete inventory at its disposal. First, in 2005 and in 2006, efforts were undertaken by the SITE (RP project team and Division de la g?omatique) under the RP project to compile as much information as possible on water mains and sewer lines, thereby building an inventory consisting of a hundred thousand assets compiled from existing data (e.g., engineering plans, compilations of old reports). More than 128,000 hard-copy plans and more than 481,700 photogrammetric 6 survey points pinpoint the locations of lines and specific items in the system such as manholes, sumps, valves and fire hydrants. Software tools were developed to digitize this basic information. In addition, a geomatic application enables users (including DGSRE users) to access the database in which all data on lines are stored, including physical data (e.g., location, diameter, year of construction, type of manufacturing material) and status data. In 2009, available physical inventory data were used to produce an inventory summary for the City of Montreal State of the Infrastructure Report for Water Distribution and Wastewater Collection Systems 7 , produced in 2009. This State of the Infrastructure Report assigns replacement values to these assets, and estimates the replacement value of the secondary water and sewer systems to be $18.59 billion (see Table 1). It can be seen, then, that the sewer system alone accounts for approximately 70% of the $18.59 billion in secondary system assets. We should point out that in order to be complete, the replacement value should also take into account pavement repair, which brings the total up to $24.24 billion. 6 7 Photogrammetry is a set of techniques used to determine the shape, size and position of an object in space from photographs. Supplementary report in the RP that provides a descriptive overview of the condition of systems for each borough and an assessment of the responses and expenditures required for secondary water system maintenance. Auditor General of the Ville de Montr?al 173 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Table 1 - Inventory and Replacement Value of Secondary Water and Sewer System Assets Type of asset Asset Unit Totala Replacement valueb $ billions Water main % Water system km 3,572 2.59 13.8 Valve and valve chamber units 32,348 0.34 1.9 Fire hydrant and isolation valve units 22,605 0.18 1.0 Water main service connection and curb stop units 40,746 2.41 13.0 5.52 29.7 Subtotal c Sanitary sewer line km 632 1.30 7.0 Storm sewer lined km 623 0.91 4.9 e km 2,979 8.64 46.5 Manhole units 64,169 0.77 4.1 Sump and drain units 145,443 1.45 7.8 13.07 70.3 Total without pavement repair 18.59 100.0 Pavement repair 5.65 Total with pavement repairf Sewer system Combined sewer line 24.24 Subtotal a b c d e f Data as of March 20, 2009, except water main service connections for which an estimate is provided. Calculation based on the budget quotation (including lines and accessories) adapted to the city's market in 2008. Sewer system designed to receive both domestic wastewater and process water generated by the industrial sector. Sewer system designed to receive rainwater only. Sewer system designed to receive domestic wastewater, process water and rainwater. Represents the replacement value of all secondary water and sewer system assets, including pavement repair costs. Source: City of Montreal State of the Infrastructure Report for Water Distribution and Wastewater Collection Systems, 2009. In 2009, the final RP was developed on the basis of these physical inventory data. Knowledge of the assets included in the RP facilitates the selection of future responses, provided that the database is complete and up to date. With respect to the completeness of inventory data, we noted that the final RP informs users of certain limitations of the methodology used. Although the city validated all the data to ensure they were reliable, the RP stresses that it is possible to spot discrepancies in relation to basic plans, reality on the ground, type of material used in manufacturing, year of installation or diameter of lines. According to the information obtained, this situation could have been avoided if as-built (AB) plans had been available for all the lines inventoried. AB plans are engineering documents produced following the construction, reconstruction or rehabilitation of an engineering or architectural work, and they take into account changes made to original construction plans. They confirm compliance with plans and specifications and are signed by the person in charge of monitoring the work. Auditor General of the Ville de Montr?al 174 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) According to the information obtained when the inventory used to produce the RP was compiled, the SITE was unable to identify AB plans for all the lines without undue effort, because some of them were too old. The impact of this situation is felt especially during project prioritization. In fact, it is not possible to have accurate knowledge of the composition of materials or the exact position of underground assets using physical inventory data. Such a situation can cause complications during emergency excavations (maintenance work), when unknown factors are discovered in underground infrastructures, which is likely to lead to longer execution times and higher costs. Furthermore, the absence of AB plans during reconstruction or rehabilitation work means that more exhaustive field surveys must be produced before a project goes to tender. In order to have up-to-date physical inventory data for developing future RPs, measures must be in place so that AB plans are obtained systematically when projects are carried out, and also so that these plans are taken into account in the databases concerned. For this purpose, in July 2011, the Direction g?n?rale approved a guideline called "Pr?paration et transmission des plans tels que construits / Plans TQC," which applies to all central departments and boroughs when they exercise a delegated power under a central authority. Its objective is as follows: [TRANSLATION] The purpose of this guideline is to optimize project costs by maintaining an up-to-date shared central data warehouse, through a spatially referenced geographic information system, of all underground public data, including subsoil on the territory, as well as to enact standards, terms and conditions to be complied with in data identification and the transmission of as-built plans (AB), response sketches and guarantees, in order to ensure data reliability, integrity and longevity. 8 According to this guideline, the Division de la g?omatique of the Direction des infrastructures (DI) is responsible for developing terms, conditions and special procedures, for formulating appropriate recommendations to ensure update and access management for the spatial geographic information system and optimal implementation of these recommendations for all infrastructures, both existing and planned, of underground public land. One of the conditions set forth in the guideline when it came into force was that AB plans and response sketches must be delivered to the Division de la g?omatique within a 8 The guideline is Directive C-OG-SDO-D-11-001 entitled "Pr?paration et transmission des plans tels que construits / Plans TQC," July 15, 2011. The updated guideline came into force on October 15, 2012 (C-OGDG-D-12-011). Auditor General of the Ville de Montr?al 175 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit maximum of three months following provisional acceptance of the work by the manager (or authorized representative) of the unit responsible for execution of the work. The guideline further stipulates that the city manager can request accountability reports on its enforcement at any time from the Division de la g?omatique. Also, in July 2011, the Direction g?n?rale approved a procedure 9 for specifying rules for preparing AB plans for all water and sewer systems. Each administrative unit (e.g., DGSRE, Direction des travaux publics 10 ) is responsible for implementing this procedure, integrating its provisions into current activities and enforcing them. It is very important that these administrative frameworks be adhered to, since their purpose is to ensure the integrity of the underground infrastructure inventory; this is all the more crucial in view of the consequences of not having this inventory available, let alone the costs incurred to produce it. However, according to the information obtained, AB plans are not systematically sent to the Division de la g?omatique after work is performed, even since the guideline produced by the Direction g?n?rale came into force. This compromises the integrity of inventory data, an especially serious problem considering that they are used for developing the RP and that managers use them for project selection. At the time of our audit, the manager of the Division de la g?omatique was unable to obtain an accurate assessment of compliance with the guideline since it was adopted. However, he recognized that efforts should be made to increase the awareness of the stakeholders involved. Even though the guideline came into force in July 2011, data compiled for work performed on secondary water and sewer systems from 2009 to 2011 show that only a small percentage of AB plans (11%) were completed and sent to the Division de la g?omatique (see Table 2). 9 10 Procedure C-OG-SDO-P-11-002 entitled "R?gles d'?laboration des plans tels que construits / Plans TQC pour l'ensemble des r?seaux d'aqueduc et d'?gouts," July 15, 2011. An updated procedure came into force on October 17, 2012 (C-OG-SITE-P-12-001). The Direction des travaux publics, or DTP, reports to the SITE, and, since August 2012, has been known as the Direction des infrastructures. Auditor General of the Ville de Montr?al 176 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) Table 2 - Profile of the Production of AB Plans for Work Performed on Secondary Water and Sewer Systems (2009-2011) Projects Outsourced AB plans completed AB plans not completed Implemented by the city AB plans completed AB plans not completed Total Number 27 0 27 65 10 55 92 % 29 0 29 71 11 60 100 Source: Data produced by the Division de la g?omatique. It should be mentioned that when the guideline was updated in October 2012, the deadline for submission of AB plans and response sketches was revised to six months. Since projects are implemented by the DGSRE, by the DI or by a borough, as the case may be, we think that every administrative unit responsible for performing work on secondary system assets should submit AB plans to the Division de la g?omatique within the allotted deadlines. Since the usefulness of the RP is based largely on the integrity of inventory data, and since considerable resources have been allocated to this area, we think that an evaluation should be carried out on the status of the enforcement of the guideline regarding preparation and submission of AB plans. Subsequently, an action plan accompanied by a deadline should be produced so that the necessary corrective action can be taken to ensure the integrity of physical inventory data input into the RP. 3.1.B. Recommendation We recommend that the Direction g?n?rale obtain from the Division de la g?omatique a status report on the enforcement of the guideline entitled "Pr?paration et transmission des plans tels que construits / plans TQC" for all water and sewer systems, along with an action plan, accompanied by a deadline, for having all physical inventory data available, so as to facilitate project planning. Business unit's response: [TRANSLATION] Prepare a status report on enforcement of the guideline entitled "Pr?paration et transmission des plans tels que construits / plans TQC" for all water and sewer systems. Auditor General of the Ville de Montr?al 177 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Submit to the city manager a plan, accompanied by a deadline, for having all physical inventory data available. (Planned completion: September 2013) 3.2. Infrastructure Condition Data 3.2.A. Background and Findings Asset management is based on keeping a detailed inventory, including physical data, and on accurate knowledge of the condition of each secondary system line. Data on the condition of lines are used to evaluate the age of the inventory and the associated risks. For the water system, they are characterized, for example, by the number of breaks, water quality complaints, the number of low-pressure sections and the presence of lead service lines. For the sewer system, data on the condition of lines include structural deficiencies and backups. In 2002, before the RP was launched, a preliminary theoretical finding of the overall condition of the system made by consulting engineering firms showed that: o One third of underground water infrastructures had passed their estimated expiry date (useful life); o Another third would meet the same fate in the next 20 years if nothing were done; o The remaining third did not show any signs of deterioration, more because the infrastructures were new than because of sound maintenance management. Following the establishment of a new organizational structure for water management, the DGSRE became responsible in 2005 for ensuring the compilation of factual data used to describe with greater accuracy the condition of the secondary systems inventory. These new data had to gradually replace the first theoretical estimates (calculated on the basis of useful life). As part of the RP project, data used to describe the condition of systems were integrated into the inventory database beginning in 2005. For the purposes of this project, the methodology used was to evaluate each line in the secondary systems (water and sewer) using an overall condition score, calculated according to a weighted grading system using indicators recognized by the MAMR. Table 3 presents a summary of these indicators and the data used for assessing the condition of the lines. Auditor General of the Ville de Montr?al 178 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) Table 3 - Indicators and Data Retained Local systems Indicators Data retained Repair rate (breaks) Brittleness index Prioritization Identification of sensitive environments: critical sections and critical users. Based on a combination of the presence of low pressure, complaints about drinking water quality and/or the possibility of lead service lines. Functional deficiencies Based on the number of backup complaints according to the number of buildings affected. Structural deficiencies Structural integrity score (SIS). Prioritization Combined and sanitary sewers Based on the material and its year of installation. Functional deficiencies Water mains Number of repairs (breaks)/5 years/1 km (January 1, 2004, to December 31, 2008). Identification of sensitive environments: critical sections and critical users. Source: Plan d'intervention des r?seaux d'eau secondaires de la Ville de Montr?al, Service de l'eau, May 2010. Since then, data compiled on the condition have been used to draw up several profiles. We tracked down an initial profile 11 that was produced in 2009, before the final RP was produced. This RP revealed that: o Factual data existed for the 3,572 km of the secondary water system and the breakage rate for this system was 29 breaks per year per 100 km, which was higher than in most comparable cities (see Figure 2). o Roughly 50% (2,300 km) of the sewer system had been inspected. Of the sections inspected, 147 km were in an advanced state of deterioration and 30 km in a very advanced state of deterioration (technically designated as a state of imminent collapse). Figure 2 - Secondary Water System Breakage Rate/100 km/Year in Seven Major Canadian Cities in 2009 Source: Rapport du Comit? de suivi du projet d'optimisation du r?seau d'eau potable - Version int?grale, September 2011. 11 Rapport du Comit? de suivi du projet d'optimisation du r?seau d'eau potable - Version int?grale, September 2011. Auditor General of the Ville de Montr?al 179 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit The final RP, produced in 2010, referred to a profile of 687 km of lines--184 km of water mains 12 and 503 km of sewer lines 13 , or roughly 8.8% of the city's secondary systems--that required short-term responses. The total investments for responses to be implemented over a five-year period (2009 to 2013) were valued at more than $1.451 billion, of which nearly 79% was attributable to sewer lines. Table 4 - Summary of Investments Provided for in the Short-Term RP Over a Five-Year Period (2009 to 2013) Priority linesa Types of secondary lines Scope km km % $ billions Water 3,572 184 5.2 0.313 Sewer 4,234 503 11.9 1.138 78.4 Total 7,806 687 8.8 1.451 100.0 a Total investment - 5 years % 21.6 Lines requiring investments in the next five years. Source: Plan d'intervention des r?seaux d'eau secondaires de la Ville de Montr?al, Service de l'eau, May 2010. Essentially, a deteriorated line is a major risk. In the case of water mains, breaks can occur, causing large water spouts and flooding, which can deprive essential services (e.g., hospitals, schools) of water. In the case of sewers, a line in poor condition can ultimately collapse, which can cause road collapse and even sewer backups. Such situations have major consequences and require very costly emergency responses. Furthermore, such situations impact on adjacent infrastructures, to say nothing of the considerable drinking water losses and risks related to public safety and citizens' quality of life (e.g., floods, rerouting of traffic, interruption of services). Table 4, which was supplied by the Service de l'eau, is relevant, but does not provide a profile of all water mains and sewer lines. According to the information obtained, the data needed were accessible in the database, but the profile could not be extracted without undue effort. In our opinion, a profile would have had to be easily accessible and show the number of kilometres of critical lines for which short-term responses are required as well as lines that are not yet critical but for which other types of responses would be necessary in order to ensure their lastingness. It is essential to have accurate knowledge of the condition of the system in order to be able to judge the gravity of a situation, determine the most critical sections, decide on the most appropriate responses based on their condition and plan their implementation in a timely, optimal manner. 12 13 Data compiled from 2004 to 2008. Inspection data as of December 31, 2008. Auditor General of the Ville de Montr?al 180 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) During our audit, we assessed the extent to which accessible data enabled the DGSRE to determine the condition of lines when the responses to be implemented had to be selected. Initially, the DGSRE had at its disposal PRPs (produced in 2007) and specific RPs for each borough (produced in December 2009). For secondary sewer system lines, status data originated mainly from inspections carried out up to December 31, 2008, and submitted to the DGSRE prior to February 2009. For this purpose, the SITE awarded six inspection contracts with a total value of $8.1 million to outside firms in 2007 and 2008. Two of these were three-year contracts (2008 to 2010), totalling $6.6 million, which made it possible to conduct inspections on an ongoing basis after the RPs were produced. According to the information obtained, when the final RP was produced (in May 2010), 54% of the territory had been inspected (for the sewer system). For the rest of the inventory-- lines on which no tests were performed--assessment of their condition was made possible thanks to a simulation based on the age of the lines and other relevant physical data (e.g., brick lines and diameter of lines). In this regard, the RP includes a warning to managers not to use RP results without conducting additional tests to confirm the condition of sewer lines (e.g., conventional camera inspections). For the secondary water system, the overall condition score was based on the number of breaks compiled by the boroughs over a five-year period, from 2004 to 2008. When the RP was produced, the consortium of consulting engineering firms working on the project had brought to light a major disparity in the historical data compiled from the boroughs that made it impossible to report with reliability on the condition of all the water mains listed. Since the RP produced in May 2010 was a profile on a specific date and the status data taken into account were not complete, we reviewed the efforts that were expended to complete knowledge of the condition of each system (sewer and water). On this subject, in the final RP the RP project team proposed the adoption of strategies for structuring and prioritizing inspection activities, in order to update status data, determine the most critical sections and decide on the most appropriate responses. Auditor General of the Ville de Montr?al 181 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Condition of the Sewer System First, a sewer line inspection involves using a zoom camera 14 to perform an initial scan. If the results reveal an imminent or probable risk of collapse, or if the line needs to be monitored, these results must be validated with a conventional camera. 15 This second inspection can confirm or refute the first results obtained. The inspection strategy prescribed by the RP produced in May 2010 by the RP project team targeted: o Lines that have never been inspected and need to be inspected for the first time (zoom camera inspection); o Lines for which the results of the first inspection revealed the presence of a risk of imminent (SIS 16 5) or probable (SIS 4) collapse, as well as monitored lines (SIS 3) (conventional camera inspection); o Follow-up inspections. The timeline for conducting such inspections can vary from 1 to 20 years, depending on the type of inspection (e.g., a first inspection), the criticality of the structural condition or the line prioritization. The strategy integrated the concept of prioritization, determined on the basis of consequences of breakdowns, interruptions of service or responses on lines that were unacceptable to the public, both economically and socially. Based on the sensitivity of line segments, system prioritization is divided into three categories: o Category A: sensitivity level - high o Category B: sensitivity level - average o Category C: sensitivity level - low To continue the inspections and after a public call for tenders was issued, the DGSRE awarded four new three-year contracts (covering the years 2010 to 2014) totalling close to $13 million to specialized firms. 14 15 16 Zoom camera inspections are used to determine the condition and the clogging potential of manholes, to evaluate the clogging potential of lines and to assess the internal structural condition of lines. Conventional camera inspections help visualize the condition of the line over the entire section, determine the precise locations of anomalies identified during a zoom camera inspection and determine the types of corrective actions that should be taken. The SIS (structural integrity score) is an evaluation of anomalies that can affect the structural condition of a line, such as cracks, breaks, displaced or open joints, missing bricks or mortar, distortions, collapses or deterioration. Auditor General of the Ville de Montr?al 182 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) In order to assess the extent to which these contracts made it possible to gain complete knowledge of the condition of lines, we attempted to obtain an overview as of December 31, 2011. In response to our request, the Division plan directeur 17 referred to inventory databases to produce an overview of the condition of inspected lines. Referring to Table 5, we note that, to date, 71% of the lines have been inspected. This does not include storm sewer lines (623 km), which have not yet been inspected because they were not covered by a government grant program. Inspections to date have therefore focused solely on combined and sanitary sewers. Of the lines inspected, 8% were critical and 9% needed to be monitored. Some of these inspections were the first conducted with a zoom camera and the results will need to be validated as required. Accordingly, as of December 31, 2011, a higher percentage of lines were inspected (71%) than was reflected by data available in the final RP (54%), but the fact remains that the DGSRE still does not have a complete overview of the condition of the lines. We would also have liked to assess the progress of the condition of the secondary sewer system as of December 31, 2011, in relation to its condition at the time the RP was prepared (December 31, 2008). In the course of our audit, we noted that the DGSRE had no such information at its disposal. According to information obtained, the profile of the condition of the system in past years can be generated from the database, but this involves a great deal of work. As a result, the DGSRE had not assessed changes, for better or worse, in the profile of the condition of secondary lines. 17 Since 2012, the "RP project team" has been under the responsibility of the Division plan directeur, which reports to the DGSRE. Auditor General of the Ville de Montr?al 183 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Table 5 - Inspections of Secondary Sewer Lines as of December 31, 2011 Conditions of associated risks km Lines inspected SIS 1 Acceptable structural condition SIS 2 Minimal risk of collapse in the short term but potential future deterioration SIS 3 Collapse unlikely in the near future but possible future deterioration SIS 4 Probable collapse in the foreseeable future Section not critical Section being monitored Critical section SIS 5 Collapse or imminent collapse 71 1,341 37 634 17 308 9 243 7 54 1 Uninspected lines 1,043 Total 3,623a a % 2,580 29 100 The difference between the 3,623 km of lines as of December 31, 2011, and the 3,611 km in the inventory appearing in the City of Montreal State of the Infrastructure Report for Water Distribution and Wastewater Collection Systems (3 611 km = 632 km of sanitary sewer lines + 2,979 km of combined sewer lines) is accounted for by changes in the inventory since this Report was produced. Source: Profile compiled by the master plan unit, DGSRE. It should be noted that the percentage of lines that were inspected differs from one borough to another. According to the information obtained, the proportion of the territory inspected as of December 31, 2011, was 56% for Ville-Marie borough and 82% for Le Plateau-MontRoyal borough. Of the lines inspected, the percentage of lines requiring short-term responses was more than 42% for Ville-Marie borough and more than 47% for Le PlateauMont-Royal borough, according to estimates. Since these two boroughs represent a vital sector, the situation is very troubling, especially in view of the high percentage of uninspected lines that are suspected of being in an even more deteriorated condition than those that have already been assessed. In conclusion, even though inspections continued after the final RP was produced, we were not given any assurance that they were conducted according to the inspection strategy planned. We think it is imperative that the Service de l'eau adopt an implementation plan that includes timelines for continuation of the testing strategy to complete its knowledge of the condition of the system so that it can plan responses based on actual priorities. Condition of the Water System With respect to the water system, the testing strategy proposed in the final RP produced in 2010 provided that the following activities should be carried out to determine the structural and functional condition of lines: o Compilation of responses deployed on the system; o Leak detection; o Testing of fire hydrants; Auditor General of the Ville de Montr?al 184 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) o o o Testing of valves; Static pressure measurement; Compilation of complaints. According to the strategy, these activities must be carried out according to frequencies established by boroughs. Some of them correspond to current maintenance activities appearing in annual partnership agreements with each borough. It should be mentioned that maintenance activities are performed either by the boroughs' blue collars, or by outside firms. Some line repair work is also contracted out to entrepreneurs. After the RP was produced in 2010, priority was nonetheless given to the entry of data on water main breaks. According to the information obtained, this decision is directly related to government grant programs. We attempted to assess the extent to which knowledge of the condition of lines had increased since the RP was produced. As was the case with the sewer system, it was in response to our request that we obtained this profile from the Division plan directeur. According to this profile, lines were grouped into two categories: critical lines and noncritical lines. Critical lines are lines that had three or more breaks per kilometre per year in the past five years. For the purposes of this indicator, a break is equivalent to repair work performed by a borough. The profile provided to us shows a high percentage of non-critical lines; nevertheless, as of December 31, 2011, 142 km of lines were considered critical because of the number of repairs carried out. The comparison presented in Table 6 between data as of December 31, 2011, and data that were used in the RP (December 31, 2008), shows that the number of critical lines rose by 2%. Table 6 - Percentage of Critical Lines as of the Final RPa and as of December 31, 2011b Secondary water mains Critical lines Non-critical lines Total a b Final RP km % 66 2 3,506 98 3,572 100 2011-12-31 km % 142 4 3,430 96 3,572 100 Data compiled over five years (2004-2008). Profile compiled by the DGSRE. Auditor General of the Ville de Montr?al 185 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit All the activities provided for in the testing strategy are necessary in order to have an indepth knowledge of the condition of the secondary water system inventory, but only data on water main breaks have been taken into account to date. According to the information obtained, data on other activities are not compiled systematically and rigorously by all boroughs. Because of this problem, the DGSRE, through partnership agreements with boroughs, insists on compliance with procedures for updating status data in digital systems. The agreements also stress the need for using institutional tools that were developed to feed digital systems and other institutional databases. In its 2009 user's guide to digital water and sewer systems, addressed to the boroughs, the DGSRE outlines a procedure for sending it the required information. Boroughs therefore play an important role in the integrity of data processed by the DGSRE. When data are not listed, especially data used for corrective responses to seal leaks, the wrong signal is sent to the system, i.e. "no break on a line," and therefore no response is necessary, thereby allowing lines that actually require a possibly high-priority response to deteriorate. According to the information obtained from the DGSRE, it has been a priority in the past few years to make boroughs aware of the importance of carrying out maintenance activities, measuring them with the aid of indicators to detect any problematic situation and entering the necessary information in digital systems. Follow-up performed by the DGSRE revealed that the use of software tools by some boroughs was inadequate. In fact, several boroughs were behind in their data entry, forcing the DGSRE to do the data entry itself. In addition, for the two boroughs reviewed, the DGSRE alleviated the problem temporarily by allocating funds from its own budget in order to catch up on data entry, in 2011, for Ville-Marie borough and, in 2012, for Le Plateau-Mont-Royal borough. Failure to update data has direct consequences for the reliability and completeness of the profile of the condition of assets. Lack of access to a complete profile of the condition of assets can result in poor decisions, especially when responses are selected and prioritized. In our opinion, efforts must be stepped up to increase boroughs' awareness in this area so that they will enter the information required using institutional tools. During follow-up work, the DGSRE detected a problem more serious than the use of digital tools: the boroughs' ability to detect leaks, as a result of high staff turnover. In producing the State of the Infrastructure Report, RP project team committees established maintenance activities reflecting good practices and the frequency with which they should Auditor General of the Ville de Montr?al 186 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) be carried out. They were later validated in interviews conducted with managers in boroughs. In the case of leak detection, this frequency should have corresponded to a complete inspection of the territory twice a year. However, in the targets set by the DGSRE, boroughs must cover 100% of their territory every year tracking leaks, which, according to good practices, is equivalent to only half the effort needed. According to results compiled by the DGSRE, the leak detection rate for all boroughs was 91% in 2010 and 87% in 2011 (see Table 7). With respect to the two boroughs surveyed, we noted an alarming situation for Ville-Marie borough with results of 40% in 2010 and 6% in 2011. As for the percentage of the territory covered by Le Plateau-Mont-Royal borough, a problem was noted in 2010 (result of 10%), but the borough reached the target in 2011 (result of 100%). We also noted a serious problem in reaching the leak detection target set by the DGSRE, especially for Ville-Marie borough. This problem would be even more serious if the target to be reached had to be in line with good practices (200%). The consequence of such a situation is that leaks go undetected and are therefore not repaired, which can lead to major breaks that inconvenience the public and require major, costly repairs. Furthermore, if not all leaks are detected, then not all the repairs are carried out and, as a result, the percentage of critical lines is underestimated and does not reflect the reality. Table 7 - Percentage of Secondary Water Mains Inspected for Leak Detection as of December 31, 2010, and December 31, 2011 Borough 2010-12-31 According to good practices (twice a year)a The whole city Ville-Marie Le Plateau-Mont-Royal 2011-12-31 200% 91% 40% 10% 87% 6% 100% a City of Montreal State of the Infrastructure Report for Water Distribution and Wastewater Collection Systems. Source: Data compiled by the DGSRE. To deal with this situation, the DGSRE set up a working committee made up of representatives of the Service de l'eau and the boroughs. Possible solutions were proposed to the Service du capital humain et des communications to resolve this issue. The solution that was decided upon was to establish a new body in charge of systematic leak detection that would report to the DGSRE. According to the information obtained, the new body should be operational some time in 2013. Auditor General of the Ville de Montr?al 187 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit In conclusion, while progress was made on sewer line inspections, and while the DGSRE made efforts to help boroughs carry out activities enabling them to describe the condition of water mains, we deplore the fact that the Service de l'eau does not have reliable knowledge of the condition of all secondary systems. The addition of factual information to inventory data is likely to change the established response prioritization. However, the fact that up-to-date status data are not available affects decisions as to which lines require responses. Some lines requiring responses may not be detected, but should be, and continue to deteriorate as a result, unless a major break occurs requiring an emergency response. Conversely, in some cases the RP calls for an "immediate" response when the situation does not require it, because more extensive testing shows that the condition does not justify this response level. Finally, we believe that an overview of current knowledge of the condition of the systems should be produced periodically. We believe that the Service de l'eau should produce this overview for the Direction g?n?rale, specifying any problems encountered in producing this information and their consequences. This overview should make it possible to monitor changes in the health of the system and determine the effects of responses on its condition. 3.2.B. Recommendation We recommend that the Direction de la gestion strat?gique des r?seaux d'eau prepare an implementation plan accompanied by a timeline for implementing the testing strategy on the sewer system so that it will have a complete, up-to-date overview of the condition of the inventory and can then decide on appropriate, timely responses. Business unit's response: [TRANSLATION] Note: The testing strategy developed as part of the RP is aimed at establishing a systematic sewer inspection program, but the RP criteria used for recurrences and priorities are based on the assumption that an initial scan of the entire system has already been performed. Our plan calls for gradual implementation of the testing strategy in 2013. The testing strategy criteria must be revised to take into account initial inspections that have yet to be conducted and the backlog in return inspections that we have been facing. The plan we propose is as follows: o Adjustment of testing strategy criteria based on the number of inspections required and suppliers' ability to provide the service. (Planned completion: June 2013) Auditor General of the Ville de Montr?al 188 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) o o Production of lists for the annual sewer system testing program using the testing strategy. (Planned completion: January 2014) Annual report on achievements. (Planned completion: March 2015) 3.2.C. Recommendation We recommend that the Direction de la gestion strat?gique des r?seaux d'eau take the necessary steps to ensure that a minimum level of leak detection activities are deployed on the secondary water system as per the targets fixed in partnership agreements and that it produce an action plan accompanied by a timeline for revising those targets gradually until they are in line with rules of good practices, thereby improving its knowledge of the condition of the system. Business unit's response: [TRANSLATION] Establishment and deployment of a team to track leaks systematically in the secondary water system. (Planned completion: July 2013) Start of systematic scanning of the secondary water system by the team assigned. (Planned completion: July to December 2013) Systematic inspection of the water system based on best practices. (Planned completion: January to December 2014) 3.2.D. Recommendation We recommend that the Direction de la gestion strat?gique des r?seaux d'eau focus on efforts to make boroughs aware of the need to update inventory status data using management tools provided for this purpose in order to improve planning, thereby improving their selection of projects to be implemented. Business unit's response: [TRANSLATION] The DGSRE will develop an action plan to raise the awareness of its own employees and borough employees. This plan will consist of two parts: o Training that focuses on reviewing objectives that involve data updating and using the different tools, and distribution of a training guide along with updates. o Periodic reminders issued to borough administrations about commitments regarding data compilation that were made when partnership agreements were signed. (Planned completion: May 2013) Auditor General of the Ville de Montr?al 189 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.2.E. Recommendation We further recommend that the Direction de la gestion strat?gique des r?seaux d'eau periodically produce an overview of the condition of sewer lines and water mains for the Direction g?n?rale to promote appropriate, timely decision-making. This overview should: o Reflect changes in the situation following inspections and responses deployed on systems; o Specify any problems that boroughs encounter in updating data and state how the situation affects the selection of projects to be implemented. Business unit's response: [TRANSLATION] The Service de l'eau compiles all the data needed to produce an overall assessment of its systems, but does not report systematically to the Direction g?n?rale. To meet the auditor general's requirement, the DGSRE will produce an assessment every year for the Direction g?n?rale. (Planned completion: December 2014) 3.3. Comprehensive Response Strategy 3.3.A. Background and Findings As part of the planning process, the determination of asset preservation needs consists in identifying responses considered necessary following inspections, then classifying them according to their priority. According to the National Guide to Sustainable Municipal Infrastructure 18 (a reference tool for good practices), it is preferable to follow a comprehensive response strategy by reviewing all the assets of a given system. It is recommended that this strategy be followed for reconstruction, rehabilitation and maintenance responses alike. The city's Politique des ?quipements et des infrastructures (which came into force in January 2009) runs along the same lines as these good practices: [TRANSLATION] In view of the condition of its assets, the Ville de Montr?al must assess the risk that components will break down or wear out prematurely. This evaluation will help determine the preventive and corrective measures needed in the short, medium and long terms. The analysis work must include a cost estimate, which is necessary for decision-making. This analysis should also make it possible 18 InfraGuide. Auditor General of the Ville de Montr?al 190 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) to classify assets under one category or another, depending on whether the decision made is to: o Replace the asset; o Keep the asset, but carry out major short-term or long-term repairs; o Keep the asset and do routine maintenance on it; o Stop maintenance work on the asset, postpone repairs and conduct more indepth studies. The purpose of this classification is to make a decision for each asset as well as to set budget allocation priorities. The city's policy also recommends the [TRANSLATION] "design of a structured preventive maintenance program giving specific information on work that must be done to protect and maintain the quality of assets." To help visualize this classification, we identified the types of responses possible based on the life cycle of lines in Tables 8 and 9 using the City of Montreal State of the Infrastructure Report for Water Distribution and Wastewater Collection Systems as a guide. Table 8 - Examples of Responses Associated with the Four Stages in the Life Cycle of Water System Assets Examples of responses Assets Water mains Stage 1 Routine maintenance Stage 2 Major maintenance Stage 3 Structural rehabilitation Stage 4 Reconstruction Detecting leaks, rinsing, thawing lines, repairing breaks, Hazen-Williams coefficient measurements. Projection of mortar or non-structural epoxy. Cladding or tubing, spot repairs, installing anodes. Complete replacement of a line with or without trench. Checking water tightness, inspection Valves and valve and complete handling chambers of valves, cleaning housings. Repairing the Partial replacement, mechanism, repairing installing anodes. the chamber. Replacement. Fire hydrants and isolation valves Complete inspection of fire hydrants, pressure and flow measurements, snow removal, painting, installing a fire hydrant indicator. Repairing the mechanism. Partial replacement, installing anodes. Replacement. Water main service connections and curb stops Minor repairs, thawing service connections. Replacing a rod. Partial replacement. Replacement. Auditor General of the Ville de Montr?al 191 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Table 9 - Examples of Responses Associated with the Four Stages in the Life Cycle of Sewer System Assets Examples of responses Assets Sewer lines Sanitary sewer manholes Sumps and drains Stage 1 Routine maintenance Stage 2 Major maintenance Inspection, cleaning, rat Sealing of joints, root extermination, minor control. repairs. Stage 3 Structural rehabilitation Stage 4 Reconstruction Cladding or tubing, spot repairs. Replacement of the line between two manholes with or without trench. Inspection, cleaning. Sealing cracks, Cladding or projection repairing rungs, of resins or mortar, replacing frames and replacing of grids, plugs. heads and frames. Complete replacement. Inspection, cleaning. Cladding or projection Sealing cracks, of resin or mortar, replacing frames and replacing grids, heads grids. and frames. Complete replacement. We noted that for critical lines and lines requiring monitoring, there was only partial compliance with the Politique des ?quipements et des infrastructures. The RP is used to classify each line according to an overall condition score for both sewers and water mains and contains a list of assets that will need to be rehabilitated or replaced over the next five years. In the case of the final RP produced in 2010, the lines are critical. However, as we mentioned in the previous section, in order to be able to determine and prioritize responses, the condition of all the assets must first be known. However, the exact condition is not known because, for most lines, inspections have not yet been done. If all the status data had been complete, the prioritization established in the RP would have turned out completely differently. In the case of less critical lines or lines requiring maintenance responses, compliance with the Politique des ?quipements et des infrastructures is no better. In fact, we were unable to track down a list identifying corrective action or preventive measures to be taken, whether on lines or on other assets (e.g., valves, fire hydrants). Although the types and frequency of maintenance activities was established when the State of the Infrastructure Report was prepared, we noted that a structured preventive maintenance program had not been set up for all boroughs. A preventive maintenance program is recognized as a best practice, both for asset management and for use of public funds. Preventive maintenance activities are valuable because they help avoid premature degradation requiring costly responses and premature replacement of infrastructures at considerable expense. Auditor General of the Ville de Montr?al 192 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) For these reasons, good practices stress the importance of a preventive maintenance program as part of an infrastructure management approach to protecting and maintaining the quality of infrastructures, thereby ensuring that the targeted level of service are attained in a sustainable, economic way. According to the National Guide to Sustainable Municipal Infrastructure, $1 of timely prevention will postpone the spending of $5 in rehabilitation. Splitting responsibilities among the DGSRE and the 19 boroughs does not make it any easier to implement a comprehensive response strategy. The DGSRE is responsible for selecting responses that can be considered as investments (rehabilitation and replacement), while the boroughs are independently responsible for planning operating expenditures (maintenance activities and minor repairs). In essence, this division of responsibilities, with boroughs in charge of operating budgets and the DGSRE in charge of capital budgets, is not conducive to integrated planning. It is important that a borough handle preventive maintenance through its operating budget, but this may be less obvious when the consequences of failure to do preventive maintenance are reflected in the DGSRE capital budget. In short, asset preservation needs and priority planning are not determined according to a comprehensive response strategy, contrary to the process recommended in: o The National Guide to Sustainable Municipal Infrastructure; o Management practices prescribed in the city's Politique des ?quipements et des infrastructures. At present, planners manage emergencies and engage in rehabilitation, reconstruction or maintenance activities without receiving any assurance that these responses are prioritized properly, on the right lines, or in a timely manner so as to optimize infrastructure life cycle. However, it should be mentioned that partnership agreements are concluded every year between the DGSRE and each borough. In section 3.6 we will discuss in greater detail what is included in partnership agreements. We should mention, however, that they cover investment projects (reconstruction and rehabilitation) that are under the responsibility of the DGSRE or a borough (such as Ville-Marie borough), and additional budgets under the responsibility of the DGSRE for certain current maintenance activities considered lossmaking or problematic. Such agreements allow boroughs to formalize their commitment to carrying out maintenance work and to provide support to the DGSRE for implementing infrastructure projects. In conclusion, despite the existence of the RP and partnership agreements and the DGSRE's efforts to follow up on maintenance activities by means of a targets table, Auditor General of the Ville de Montr?al 193 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit knowledge of the condition of assets at this time does not allow response prioritization involving rehabilitation, reconstruction or maintenance work. Consequently, response determination and priority planning are not done according to a comprehensive response strategy. 3.3.B. Recommendation We recommend that the Direction de la gestion strat?gique des r?seaux d'eau, in collaboration with the boroughs, develop a comprehensive response strategy for determining and prioritizing responses that integrates both maintenance work (preventive and corrective) and major rehabilitation and reconstruction work in order to keep the structures in a desirable condition at the lowest possible cost. This strategy must be accompanied by an action plan and a timeline for its implementation. Business unit's response: [TRANSLATION] The DGSRE prioritizes and establishes responses in accordance with the city's various issues and objectives, which involve all water assets, and in accordance with the budget placed at its own disposal and at the disposal of the boroughs for systems maintenance. Develop a strategy for coordinating maintenance activities with infrastructure renewal planning. (Planned completion: December 2014) 3.4. Level of Service 3.4.A. Background and Findings Knowledge of the condition of assets is the basis upon which a business unit proposes a level of service that is aligned with the assets' desired condition. If all the efforts and resources required are to be channelled towards reaching this targeted condition, it is essential that authorities make a clear commitment regarding a level of service. To do this, they must make choices regarding the safety and reliability of these assets. Clearly, there must be a strong correlation between the desired level of service and the investment level established. In the case of water assets, in 2003 the SITE commissioned a study by consulting engineering firms that found that, according to theoretical estimates, $200 million/year was the average level of investment that would be required over a 20-year period to recover an accumulated investment deficit of $1.5 billion and to ensure renewal of assets. According to the Service de l'eau, these investments would allow all water infrastructures to be Auditor General of the Ville de Montr?al 194 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) preserved, including those in the secondary systems. This investment level was not associated with a level of service defined or determined by authorities. As shown in Figure 3, the average investment level ($200 million/year) could not be reached in the period from 2004 to 2010, which led to a $484 million increase in the investment deficit, for a total of close to $2 billion, despite the $917 million invested during this period. Figure 3 - Service de l'eau Accumulated Investment Deficit (2004-2010) Source: Rapport du Comit? de suivi du projet d'optimisation du r?seau d'eau potable - Version int?grale, September 2011. Then, in 2010, the Direction g?n?rale launched a project for developing a 10-year investment plan. The mandate of this project, which targeted all business units, including the Service de l'eau, was to establish a planning process for developing: o The RPs necessary for the maintenance and development of their assets over a 10year period; o Investment management policies, to ensure, among other things, maintenance of the condition of assets at an optimum level. As a result of this work, business units (except the boroughs) submitted reports on longterm investment planning to standing committees on finance in September 2010. In its report entitled Plan d'investissement sectoriel ? long terme, the Service de l'eau showed that the theoretical estimate of $200 million/year for 2003 no longer reflected the Auditor General of the Ville de Montr?al 195 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit current reality. At that time, the average investment level was underestimated because some costs were omitted due to lack of information, and because new needs emerged. Indeed, as a result of work carried out since then, including the RP, the Service de l'eau had a more accurate, detailed inventory of the water and sewer systems (e.g., length, age, diameter, manufacturing materials) that enabled it to correlate investment needs with the condition of its underground infrastructures. As a result of this more accurate diagnosis, it became possible in 2010 to establish an annual investment level of $400 million/year, or double the theoretical estimate of $200 million/year. 19 This new annual target for all Service de l'eau infrastructures was equivalent to 1% of the replacement value of the assets, over and above the deficit inherited from previous years, which was $1.5 billion. Since annual investments of $400 million/year would have been required, the accumulated deficit as of December 31, 2011, would then be roughly $3.6 billion. Since the scope of our audit was limited to secondary systems, we noted that, according to the Plan d'investissement sectoriel ? long terme submitted in 2010 by the Service de l'eau, investments of $2.456 billion would be required over a 10-year period (2010 to 2020) to ensure the continuity of secondary water and sewer systems. In its presentation, the Service de l'eau described the level of priority or urgency given to this investment need as "immediate response." The annual investment level was then set at $246 million, compared to the estimated level of $127 million in 2003. Table 10 shows a comparison between investments made and the deficit that accumulated from 2004 to 2011. 19 Rapport du Comit? de suivi du projet d'optimisation du r?seau d'eau potable - Version int?grale, September 2011. Auditor General of the Ville de Montr?al 196 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) Table 10 - Secondary Water and Sewer Systems Percentage of the Target Reached by Investments from 2004 to 2011 (in millions of dollars) Target and actual investments - DGSRE 2004 Percentage of the target reached Revised target 2010 2010 2011 Total Average 53 54 44 68 133 55 71 519 65 127 127 127 127 127 127 127 1,016 127 74 73 83 59 -6 72 56 497 62 42% 43% 35% 54% 105% 43% 56% 51% 51% 246 246 246 246 246 246 246 246 1,968 246 205 Percentage of the revised target reached b 2009 32% b Corrected deficit a 2008 86 Deficit 2007 127 Target according to the 2003 studiesa 2006 41 Total invested 2005 193 192 202 178 113 191 175 1,449 181 17% 22% 22% 18% 28% 54% 22% 29% 26% 26% Estimated target in 2003: $1,267 million/10 years, or $127 million/year. Estimated target in 2010: $2,456 million/10 years, or $246 million/year (revised amount that should have been invested beginning in 2004). Source: Presentation given to members of the joint committees on finances, Service de l'eau, September 28, 2010. We note that the investments made from 2004 to 2011 were well below the targets, attaining only 26% of the (revised) average investment level needed to eliminate the deficit. Furthermore, the average level of investments from 2004 to 2011 was nearly $65 million/year, well below the investment level required in accordance with good practices, which is equivalent to 1% of the replacement value of the assets, or $185 million/year 20 (see Table 11). Finally, the underinvestment seen from 2004 to 2011 would have been even greater if the calculation had taken into account investments proposed in the final RP produced in May 2010, or $290 million/year ($1.451 million for five years). Imagine how great the deficit would be if all the systems (primary and secondary) had been considered. Table 11 - Comparison of Targeted Investment (in millions of dollars) Annual investments Target according to the 2003 studies 127 According to good practices (1%) According to the 2010 target Target according to the final RP Actual averages (2004 to 2011) 185 246 290 65 No matter which target is used, whether it be the 2003 target, the revised 2010 target, the target equivalent to 1% of the replacement value of the assets or the target calculated in the RP (2009-2013), underinvestment since 2004 has resulted in a heightened deterioration of 20 Calculation based on a replacement value of $18.59 billion. Auditor General of the Ville de Montr?al 197 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit the condition of the systems, which is far from complying with the government's water policy commitments. The consequences of this underinvestment will be very serious in the years to come if the situation is not rectified, because the number of lines in poor condition will increase and the number of emergency responses for the system will also increase. Furthermore, over the years, projects required will be on a greater scale, not just because of the types of work involved (e.g., in the short term, a line will require rehabilitation work, but in the medium term, reconstruction work will be required), but also because of the costs. Indeed, reconstruction is more costly than rehabilitation. What is more, this situation is likely to have major disruptive effects for citizens. During our audit, we would have liked to assess the impact of underinvestment on the condition of the systems, but, as we mentioned in previous sections, the Service de l'eau did not have complete information. In order for such an analysis to be workable, the condition of the systems should have been evaluated periodically so that its evolution could be measured. In section 3.2, we referred to the profile of the known condition of inspected water mains and sewer lines as of December 31, 2011. It was incomplete, and we were unable to compare it with a similar profile for a previous period. In June 2012, the urban agglomeration council approved the comprehensive 10-year vision of Montr?al's water strategy and its related strategic objectives. It also adopted the implementation of the proposed funding plan (2011-2020), which provides for full selffinancing of water management in 2020. Gross investment needs for water assets for 2011 to 2020 now total $4.6 billion, while net investment needs, which are presented in Table 12, total $3.2 billion. To fund these net investments, the Service de l'eau is proposing a funding plan based on an annual indexing rate of 9% of water management revenue, on additional grants that will be obtained, on the use of accumulated reserves and on recourse to loans for the first five years. The Service de l'eau expects Montr?al's water management to be fully self-financing by 2016, without having recourse to loans. Auditor General of the Ville de Montr?al 198 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) Table 12 - Service de l'eau Comprehensive Funding Plan for Montr?al's Water Strategy (2011-2020) (in millions of dollars) 10-year timeframe (2011-2020) Amount Comments Needs Total operating needs 3,844 Gross investment needs (without development) Drinking water production 1,021 Wastewater treatment 1,127 Secondary systems (downtown) - agglomeration 540 Secondary systems (local) 1,916 Total investment needs (indexed at 2%) 4,604 (Grants) (829) Grant programs in force (2011-2015) Theoretical net funding needs 3,775 100% of needs Net investment needs, actual total 3,020 Total needs (operating and investment) Assumption: needs implementation rate of 80% (based on past experience) 6,864 Revenue Revenue required 6,864 Investment and operating Revenue programmed 3,618 Based on the year 2010 Funding Additional funding required 3,246 Sources of funding Additional grants required Annual indexing of 9% of water management revenue 836 Additional grants required (2014-2020) 2,011 Revenue increased by 9% per year (over 10 years) Use of accumulated reserves 95 Actual loan required (implementation capacity of 80%) Total sources of funding 304 Balance of the $95 million reserve used in the first two years Recourse to loans for the first five years (2011- 2015) 3,246 Source: Rapport du Comit? de suivi du projet d'optimisation du r?seau d'eau potable - Version int?grale, September 2011. Even though this funding plan is itemized every year for 10 years (from 2011 to 2020), the urban agglomeration council has still not made a clear commitment to maintain the annual planned investment level if the assumptions used in the funding plan are not realized (e.g., grants obtained or water revenue increased). Indeed, if, one of the funding sources is lower than expected in any given year, another source must be favoured; otherwise all the investments planned cannot be made, which will delay the elimination of the accumulated deficit. A similar situation occurred when the 2013 budget was adopted, when authorities did not authorize a tax increase for water management. This raises serious questions for us regarding the realism of this strategy. Auditor General of the Ville de Montr?al 199 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit The TCEP 21 is prepared over a three-year period and adopted on an annual basis. Underground infrastructure continuity management requires very-long-term planning, with the assurance that funds will be available over a similar timeframe. Because of the deteriorated condition of the water and sewer systems, the investment level established will depend on two major factors. The first is authorities' orientations concerning deficit elimination and the routine maintenance level for business units. The second is funding scenarios for sustaining these investments. In the current context, such scenarios must be drawn up as realistically as possible, taking into account cuts to infrastructure grant programs announced by the government, the city's budget, which eliminates the water tax increase for 2013, and the recent allegations about 30% cost inflation. In our opinion, the amounts required will need to be re-evaluated in light of these new data and the updated RP in order to offer a level of service that is in line with authorities' orientations. Accordingly, before the necessary long-term investments are approved, clear commitments should be made concerning the desired level of service or condition for all water assets. As we mentioned above, the Service de l'eau has given several presentations to date to inform authorities about the investment deficit and to request the estimated investment amounts. In its various presentations, the Service de l'eau stated that the investment it needs would help preserve infrastructures. This assertion raises several questions that have remained unanswered. What is meant by "preserving infrastructures"? Does it mean maintaining them in their current condition? Does it mean having a targeted percentage of lines that are "in good condition"? In our opinion, the orientations of the Service de l'eau regarding secondary systems should be expressed in precise terms. It would then be possible to reconcile the current investment level, the projected investment level and the condition of the systems. To date, the disparity persists between the level of investments established and the intention of the Service de l'eau to keep the infrastructures in good condition. Authorities did not make clear commitments concerning the level of service and the concomitant long-term investment level that will be required. Nevertheless, several references agree on the importance of making decisions in these areas: o According to the National Guide to Sustainable Municipal Infrastructure, levels of service must be established based on the condition of systems before responses are determined and prioritized. Responses must be based primarily on the condition of the system, but other factors as well, such as the city's strategic orientations (e.g., large 21 Three-year capital expenditures program. Auditor General of the Ville de Montr?al 200 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) o o o projects such as Griffintown) and financial resources available. The Guide further specifies that city council must approve the level of service before formalizing it. In June 2010, standing committees on finance and administration established 15 guidelines concerning the municipal administration's orientations and the city's financial framework for 2011. One of these is the need for clearly determining levels of service. The Direction g?n?rale, in its Politique des ?quipements et des infrastructures, which came into force on January 30, 2009, pointed out the need for setting clear objectives regarding the condition in which property holdings must be kept. As part of the work of developing a 10-year investment plan, the Service des finances planned to develop a new financial policy addressing financial goals to help determine a desirable investment level. Such a policy was to be approved in August 2010, but this did not occur. Consequently, without specific objectives, it is difficult to determine what investment needs must be met, plan response prioritization for the system over the long term and reconcile all this with an investment plan. To reverse the underinvestment trend and slow the deterioration of systems, authorities must determine the level of service to be achieved and concomitant long-term investment levels to be targeted. They must also come to a decision about a realistic funding method. These commitments will enable managers to: o Plan the implementation of all priority responses in a timely manner as part of a comprehensive response strategy; o Assess the attainment of objectives. 3.4.B. Recommendation We recommend that the Direction g?n?rale: o Express the Service de l'eau orientations that were retained by authorities in terms of precise goals concerning level of service; o Set the required long-term investment level; o Obtain approval from authorities concerning the targeted level of service and the corresponding long-term investment level and funding method; o Evaluate the results on a yearly basis; to enable it to plan and implement responses in a timely manner, slow the further deterioration of secondary water and sewer systems and curb the investment deficit, thereby minimizing disruptive effects for citizens. Auditor General of the Ville de Montr?al 201 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Business unit's response: [TRANSLATION] Prepare an information document and submit it to authorities, structuring the investment plan according to the levels of service defined, and a mechanism for evaluating results at an appropriate frequency. (Planned completion: March 2014) Submit to authorities the investment plan according to the levels of service. (Planned completion: June 2014) Note: The information document will be submitted to authorities. It will be up to authorities to approve or not the levels of service and associated investments. 3.5. Priority Planning 3.5.A. Background and Findings The state of the inventory is an essential data component that can be used to assess risks of deficiencies in infrastructures. Knowledge of the state of the inventory must also be sufficient to determine and prioritize responses for the purpose of implementing them in the short, medium or long term. In a context in which budgets are not sufficient to eliminate the investment deficit that has accumulated in the past few years, it is essential to prioritize projects that will help improve the condition of assets, especially in cases considered critical. During our audit, we assessed the extent to which projects selected truly reflected priorities. According to the information obtained, responses involving secondary water and sewer systems are determined in conjunction with the response priorities for other city assets, such as the road system and the primary water system, as well as major projects and boroughs' priorities. It should be clarified at this stage that the RP is not the only source of responses for the DGSRE. In fact, the priorities and needs of DGSRE partners (SITE, other directorates of the Service de l'eau, etc.) can account for new sources of responses, whether or not they appear in the RP. We list other examples of possible sources of responses below. o Emergency responses Some projects arise from emergency responses, triggered by recurring malfunctions (breaks, leaks or backups) on the same line that cannot be repaired by minor responses implemented by boroughs. Auditor General of the Ville de Montr?al 202 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) o Road works Surface work can have an impact on underground infrastructure. In fact, work performed on pavement can weaken or cause major damage to underground lines. It is therefore important that the condition of underground infrastructures be considered before pavement work is undertaken so as to avoid repeated responses on the same section in a short period of time. Because of the close relationship of these two types of infrastructures, it is essential that the DGSRE coordinate responses with road repair projects. For each planned response involving a road repair project, whether it originates from a borough or from the Direction des transports of the SITE (beginning in 2013, road repair projects will fall under the purview of the DI of the SITE), DGSRE stakeholders must determine whether or not a response at the underground level is necessary. o Water projects (primary system) Other responses may be warranted by planned projects for the primary water system under the management of the DEP for the water system and of the DEEU for the sewer system. o The authorities' priorities Another factor that triggers responses on secondary systems involves major projects in progress or in the planning stages, such as the Quartier international de Montr?al, the Quartier des spectacles or the Griffintown project. Large-scale projects such as these were implemented as a result of commitments made by authorities, and responses associated with these projects on secondary systems are therefore prioritized. These large projects require that the DGSRE analyze the condition of existing underground structures and their capacity to meet new needs. o Real estate development projects Finally, real estate development projects are also considered in response planning, mainly because they necessitate extension of the network and a greater capacity for existing lines, for both drinking water distribution and wastewater recovery. The priorities of authorities and other stakeholders must be taken into account when projects are selected, but in most cases the source of projects selected should still be the Auditor General of the Ville de Montr?al 203 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit RP, since it lists the priority responses. The responses retained should be selected based on their order of priority, in line with what authorities were told when the RP development contract was awarded. Each borough's final RP forms the basis for deciding on responses since it includes a list of sections classified according to their priority. These priorities are established on the basis of the importance of an integrated overall condition score. For each section, this score is equal to a combined overall condition score for the sewer line and a combined overall condition score for the water main. For each section, the types of responses (rehabilitation or replacement) and their costs are proposed on the basis of the data used to establish overall condition scores. Since these are high-priority responses, they are planned for the next five years for one or both of the lines, as the case may be. Although the RP of each borough lists responses in order of priority based on an integrated overall condition score, in section 3.2, we addressed the fact that data on the condition of the inventory were not always complete. Such a situation affects the degree to which the order of priority should be relied upon. As a result, the DGSRE cannot place undue reliance on the classification of sections provided for in the RP when selecting projects to be implemented. It can identify lines for which the overall condition score shows a problem, but it must confirm those results through inspections and hydraulic studies. During our audit, we therefore wanted to assess the extent to which investment projects appearing in partnership agreements were provided for in the RP, since these are the most high-priority projects. In the following paragraphs, we will first describe the major steps in the investment project selection process currently used at the DGSRE at the time of our audit. Second, we will analyze the sources of responses planned in 2010 and 2011. Every year, DGSRE engineers first draw up a preliminary list of lines, referring to the RP. According to the information obtained, the RP informs the choice of projects. These engineers then check the data in the inventory database to describe the condition of lines and confirm the need for a response and, if necessary, the type of work to be carried out. They take into account lines that have reached the threshold criteria. For the water system, for example, this threshold corresponds to more than three breaks/100 km/year, while for the sewer system, the threshold criterion corresponds to a SIS of 4 or 5 and, in the case of brick lines, to a SIS of 3. Other lines located in high-priority sections may be selected when they have reached the end of their theoretical useful life. Auditor General of the Ville de Montr?al 204 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) The DGSRE conducts inspections on sewer lines for which data are missing. When potential projects are analyzed, engineers determine whether responses can be deployed on water or sewer systems at the same time. Engineering project leaders make a selection from the potential projects retained, taking into account the priorities of other stakeholders (e.g., the Direction des transports of the SITE, boroughs, major project leaders, other directorates of the Service de l'eau). According to the information obtained, the selection is also made to minimize the impact on traffic. For each of the projects selected, engineers produce a project information sheet giving reasons why sections require responses. Information such as the condition of assets, type of work, technical design data and estimated budget for the work appear on this sheet. The engineering project leader then signs this project information sheet. Next, the DGSRE, in collaboration with each borough, selects the investment projects to be carried out for the year based on the budget provided. This project selection is then communicated to the various stakeholders involved (e.g., the Direction des transports of the SITE) so that they in turn can review them and take them into account in developing their TCEP. As the planning process continues, the design parameters of selected projects are defined. At this stage, the type and scope of projects are adjusted on the basis of inspections and other studies conducted (e.g., hydraulic studies demonstrating the need to review the capacity of existing lines, soil characterization studies). At this stage, projects may be postponed or delayed in light of results obtained. When projects are chosen definitively, they are included in partnership agreements concluded between the DGSRE and the boroughs concerned. We note that projects are determined and prioritized on the basis of the expertise and judgment of DGSRE engineers and take into account a combination of a large volume of technical and qualitative data and a set of constraints. Although the investment project selection process used was described to us, the only proof that these were high-priority projects came from the corresponding project information sheets. We observed that these project information sheets included reasons for supporting project selections. However, we could not track down any documents showing that analysis of all the projects formed the basis of the preliminary list. Auditor General of the Ville de Montr?al 205 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit In our opinion, for all the projects selected for planning purposes in 2010 and 2011, project information sheets should have been produced, both for the projects selected and for those not selected. These sheets should have included cost/benefit analyses that: o Evaluate the impact of an accelerated or postponed response on both the related cost estimates and the quality of life for citizens; o Evaluate compromises between less costly responses that must be paid for now and more expensive responses that must be paid for later. For the investment project selection process established by the DGSRE, we assessed the extent to which projects appearing in partnership agreements matched priorities established by the DGSRE for Le Plateau-Mont-Royal and Ville-Marie boroughs in comparison to those established by other stakeholders. To produce this comparison, we asked the division head at the DGSRE in charge of these two boroughs questions about the source of each of the projects provided for in the 2010 and 2011 partnership agreements. Table 13 shows the source, number and estimated costs of the projects. We note a net increase in the percentage of projects initially retained by the DGSRE: 34.2% in 2010, compared to 69.5% in 2011. As for costs, they climbed from 33% in 2010 to 68.6% in 2011. This suggests that the DGSRE initially favours a greater number of projects after better knowledge of systems is acquired. Table 13 - Le Plateau-Mont-Royal and Ville-Marie Boroughs Source of Projects Provided for in the 2010 and 2011 Partnership Agreements (amounts in thousands of dollars) Source of projects Partnership agreements - 2010 Partnership agreements - 2011 Amount DGSRE Boroughs (emergencies) Direction des transports DEP Residential development Large-scale projects Parc Jean-Drapeau Total for projects % No. % Amount % No. % 10,021 4,489 205 5,119 1,674 6,845 2,011 30,364 33.0 14.8 0.7 16.9 5.5 22.5 6.6 100.0 14 5 1 6 2 12 1 41 34.2 12.2 2.4 14.6 4.9 29.3 2.4 100.0 23,856 2,309 692 5,439 1,671 594 222 34,783 68.6 6.6 2.0 15.6 4.8 1.7 0.7 100.0 32 3 4 4 1 1 1 46 69.5 6.5 8.7 8.7 2.2 2.2 2.2 100.0 In view of the resources (financial and human) that have been deployed by the RP project team since 2005 to develop the RP, we also examined the extent to which the projects selected corresponded to priorities set out in that RP. To do this, we checked whether projects retained in the partnership agreements of Ville-Marie and Le Plateau-Mont-Royal boroughs (regardless of their origin) appeared in the final RP concerned. Table 14 shows Auditor General of the Ville de Montr?al 206 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) that 80% (33/41) of 2010 projects and 65% (30/46) of 2011 projects appeared in the RP. Implementation of these projects within five years was thus considered a priority. But the percentage of projects not appearing in the RP (20% [8/41] in 2010 and 35% [16/46] in 2011) shows that they were not given priority responses according to the criteria used when the RP was developed. Of those projects not included in the RP, we note that 21% were selected by the DGSRE in 2011. As well, 7% (2010) and 2% (2011) of these projects corresponded to emergency responses reported by either the boroughs concerned or the Soci?t? du parc Jean-Drapeau. According to the information obtained, these responses were not listed in the RP when it was developed because inspection data were missing. Table 14 - Le Plateau-Mont-Royal and Ville-Marie Boroughs Comparison of Projects Provided for in Partnership Agreements with Projects Proposed in RPs Partnership agreements - 2010 Projects Partnership agreements - 2011 Le PlateauMont-Royal Ville-Marie Total Le PlateauVille-Marie Mont-Royal Provided for in the RP 13 20 33 17 13 30 Not provided for in the RP 1 7 8 4 12 16 Total 14 27 41 21 25 46 Total Along the same lines, the priorities established in the RP raised questions for us when three major water main breaks occurred on secondary systems in Ville-Marie borough in May, June and July 2012. Lines requiring major repair work were not identified when the RP for Ville-Marie borough was developed, nor were they selected during the project selection process in the 2010, 2011 and even the 2012 partnership agreements. According to the DGSRE, these breaks were apparently not foreseeable. Accordingly, we noted that boroughs' final RPs are not used for selecting projects to be included in partnership agreements based on an order of priority. We extended our analysis to include selected projects appearing in RPs in order to assess the extent to which this information was useful for deciding on responses to be implemented (e.g., rehabilitation or replacement of lines). To do this, we compared the types of work proposed by RPs with those provided for in partnership agreements to see whether they were similar. Table 15 shows that in more than half the cases, the types of responses provided for in partnership agreements were different from those proposed in RPs: 67% (22/33 projects) in 2010 and 63% (19/30 projects) for 2011. Our audit showed that in almost all cases, the RP recommended rehabilitation work, but that when a project was included in a partnership agreement, reconstruction work was provided for instead, which represents higher costs. Auditor General of the Ville de Montr?al 207 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Accordingly, we note that even when selected projects appeared in the final RPs of the boroughs concerned, information that could have been be useful for planning the work (types of responses and estimated costs of work) are usually changed. This leads us to question the usefulness of data provided in RPs to managers when they decide on responses, since it takes additional time and effort to analyze such data. We understand that responses recommended in RPs can be changed as a result of further inspections and analyses. However, this situation leads us to question the accuracy of the investment needs set out in the RP for the next five years. Table 15 - Le Plateau-Mont-Royal and Ville-Marie Boroughs Comparison of Work Provided for in RPs with Work Provided for in Agreements Partnership agreements - 2010 Type of work provided for in partnership agreements Le Plateau- Ville-Marie Total Mont-Royal Partnership agreements - 2011 Le PlateauMont-Royal Ville-Marie Total Similar to RP 6 5 11 10 1 11 Not similar to RP 7 15 22 7 12 19 Total 13 20 33 17 13 30 In closing, the RP was to be a tool for determining potential responses to be implemented according to an order of priority. However, our audit showed that when the time came to select projects to be included in partnership agreements, managers did not systematically use the RPs of the boroughs involved, since many selected projects are not found in them. Furthermore, some major projects needed to be implemented even though they were not included in the RP or in the DGSRE process. Also, projects included in partnership agreements that appear in RPs were not selected according to the priority set forth in the RPs, because this order is inaccurate as a result of inspections that were not completed and because some boroughs did not perform rigorous data entry. Also, information in RPs on the type of work and on project costs is not always reliable or used by managers because many changes are made to projects when they are included in partnership agreements. In short, we did not feel that this mode of operation ensured that all priority responses were identified, or that short-term work was carried out, or would be carried out within a short timeframe, on water and sewer lines identified as critical since the RP was developed. We think that it would be necessary to show evidence of a correlation between priority work identified in the RP and its inclusion in partnership agreements. Given the large number of potential responses in the final RP to be implemented, it is essential that responses be classified according to their priority over a time line of a few years. A reliable classification could be used to create an implementation plan to guide annual project planning when Auditor General of the Ville de Montr?al 208 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) partnership agreements are concluded. Clearly, this implementation plan should be followed up to ensure compliance. If projects not appearing in the RP need to be undertaken, a rationale should be provided. As we mentioned in the previous section, such a situation justifies the continuation of inspections with a view to upgrading the system, because when a line breaks or sewer collapses, the costs of emergency repairs are very high. 3.5.B. Recommendation We recommend that the Direction de la gestion strat?gique des r?seaux d'eau integrate, in an implementation plan covering a definite timeframe, the responses to be implemented following analysis of the priorities emerging from the response plan to guide the selection of projects when partnership agreements are concluded with boroughs. Business unit's response: [TRANSLATION] The DGSRE will put in place a three-year plan based on available data. This plan will take into account both the responses identified in the IRP, new structural data corresponding to emergencies, road repair projects (local and arterial), large-scale projects and other outside stakeholders, all with the ultimate goal of optimizing responses from technical and financial standpoints. (Planned completion: December 2014) 3.5.C. Recommendation We recommend that the Direction de la gestion strat?gique des r?seaux d'eau document, during the annual planning stage, postponement of projects provided for in the implementation plan (arising from the response plan) or decisions to implement projects that are not provided for in the response plan so that it will be able to justify the decisions made. This documentation should include, in particular, an assessment of the impact of project postponement and the decisions to implement unplanned projects on the related cost estimates and on the quality of life for residents. Business unit's response: [TRANSLATION] To make it possible to document all projects that appeared on a preliminary list, we plan to produce a project follow-up document as soon as a project is analyzed. A project follow-up document will be created, both for projects retained and for projects not retained during the planning stage. (Planned completion: December 2013) Auditor General of the Ville de Montr?al 209 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.6. Planning Investment Projects and Maintenance Expenses As part of the implementation of the organizational model for water management, approved by the executive committee in 2003, the DGSRE, in collaboration with each borough, was mandated to develop and ensure the progress of the master plan management. In the case of secondary water systems, this mandate involved both renewal and maintenance. For renewal, the master plan for water asset management is in fact the RP, while for maintenance, the master plan is based instead on responses required for maintenance of the systems of the whole city appearing in the 2009 City of Montreal State of the Infrastructure Report for Water Distribution and Wastewater Collection Systems. This State of the Infrastructure Report focused on evaluation of the responses and investments required for maintenance of secondary water systems, while the RP presented more complete analyses of responses and capital investments. To carry out its mandate, the DGSRE has been concluding an annual partnership agreement with each borough since 2007, for capital investments as well as for certain maintenance expenses. For capital investments, these agreements list the projects planned for the year according to their types (reconstruction, rehabilitation) and estimated costs. Generally, these investment projects should be found in the annual plan of the DGSRE and the resources needed to implement them should be provided. Maintenance activities come under the jurisdiction of the boroughs, in accordance with the By-law concerning the delegation of city council powers to borough councils (02-002). In fact, since the new Ville de Montr?al was created in 2002, boroughs have been carrying out maintenance activities using the budgets allocated to them. However, since 2006, the DGSRE also pays for some of these activities out of its budget, either to compensate for the lack of resources or to correct problems noted in boroughs. Partnership agreements provide a list of certain maintenance activities for various components (e.g., fire hydrants, valves, sumps), specify whether these activities are under the responsibility of the borough or the DGSRE and set targets for each of them. During our audit, we first assessed the extent to which the projects provided for in these partnership agreements in 2010 and in 2011 had been implemented. Second, we looked at the extent to which resources were allocated to boroughs so that they can assume the responsibilities entrusted to them concerning maintenance activities. Auditor General of the Ville de Montr?al 210 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) 3.6.1. Investment Project Planning in 2010 and 2011 3.6.1.A. Background and Findings During our audit, we assessed the extent to which the projects provided for in the 2010 and 2011 partnership agreements had been implemented. To do this, we conducted a comparative analysis of the total investments provided for in the partnership agreements, the budgets allocated during the year for project implementation and actual spending. First, we note that the DGSRE's TCEP budget was lower than the investment amounts provided for in partnership agreements (see Table 16). In fact, the original DGSRE budget was $100 million for 2010 and $127 million for 2011, while partnership agreements provided for total investments of $129 million for 2010 and $138 million for 2011. Table 16 - Investments Planned for 2010 and 2011 (amounts in millions of dollars) Partnership agreements TCEP Year No of projects Amount Original budget Amended budget 2010 2011 234 238 129 138 100 127 102 109 Such a situation arose because the DGSRE budget had not yet been confirmed at the time the partnership agreements were prepared. Partnership agreements are developed on the basis of a projected investment value for implementation costs. According to the information obtained, partnership agreements are not reviewed if the TCEP is lower. Consequently, despite all the efforts that go into selecting projects, it is the annual investment budget allocated to the DGSRE that is used to set the limits for project planning. Because of the budget limit set, not all projects that were initially planned under partnership agreements can be implemented. In the course of a year, the initial budget (original gross budget) may be amended as the result of decisions made by the management of the Service de l'eau. This is the amended investment budget for the DGSRE, which is allocated to projects as they are undertaken. According to Table 17, the budgets allocated to investment projects was $64 million for 2010 and $74 million for 2011, compared to the total for projects provided for in partnership agreements, which was $129 million for 2010 and $138 million for 2011. Auditor General of the Ville de Montr?al 211 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Table 17 - Allocation of Capital Budgets and Implementation Rate (amounts in millions of dollars) Partnership agreements Projects TCEP Amended budgeta Year No. Amount Amount Budget allocated to projects Actual expenditures % % % % amended Amount Amount agreements agreements agreements budget 2010 a 234 129 102 79 64 50 55 43 54 2011 238 138 109 79 74 54 71 51 65 Amended gross budget corresponding to amounts allocated to projects provided for in partnership agreements between boroughs and the DGSRE. Only 50% of the amounts provided for in partnership agreements for 2010 and 54% of the amounts for 2011 were allocated specifically to projects during the year. Moreover, close to 63% ($64/$102) of the 2010 budget and 68% ($74/$109) of the 2011 budget was allocated during the year to projects when they were started. Such a situation reveals that a considerable portion of the budget is not used to start investment projects. However, partnership agreements were to reflect the investment needed for the implementation of priority projects following a long, complex planning process. To assess the reliability of investment project planning, we calculated the implementation rate of these projects by reconciling actual expenditures with the amounts initially projected in partnership agreements. This gives an average implementation rate of only 43% for 2010 and 51% for 2011 for all boroughs. However, since the DGSRE's amended budget was lower than the amount of investments provided for in these partnership agreements, it was the basis we used for calculating the implementation rate. This new implementation rate is 54% for 2010 and 65% for 2011. Since our audit focused more specifically on Le Plateau-Mont-Royal and Ville-Marie boroughs, we did a similar comparison to determine whether these boroughs were different from the average. We therefore compared the total investments provided for in the partnership agreements, the budgets allocated to them and the amount of actual spending (see Tables 18 and 19). We made the same comparisons for the number of projects. Auditor General of the Ville de Montr?al 212 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) Table 18 - Le Plateau-Mont-Royal and Ville-Marie Boroughs Capital Budget Allocations and Implementation Rate - 2010 TCEP (amounts in thousands of dollars) Partnership agreements 2010 TCEP Budget allocated to projectsa Borough No. Amount Le Plateau-Mont-Royal 14 12,564 5 36 Ville-Marie 27 17,800 15 Total 41 30,364 20 a % No. Amount agreements % agreements 2,722 22 56 9,752 49 12,474 Actual expenditures Amount % agreements 1,723 14 55 8,811 50 41 10,534 35 Amended gross budget for amounts allocated to projects provided for in agreements. Table 19 - Le Plateau-Mont-Royal and Ville-Marie Boroughs Capital Budget Allocations and Implementation Rate - 2011 TCEP (amounts in thousands of dollars) Partnership agreements 2011 TCEP Budget allocated to projectsa Borough No. Amount Le Plateau-Mont-Royal 21 16,165 14 67 Ville-Marie 25 18,618 11 44 Total 46 34,783 25 54 a No. % Amount agreements Actual expenditures % agreements Amount % agreements 10,237 63 10,516 65 3,527 19 4,612 25 13,764 40 15,128 43 Amended gross budget for amounts allocated to projects provided for in agreements. Overall, for both years, the comparison confirms that approximately 40% of investment amounts provided for in the 2010 and 2011 partnership agreements were allocated directly to specific projects. As for the number of projects, the comparison shows that close to half of them were started (49% for 2010 and 54% for 2011). However, half the projects were not started, even though the secondary systems of these two boroughs are the most deteriorated. Not only are the projected annual investments insufficient to make up for the investment shortfall, but such a situation contributes to increasing this deficit. We also assessed the reliability of planning for these two boroughs using the implementation rate. We calculated this rate by comparing actual spending with the total amount of investments provided for in partnership agreements. We note that this rate is lower (35% for 2010 and 43% for 2011) than for all the boroughs (43% for 2010 and 51% for 2011). For projects provided for in partnership agreements, Table 20 shows the progress of projects for the two boroughs in our sample. We noted that a considerable percentage of Auditor General of the Ville de Montr?al 213 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit projects were actually postponed or cancelled. Out of 41 projects in 2010 and 46 projects in 2011, all chosen because of their priority, less than 40% (39% in 2010 and 33% in 2011) were terminated during the year, while close to half the projects were postponed or cancelled (51% in 2010 and 46% in 2011). The number of projects in progress at the end of the year was equivalent to a rate of 10% in 2010 and 21% in 2011. We did not obtain evidence that projects begun in 2010 or in 2011 had been selected according to priority. Table 20 - Le Plateau-Mont-Royal and Ville-Marie Boroughs Status of Projects Provided for in Partnership Agreements for 2010 and 2011 Le Plateau-Mont-Royal 2010 Ville-Marie 2011 2010 Total 2011 2010 2011 Project status No. % No. % No. Completed 4 29 In progress 1 7 Postponed 7 50 4 % No. % No. % No. % 8 38 6 29 12 44 7 28 16 39 15 33 3 12 4 16 4 10 10 19 21 12 44 13 52 19 46 17 37 Cancelleda 2 14 3 14 0 0 1 4 2 5 4 9 Total projects 14 100 21 100 27 100 25 100 41 100 46 100 a Projects for which the types of responses were revised and for which implementation was postponed indefinitely. Because of the large number of postponed or cancelled projects and the impact that this has had on the accumulated investment deficit, we asked the division head responsible for each borough in our sample whether the main causes had been systematically determined and analyzed. According to the information obtained, this is not done in a structured way that would enable the evaluation of the consequences and costs. As a result, we noted, the percentages of postponed or cancelled projects could not be compared with all boroughs, since this information was not compiled. We were informed of several problems, both operational and administrative, that limited the full implementation of projects planned for the two boroughs (Ville-Marie and Le PlateauMont-Royal). Below, we have classified the causes under three headings: lack of coordination, limited ability to implement projects by the stakeholders involved in project implementation and, lastly, administrative constraints. o Lack of coordination Projects covered by the partnership agreements of the two boroughs in 2010 and 2011 were postponed because of difficulties in coordinating them with the investment projects Auditor General of the Ville de Montr?al 214 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) of other city stakeholders. For example, large-scale projects such as the CHUM 22 , the MUHC 23 , the Bonaventure Highway, the Griffintown district and the Turcot Interchange limited the DGSRE's scope of action on streets bordering these projects, even though responses had been planned. Other projects had to be postponed because they restricted traffic or because of the adverse affects they had on citizens and merchants. According to the information obtained, projects involving different administrative units also had to be postponed because of the way budgets were allocated by each business unit. For example, if a project is part of DGSRE planning and involves the Division de la gestion des actifs de voirie (DI of the SITE), it is not certain that this project will necessarily be completed with the budget intended for this division. Even if it can be completed within this budget, unforeseen events can occur to affect its implementation. o Limited ability of stakeholders involved to implement projects According to the information obtained, some projects were postponed or cancelled because the business units involved in planning, designing or implementing projects did not have the ability to carry out the requests received. Some projects were postponed or cancelled as a result of changes in the type of work involved; for example, the response needed was rehabilitation rather than reconstruction, as per the plan, or vice versa. According to the information obtained, the results of hydraulic studies requested, which could be used to decide on types of projects to be planned, were received late by the DGSRE because the stakeholders involved lacked the necessary resources. The DGSRE would have contributed additional resources for conducting hydraulic studies to help the units involved respond to the request. We were unable to assess the consequences of these actions. Capital works planning is carried out on the basis of the needs determined, up to the established budget limit, but with no regard for the DI's ability to implement projects. (The DI was formerly known as the DTP when the projects reviewed for 2010 and 2011 were implemented.) Thus, in 2010 and 2011, projects were postponed or cancelled because of a lack of ability to implement them, either during the preparation of plans and specifications or during execution of the work. For projects provided for in partnership agreements, the DGSRE itself manages rehabilitation contracts, but leaves this task up to the DI for reconstruction work. In some cases, boroughs carry out 22 23 Centre hospitalier de l'Universit? de Montr?al. McGill University Health Centre. Auditor General of the Ville de Montr?al 215 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit reconstruction work. Of the projects provided for in the partnership agreements of the two boroughs in our sample, 68% of projects were awarded to the DI (28/41 projects) in 2010 and 57% (26/46 projects) in 2011. However, we were unable to assess the extent to which these project postponements and cancellations were attributable to the DI, to the DGSRE or to the boroughs. The DGSRE would be well advised to submit a list of projects provided for in partnership agreements as promptly as possible to the DI during the planning phase so that it can assess its ability to provide the services requested according to the planned schedule. o Administrative constraints As we mentioned above, in 2010 and 2011, budgets were allocated to the city's business units in the first few months of the year. Only then were partnership agreements with boroughs approved, from February to May, thereby delaying the contract awarding process. Since the contract awarding process demands adherence to strict administrative deadlines, work performance contracts could be awarded only late in the spring and in some cases, even after the summer period. Such a situation affects project launching and sometimes requires that projects be postponed. Furthermore, once projects that were delayed or cancelled because of the abovementioned deficiencies were ready to go to tender, they were delayed once again because of administrative constraints. According to the information obtained, it is unlikely that budgets freed by postponed or cancelled projects will be reallocated to other projects in the same year, since other important deadlines must be met. At the time of our audit, the DGSRE had a list of projects that were included in partnership agreements, but did not have a more comprehensive log of projects that were ready to be implemented to replace cancelled or postponed projects. It had set the objective of creating a project bank over a threeyear period, which would speed up project implementation. We encourage the DGSRE to take the necessary steps to achieve this objective. Managers deplore the impacts that these administrative constraints have on work performance and their inability to make commitments for the following year, since such commitments can be made only after annual budgets are approved. Auditor General of the Ville de Montr?al 216 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) In closing, all project postponements and cancellations have serious consequences. Not only do they affect project implementation rates for all stakeholders, they entail additional costs as a result of mitigation measures that must be adopted, sometimes for long periods. Furthermore, planning these projects in future years will once again require project coordination with partners. This problematic situation has not been resolved because it was postponed, leading to the deterioration of assets, risks of system breakdown for the population served and public safety risks. Finally, postponed responses are likely to be more costly in the future. In our view, determination and analysis of the causes and consequences of project postponements and cancellations and the evaluation of costs arising therefrom constitute important management information. The DGSRE does not conduct such analyses in a structured way even though they would be useful for supporting accountability reporting on postponement of projects considered high-priority, given the size of the investment deficit and the extent of investments that will have to be made over the next few years. 3.6.1.B. Recommendation We recommend that the Direction de la gestion strat?gique des r?seaux d'eau develop tools to provide it with up-to-date information on the progress status of investment projects included in partnership agreements so that it can rigorously follow up projects planned and find solutions to irritants that delay project implementation. In particular, this information should include: o the progress of investment projects; o the causes and consequences of postponed and cancelled projects; o future costs of such postponements and cancellations; o temporary measures put in place to ensure public safety. Business unit's response: [TRANSLATION] Ensure follow-up on the degree of progress of investment projects provided for in partnership agreements. Produce a project follow-up document as soon as a project is analyzed. A project follow-up document will be created, both for projects retained and for projects not retained during the planning stage. This document will also serve to document the causes and consequences of postponed and cancelled projects, future costs of such postponements and cancellations, and temporary measures adopted to ensure public safety. (Planned completion: December 2013) Auditor General of the Ville de Montr?al 217 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.6.1.C. Recommendation We recommend that the Direction de la gestion strat?gique des r?seaux d'eau draw up a list of replacement projects when unforeseen events force it to postpone or cancel projects that were originally provided for in partnership agreements so that it will be able to make adequate use of the budgets allocated to it for asset maintenance. Business unit's response: [TRANSLATION] The DGSRE is busy drawing up a plan for the three years ahead so that it will have the flexibility it needs to achieve its mission. (Planned completion: December 2014) 3.6.2. Maintenance Spending 3.6.2.A. Background and Findings In 2009, as part of the contract awarded to the consortium of consulting engineering firms, the DGSRE requested that a State of the Infrastructure Report be produced, as part of the RP development, in accordance with a procedure recommended by good practices. 24 The work consisted in compiling inventory data on secondary systems (from the RP) and in obtaining a qualitative evaluation of the condition of these groups of assets from managers of the systems. As a result of this procedure, a state of the infrastructure report on secondary systems was produced for each borough and for the city as a whole. It included the following objectives: o Provide the city with a descriptive overview of the condition of the systems for each borough; o Define additional investment needs for maintenance required by each borough in order to attain required levels of service; o Make managers aware of results from the RP. In keeping with this procedure, a list of activities required for maintenance of the various secondary system assets was established, and the response frequency associated with each activity was determined. These maintenance activities had to apply to all boroughs for all assets. Accordingly, a level of maintenance expenses was established for each borough. 24 "Developing a Water Distribution Renewal Plan," National Guide to Sustainable Municipal Infrastructure, InfraGuide, November 2003. Auditor General of the Ville de Montr?al 218 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) Good practices define water and sewer system maintenance as an essential need that must be accompanied by response planning and funding. The establishment of maintenance activities increases the likelihood that lines that are not yet deteriorated will reach their probable maximum useful life, thereby preventing premature deterioration of the system. Conversely, deficient routine maintenance could potentially lead to greater investments for asset renewal. According to the 2009 State of the Infrastructure Report, the funding required to implement the annual maintenance program was evaluated using average unit costs determined for each maintenance activity selected. The theoretical costs of annual maintenance were $68.3 million, including $36.8 million for water system maintenance and $31.5 million for sewer system maintenance (see Table 21). Table 21 - Theoretical Annual Costs of Maintaining Secondary Water and Sewer System Assets for all Boroughs (in millions of dollars) Type of asset Theoretical annual maintenance costs Asset Water mains 11.9 Valves and valve chambers 12.5 Fire hydrants and isolation valves 6.1 Water main service connections and curb stops Water system 6.3 Subtotal Sanitary, storm and combined sewer lines Sewer system Manholes 36.8 17.0 3.9 Sumps and drains 10.6 Subtotal 31.5 Total 68.3 Source: City of Montreal State of the Infrastructure Report for Water Distribution and Wastewater Collection Systems. During our audit, we examined the extent to which budgetary resources were allocated to boroughs to enable them to assume the responsibilities delegated to them. We then assessed the DGSRE's level of knowledge of the maintenance activities performed by boroughs and whether they were in line with expectations. To do this, we compared the theoretical maintenance costs ($68.3 million), set out in the 2009 State of the Infrastructure Report, with annual budgets for maintenance activities (boroughs and DGSRE). For the purpose of this comparison, presented in Table 22, we Auditor General of the Ville de Montr?al 219 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit took into account actual spending since 2006. This comparative evaluation revealed that actual maintenance fell considerably short of the level required, based on standard costs estimated at $68.3 million per year. Table 22 - Actual Spending for Maintenance of Water and Sewer Systems for all Boroughs (in millions of dollars) Year Expenses assumed by the DGSRE 2006 2007 2008 2009 2010 2011 Total 3.7 5.9 7.7 8.7 8.9 11.2 46.1 2012 2013 2014 15.9 17.6 19.6 Total the boroughs Actual spending 32.7 36.4 33.0 38.9 31.2 38.9 32.8 41.5 32.8 41.7 34.8 46.0 197.3 243.4 Projections 36.1 52.0 36.1 53.7 36.1 55.7 % of the Objective spending objective 68.3 68.3 68.3 68.3 68.3 68.3 409.8 53 57 57 61 61 67 59 68.3 68.3 68.3 76 79 82 Source: DGSRE presentations: "R?trospective de l'ann?e 2011" and "Objectifs 2012." Figure 4 gives a visual breakdown of the accumulation, from 2006 to 2011, of the total maintenance deficit of $166.4 million, which is the difference between budgets allocated to boroughs for their maintenance activities ($243.4 million) and financial needs (types and frequency of activities) ($409.8 million). This maintenance deficit accounts for 41% (100% - 59%) of the additional amounts that should have been invested according to the theoretical annual maintenance costs presented in the 2009 State of the Infrastructure Report. Auditor General of the Ville de Montr?al 220 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) Figure 4 - Theoretical Needs (Target) Compared to Actual Maintenance Spending for Secondary Water and Sewer Systems (2006-2014) in $ millions 70 60 50 40 30 20 10 0 2006 2007 2008 Maintenance - boroughs 2009 2010 Maintenance - DGSRE 2011 Projection 2012 Maintenance - total Projection 2013 Projection 2014 Target Source: City of Montreal State of the Infrastructure Report for Water Distribution and Wastewater Collection Systems and DGSRE presentations: "R?trospective de l'ann?e 2011" and "Objectifs 2012." In view of the crucial role that system maintenance plays in the useful life of assets, it is essential for the DGSRE to know the type and frequency of current maintenance activities achieved with operating budgets. Any maintenance activities that are not carried out must also be identified so that the consequences can be assessed. According to the information obtained, the few boroughs whose systems are in good condition devote their resources to maintenance activities and even to refining their activities, often through new techniques designed to protect assets. Conversely, most boroughs whose systems are among the oldest and in the most deteriorated condition or for which the necessary responses are the most complex operate more in reactive mode and do not manage to carry out all routine maintenance work. Under the partnership agreements, the DGSRE sets targets for each borough every year, based on the number of inspections to be conducted on different assets, the number of Auditor General of the Ville de Montr?al 221 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit assets to be replaced (e.g., fire hydrants, valves) or the territory to be covered for leak detection. Throughout the year, each borough must complete a monthly "targets table." In looking over these tables, we noted that they provide information on whether each borough reached the targets set (e.g., percentage of the territory covered to detect leaks, number of fire hydrants inspected). Other items include the number of repairs carried out on different assets and the amounts spent every month to carry out all maintenance activities. Follow-up performed with the aid of this targets table, combined with monthly meetings with boroughs, enables the DGSRE to detect problems and propose corrective action, if necessary (e.g., additional financial support through the DGSRE operating budget, institutional procedures to make up for the lack of human resources for leak detection). Thanks to this follow-up, the DGSRE noted a disparity in the methods used by boroughs. For example, for some boroughs, inspecting an asset means simply confirming its existence, while, for others, it means conducting tests and repairing or replacing the asset, if necessary. According to the information received, this situation can be explained by the difference in the level of qualifications in the workforce, by the lack of resources in some sectors and by the lack of rigour in the execution of certain maintenance activities. Every year, the DGSRE completes the targets table by showing the standard cost of activities performed by boroughs. The comparison shows significant variances between this standard cost and what boroughs actually spent (see Table 23). For 2010 and 2011, we note that the amounts that boroughs actually spent for maintenance activities were higher than the average expected cost for the execution of those same activities. For Ville-Marie and Le Plateau-Mont-Royal boroughs, we note that variances are larger than for the boroughs as a whole. Such a situation contributes to the annual maintenance deficit. Auditor General of the Ville de Montr?al 222 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) Table 23 - Comparison of Maintenance Costs for Activities Performed by Boroughs (in thousands of dollars) 2010 Borough The whole city 2011 Standard Actual Overrun Standard costa cost (actual/standard) costa 28,573 33,336 17% 34,552 Actual cost Overrrun (actual/standard) 40,665 18% Ville-Marie 2,663 4,043 52% 3,242 5,565 72% Le Plateau-Mont-Royal 1,197 2,166 81% 1,360 2,686 98% a Established according to the annual theoretical maintenance costs appearing in the 2009 State of the Infrastructure Report. Source: 2010 and 2011 Targets Tables, DGSRE. According to the information obtained, it is not possible for the DGSRE to know with certainty the reasons for this situation, mainly because of the accounting methods used by boroughs for their maintenance activities. While it is important to take the accumulated maintenance deficit into account when future maintenance expense needs are established, we think that an analysis of boroughs' maintenance activities must also be carried out to determine what they actually accomplish with these expenditures, in order to justify variances between these and the average cost of activities. During our audit, we also examined the by-laws under which powers are delegated to borough councils for maintenance of secondary systems. This delegation of powers is set out in the general by-law entitled By-law concerning the delegation of city council powers to borough councils (02-002). This by-law is not specifically aimed at the maintenance of secondary water and sewer systems, unlike the By-law concerning the sub-delegation to borough councils of the maintenance of waterworks and sewer systems (05-090) or the Bylaw concerning the delegation to borough councils of certain powers relating to the arterial road system (08-055). Accordingly, the delegation of powers associated with maintenance activities for secondary systems is not accompanied by specific conditions, such as: o Performing maintenance activities in accordance with instructions in the maintenance guide accompanying the by-law; o Keeping separate accounting for the system's maintenance activities so as to provide a full follow-up of related activities; o Producing reports on the delegated activities. Such a situation could explain in large part the disparity among boroughs in the performance of maintenance activities. We think it would be beneficial for the delegation of powers for the secondary water and sewer systems to be covered by a separate delegation-of-powers by-law. Under such a by-law, conditions could be imposed on the performance of delegated activities, just as they are for maintenance of the primary water Auditor General of the Ville de Montr?al 223 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit and sewer systems and maintenance of the arterial road system. This would help standardize the execution of maintenance activities, facilitate comparisons among boroughs and thereby better equip the DGSRE to perform thorough follow-up on delegated activities. 3.6.2.B. Recommendation We recommend that the Direction de la gestion strat?gique des r?seaux d'eau periodically evaluate the boroughs ' accumulated maintenance deficit for the secondary water and sewer systems so that it can decide on steps to be taken in order to comply with the comprehensive response strategy that was developed. Business unit's response: [TRANSLATION] Periodically define the variance between maintenance activities (in accordance with best practices) already identified in the 2009 City of Montreal State of the Infrastructure Report for Water Distribution and Wastewater Collection Systems and the activities carried out in boroughs. (Planned completion: March 2014) 3.6.2.C. Recommendation We recommend that the Service de l'eau take the necessary steps to ensure that city council adopts a separate delegation-of-powers by-law for boroughs concerning maintenance activities for secondary water and sewer systems so that they are standardized, thereby facilitating follow-up by the Direction de la gestion strat?gique des r?seaux d'eau so that assets may be preserved in accordance with the comprehensive response strategy. This delegation of powers should be accompanied by specific conditions governing the execution of maintenance activities (e.g., maintenance guide, separate accounting and accountability reporting). Business unit's response: [TRANSLATION] The DGSRE, in collaboration with the Service des affaires juridiques et de l'?valuation fonci?re and the boroughs, will develop a separate delegation-ofpowers by-law for boroughs regarding maintenance activities carried out on secondary water and sewer systems in order to have it adopted by city council. (Planned completion: December 2015) Auditor General of the Ville de Montr?al 224 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) 3.7. Accountability Reporting 3.7.A. Background and Findings In order to ensure that the Direction g?n?rale and authorities can stay informed about the progress of the master plan for the management of secondary system assets, in terms of both renewal and maintenance, accountability mechanisms must be introduced. To do this, management reports containing relevant information must be produced periodically so that it is possible to assess changes in the condition of assets. These reports must be useful for informed decision-making. We examined the extent to which the DGSRE reports to the Direction g?n?rale on the progress of the master plan for the management of secondary water system assets as it pertains to renewal and maintenance and the extent to which boroughs account for powers delegated to them. While perusing partnership agreements with boroughs, we noted that the DGSRE made a commitment to submit an annual report on achievements for all 19 boroughs to the municipal administration, and to report to it in the course of the fiscal year as needed. According to the information obtained, however, no such reports highlighting annual achievements have been produced to date. We think that the DGSRE should report on an annual basis to the Direction g?n?rale on the implementation of investment projects provided for in partnership agreements. This accountability reporting should also cite project postponements and cancellations and describe the consequences associated with them. We also noted that the Service de l'eau gave several presentations to make authorities aware of the scale of investments that will be needed in coming years to eliminate the accumulated investment deficit and to preserve assets. Since 2002, the amounts required have been rising steadily as a result of better knowledge of the systems and their needs. However, reports showing the progress of the master plan, in terms of both renewal and maintenance, were not produced since the final RP was produced in May 2010. In view of the high direct and indirect costs incurred by the city for developing the RP, for updating the information it contains (e.g., the results of sewer system inspections) and for maintenance activities carried out by boroughs, we think that the Service de l'eau should report periodically to the Direction g?n?rale and to authorities on the progress of this master plan. The Service de l'eau should also clearly report any variances between investments and maintenance expenses required under the strategy established and actual spending, and it should produce scenarios accordingly. The reporting should also include an assessment of Auditor General of the Ville de Montr?al 225 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit the funding method obtained, especially as it pertains to grants contributed by various partners. Of course, the reporting should also establish correlations with the overall condition of secondary system assets, taking into account renewal and maintenance expenditures, so that any necessary adjustments can be made. 3.7.B. Recommendation We recommend that the Service de l'eau periodically report to the Direction g?n?rale and authorities on the current status of secondary water and sewer system management to ensure informed decision-making with respect to expected results. In particular, this accountability report should: o Focus on the degree of implementation of investment projects provided for in partnership agreements with boroughs, including costs incurred; o Identify the status of maintenance activities that have been delegated to boroughs, including costs incurred, with respect to set targets; o Report on the progress of the master plan in terms of both renewal and maintenance, including the degree of progress made in implementing projects considered critical and relating this to the overall condition of assets; o Clearly show any variances between actual expenditures and investments and maintenance expenses required under the strategy established and draw up scenarios accordingly; o Include an evaluation of the funding method obtained, especially with respect to grants contributed by various partners. Business unit's response: [TRANSLATION] Establish structured accountability reporting that is compliant with the recommendation: o Status report on investment projects provided for in partnership agreements with boroughs. o Status report on maintenance activities (allocation of boroughs and the DGSRE). o Incorporation into the status report of the extent to which the different grant programs are used. (Planned completion: March 2014) 3.8. Overall Conclusion The first studies, conducted in 2002 by a consortium of consulting engineering firms, highlighted the deterioration of water systems. They also made the municipal administration aware that substantial investments would be required over the next few years to maintain and rehabilitate these systems. Auditor General of the Ville de Montr?al 226 2012 Annual Report 5.5. Implementation Plan for Secondary Water and Sewer System Infrastructure Work (Service de l'eau - Direction de la gestion strat?gique des r?seaux d'eau) Since then, the city has allocated substantial resources to produce a detailed inventory of the systems, assess the condition of those systems, produce an RP according to the methodology proposed by the MAMR for the city and related municipalities and use an IDAS. In 2005, a $30.7 million 10-year contract was awarded for the development of an RP for all the water and sewer systems (primary and secondary systems). In 2008, an additional $10.1 million budget was required in order to complete the work and integrate road assets. Since the RP project began, the city has also awarded contracts for the testing of secondary sewer lines totalling nearly $20 million. Finally, city employees, mainly from the Division de la g?omatique, the DGSRE and the boroughs, helped compile the data necessary for developing, updating the RP and developing institutional management tools. Over time, a better knowledge of the systems made it possible to assess investment needs to preserve secondary system assets. For this purpose, the Service de l'eau gave several presentations to make authorities aware of the large sums involved. Capital budgets allocated by the city were insufficient and as a result, the accumulated investment deficit has continued to rise. For secondary systems alone, the estimated investment deficit jumped from $0.9 billion in 2003 to $2.3 billion in 2012. Looking at the system as a whole, the deficit surged from $1.5 billion to $3.8 billion. Because of the gulf that exists between investment needs and the budgets allocated, projects to be implemented must be selected on the basis of the priorities established. When the Service de l'eau had its final RP approved by the urban agglomeration council in March 2011, it stated in its decision-making summary that this management tool was useful for rational, discerning and effective decision-making, that it gave it the ability to make informed decisions. The Service de l'eau also stated that now that this report has been submitted to the government, the city had a thorough knowledge of the condition of its infrastructures and that response planning was based on the priorities established by the MAMR and on a diagnosis of the structural and functional condition of lines. However, our audit revealed that potential responses to be implemented were not selected entirely according to either the prioritization set forth in the RP or the integrated overall condition scores. Furthermore, a large number of projects identified in the planning stage were not identified in the RP. As a result, we are not confident that the responses deployed on secondary systems were those with the highest priority. In order for RP data to be able to detect real priorities as effectively as possible, considerable, ongoing efforts must be expended to have complete information (e.g., data on system inspections, standardized compilation of data on water main repairs). Later, sustained efforts will be required to update these data to make them useful for planning. Auditor General of the Ville de Montr?al 227 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Besides the selection of responses based on priorities established by the RP, we also noted that nearly half the projects that are selected at the end of a planning process and included in partnership agreements with boroughs are not implemented in the year in which they are planned. A great many projects must be postponed or cancelled, which contributes to the accumulated investment deficit. Worse still, the amounts provided for in these agreements are well below the amounts they should correspond to in the RP established to implement short-term priority responses. We noted a similar problem concerning the maintenance of secondary systems. Over the years, low levels of actual spending in comparison to the targets established on the basis of good practices will heighten the accumulated maintenance deficit and consequently affect the level of future investments. We are aware that the final RP produced in 2010 has not yet been completed and that considerable efforts have been made to acquire management tools. However, given the substantial resources invested in this project, we think that a timeframe should be established to accelerate completion of the RP so that it can be used as a planning tool for determining and implementing virtually all short-, medium- and long-term priority responses. Finally, at present the large number of unforeseeable factors (level of service to be established, insufficient budgets allocated, uncertain method and level of funding, etc.) lead us to believe that this situation is not about to be corrected. The fact remains that the amounts currently invested are not sufficient to reach the investment level set by the Service de l'eau and to become self-financing within the time limits set by authorities. This issue is all the more important in light of the fact that a similar situation exists for all infrastructures (bridges and tunnels, water systems and road systems). The solution requires deep reflection. That is why the municipal administration must make wise choices in developing its strategy to slow and ultimately counter the deterioration of the infrastructure system. Auditor General of the Ville de Montr?al 228 2012 Annual Report Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 Implementation Plan for Arterial Road System Infrastructure Work (Service des infrastructures, du transport et de l'environnement - Direction des infrastructures) 5.6 5.6. Implementation Plan for Arterial Road System Infrastructure Work (Service des infrastructures, du transport et de l'environnement - Direction des infrastructures) Table of Contents 1. Introduction ................................................................................................................... 233 2. Audit Scope................................................................................................................... 235 3. Findings and Recommendations .................................................................................. 236 3.1. Inventory Data ....................................................................................................... 239 3.2. Comprehensive Response Strategy ..................................................................... 245 3.3. Level of Service..................................................................................................... 249 3.4. Determination of Priorities ..................................................................................... 254 3.5. Allocation and Use of Resources .......................................................................... 259 Auditor General of the Ville de Montr?al 231 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit List of Acronyms AB as built PPI pavement performance index CRCAC Centre de recherche et de contr?le appliqu? ? la construction RP response plan SGIS spatial geographic information system SITE Service des infrastructures, du transport et de l'environnement TCEP three-year capital expenditures program DGAV Division de la gestion des actifs de voirie DI Direction des infrastructures IDAS integrated decision aid system IRP integrated response plan Auditor General of the Ville de Montr?al 232 2012 Annual Report 5.6. Implementation Plan for Arterial Road System Infrastructure Work (Service des infrastructures, du transport et de l'environnement - Direction des infrastructures) 5.6. Implementation Plan for Arterial Road System Infrastructure Work (Service des infrastructures, du transport et de l'environnement - Direction des infrastructures) 1. Introduction The Act respecting the exercise of certain municipal powers in certain urban agglomerations 1 , adopted in December 2004, awarded Ville de Montr?al (the city) exclusive jurisdiction, including road maintenance and management, over thoroughfares forming the arterial system in the urban agglomeration. All other thoroughfares, those not belonging to the arterial system, are part of the local system, and they have been under the jurisdiction of borough councils since the new city was created in 2002. On June 20, 2008, the Act to amend various legislative provisions concerning Montr?al 2 modified the exclusive jurisdiction of the urban agglomeration council over thoroughfares forming the arterial road system. Subject to certain exceptions, each of the related municipalities, including Ville de Montr?al, recovered jurisdiction over the thoroughfares that form the arterial system on its own territory. In 2010, the city's road system (roads, sidewalks, curbs, manhole heads and sump heads) (local and arterial systems) included a total of 4,058 km of roads and 6,677 km of sidewalks, valued at $10.9 billion. The arterial system, which accounts for roughly 20% of the entire road system, consists of 845 km of roads and 842 km of sidewalks, and is valued at roughly $3.4 billion, or 31% of the entire road and sidewalk system. The local system consists of close to 3,213 km of roads and 5,835 km of sidewalks, and is valued at $7.5 billion, or 69% of the value of the entire road and sidewalk system. City council is responsible for decisions concerning the arterial system, which is managed by the Division de la gestion des actifs de voirie (DGAV) of the Service des infrastructures, du transport et de l'environnement (SITE). Until December 31, 2012, this division reported to the Direction des transports; since January 1, 2013, it has been under the authority of the Direction des infrastructures (DI). However, maintenance of the arterial system is handled by the boroughs. Each borough is also responsible for managing and maintaining the local system. 1 2 RSQ, chapter E-20.001. SQ, 2008, chapter 19. Auditor General of the Ville de Montr?al 233 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit City council is the main source of funding for the implementation of arterial system projects, providing 95% of the funds (2011 TCEP 3 , $39.8 million). The amount contributed by the urban agglomeration council refers to the part of the arterial system that is located downtown and accounts for 5% (2011 TCEP, $1.9 million). Every day, the arterial system is heavily used by automobile traffic. Its main function is to ensure the smooth flow of traffic, and it is connected to both the highway network, which is administered by the Minist?re des Transports du Qu?bec, and to collector streets, which are administered by boroughs. Thoroughfares usually run continuously over a long distance. In Montr?al, they are of two types. Secondary thoroughfares carry traffic volumes of between 10,000 and 30,000 vehicles a day, while main thoroughfares can carry more than 30,000 vehicles a day. A large number of responses implemented by the city's business units also have an impact on the arterial road system because they involve cutting into the pavement in different places (e.g., responses on water systems, sewer systems, electrical conduits), which accelerates the deterioration of the system. Similarly, responses implemented by public utilities also have impacts on the condition of the road system (e.g., burying electric wires, telephone and television cables, gas feed lines). Various documents produced by the Direction des transports describe the current condition of the road system as follows: o Aging system, much of which has already reached the end of its useful life and in some cases even exceeded its useful life (a large percentage of the arterial system was built or rebuilt before 1960); o Condition of the system rapidly deteriorating; o Heavy demands placed on the system by high volumes of traffic, often greater than what it was designed to carry when it was planned more than 50 years ago; o Many roads and sidewalks already in poor condition and the subject of numerous complaints. To assess the extent of deterioration, the city inspects its systems regularly, which enables it to respond to emergencies and plan the responses required to extend the useful life of assets and maintain them in satisfactory condition. In the current context the planning process is especially important, because the city is running substantial deficits in the maintenance and rehabilitation of roads and sidewalks. In 3 Three-year capital expenditures program. Auditor General of the Ville de Montr?al 234 2012 Annual Report 5.6. Implementation Plan for Arterial Road System Infrastructure Work (Service des infrastructures, du transport et de l'environnement - Direction des infrastructures) 2010, the Direction des transports estimated that $326 million in investments would be needed every year for the next 10 years to maintain roads and sidewalks in their current condition. The budget for infrastructure repair on the arterial road system remains insufficient to meet all needs. Stakeholders think these assets will tend to deteriorate as the years go by. It is imperative that the right choices be made and that the effects of the investments made are maximized. The challenge faced during planning is to determine, as efficiently as possible, the work to be undertaken, to decide on the best time for implementing those responses based on the life cycles of roads and sidewalks, to take into consideration responses to be deployed on water assets while maintaining service for users, all within a limited budget. The DI, in collaboration with the Service de l'eau, is currently finalizing an integrated response plan (IRP) (water, sewer and road) designed to help prioritize work. Since major water system projects have a direct impact on the road system, such work requires effective coordination among the various business units responsible. To this end, an integrated decision aid system (IDAS) helps identify suggested responses, taking into account the condition of systems. 2. Audit Scope Our audit focused on the implementation of infrastructure work. The goal was to ensure that responses deployed on the city's infrastructures resulted from priorities established. In view of the substantial investments that will be required in coming years, we targeted infrastructures in three sectors of activity: o Bridges, tunnels and related structures; o The infrastructures of secondary water and sewer systems; o The arterial road system. In the first phase, an audit report on bridges, tunnels and related structures was produced in March 2011. In the second phase, an audit report on secondary water and sewer systems was produced in January 2013. In the third and final phase, this audit report on the arterial road system (roads and sidewalks) focuses on responses implemented by the city to maintain and renew this system. These responses are strategically important because they are concerned with the safe, efficient transportation of people and goods. To do this, we reviewed the planning process implemented by the DGAV. This process includes the inventory, assessment of the condition and determination and prioritization of Auditor General of the Ville de Montr?al 235 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit investment needs. We were also interested in the establishment of a level of service and the planning component that consists in coordinating, organizing and allocating the necessary funds to response implementation. Our audit focused mainly on data from 2010 and 2011, but we also took into account information from previous years and, when the situation required it, from the year 2012. 3. Findings and Recommendations The DGAV of the Direction des transports established a project planning process for the arterial road system. At the time of our audit, some aspects of this process were either being implemented, or were about to be, as a result of the integration of the road system (arterial and local) with the response plan (RP) for water systems. We took this into account in conducting this audit. The city is currently coming to grips with an aging road system in an increasingly dilapidated state, with the result that significantly higher levels of investment are required to renew and maintain the assets in this system. With limited budgets available to meet these investment needs, they are in direct competition with the priorities of the city's other sectors of activity. It is therefore necessary to practise sound management of road assets. Before addressing each of the improvements that need to be made to the different steps in the planning process and monitoring, we will first discuss the distribution of jurisdictions and responsibilities for the road system (arterial and local systems), which will facilitate an understanding of the various sections of this report. Distribution of Jurisdictions Jurisdictions and responsibilities for the road system are distributed among several of the city's administrative units and authorities. From the time the new Ville de Montr?al was created (on January 1, 2002), until the Act to amend various legislative provisions concerning Montr?al came into force, the responsibility for asset maintenance and renewal activities on the Island of Montr?al's road system was divided into two major categories: o Jurisdiction over the arterial road and sidewalk system was granted to the city or related municipalities, and to the urban agglomeration for the downtown area; o Jurisdiction over the local road and sidewalk system was granted to respective boroughs or to related municipalities, as the case may be. Auditor General of the Ville de Montr?al 236 2012 Annual Report 5.6. Implementation Plan for Arterial Road System Infrastructure Work (Service des infrastructures, du transport et de l'environnement - Direction des infrastructures) With regard to the city specifically, responsibility for maintenance activities (regular and preventive) on the arterial road and sidewalk system assets was delegated to each borough council through the By-law concerning the delegation to borough councils of certain powers relating to the arterial road system (08-055). Although the distribution of powers has changed since the new city was created, Montr?al's arterial system was developed in 2001 when the new city was created, then adopted by bylaw (02-003) 4 on January 1, 2002. The streets and roads forming the arterial road system are shown on a map, while all other streets and roads form the local road system. The division of road systems into arterial and local is based on a classification of the road system that the Minist?re des Transports du Qu?bec developed in collaboration with the Montr?al urban community in August 2000. In 2006 and 2007, the Commission permanente du conseil municipal sur le transport, la gestion des infrastructures et l'environnement held hearings on the review of the arterial road system. The goal of this review was to reclassify certain pavement sections in order to ensure consistency in transportation management and to define an investment and response plan for those sections. In particular, the SITE had proposed to integrate collector streets of the local road system into the arterial road system. A bus route ran along those collector streets. Following hearings, authorities decided that a reclassification of collector streets would not be part of the review of the arterial system. As a result, the legal framework remained unchanged: collector streets remained part of the local road system and boroughs were still responsible for carrying out all repair work required on those streets. However, to assist the boroughs and accelerate the work of upgrading the local road system, in 2007 the city council approved a program for repairing the local system's collector streets that covered both the cost of performing the work and professional design and monitoring services. To do this, it mandated the SITE to implement this program and informed the boroughs that the SITE would take responsibility for execution of the work, pursuant to section 85 5 of the Charter of Ville de Montr?al. City council had decided that since this was a non-recurring program, there was no guarantee of continuity of funding from one year to the next; nevertheless, work was carried out on collector streets under the program in 2007 and 2009. The program was interrupted in 2010 and 2011, then city council renewed it in 2012, leaving the choice of collector streets up to the boroughs. 4 5 By-law concerning the arterial and local road systems, city council, January 1, 2002. This section stipulates that "The city council may, subject to the conditions it determines, provide a borough council with a service related to a jurisdiction of the borough council; the resolution of the city council shall take effect on passage by the borough council of a resolution accepting the provision of services." Auditor General of the Ville de Montr?al 237 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit According to the information obtained, the concept of collector streets was, however, more of a subjective one. As shown in Table 1, the distribution of responsibilities was based on the jurisdictions prescribed by law, municipal by-laws or decisions made by authorities. Table 1 - Distribution of Responsibilities for Road System Management and Budget Allocations Investments Maintenance a Inspections Responsibility Capital budget Responsibility Operating budget Responsibility Local system Arterial system a b c d e Direction des travaux publics b of the boroughs DI d of the city Direction des travaux publics of the boroughs Direction des travaux publics e of the boroughs Direction des travaux publics of the boroughs Direction des transports of the city Operating budget Before 2010: Direction des travaux publics of the boroughs 2010 and 2011: c Service de l'eau Before 2010: Direction des transports of the city 2010 and 2011: c Service de l'eau Maintenance: includes regular and preventive maintenance. Except for a program for repairing collector streets in the local system, approved by city council for 2007, 2009 and 2012. The Direction des transports of the city had been entrusted with the management of this program. Under the IRP project for water, sewer and road systems. The DGAV reported to the Direction des transports until December 31, 2012. Responsibility delegated by city council in accordance with by-law 08-055. In short, regular and preventive maintenance carried out on the arterial and local road systems is under the responsibility of the boroughs, each of which receives an operating budget for these activities. Rehabilitation and reconstruction responses for these assets are under the purview of the DGAV of the Direction des transports for the arterial system and of the Direction des travaux publics des arrondissements for the local system, except for collector streets for the years in which the program was approved by city council. In view of these jurisdictions and responsibilities for the road system, we assessed the extent to which work done on the city's arterial road system resulted from the priorities established. To do this, we first examined the inventory in the planning process, then examined the comprehensive response strategy in place, the level of service decided upon, the determination of planning priorities for 2010 and 2011, and, finally, the allocation and use of resources. Auditor General of the Ville de Montr?al 238 2012 Annual Report 5.6. Implementation Plan for Arterial Road System Infrastructure Work (Service des infrastructures, du transport et de l'environnement - Direction des infrastructures) 3.1. Inventory Data 3.1.A. Background and Findings Optimal planning of arterial road rehabilitation and reconstruction responses is based primarily on an overview of the system. To achieve this, managers must be able to rely on a complete, up-to-date inventory. One of the objectives of the Politique des ?quipements et des infrastructures, approved by the city manager in January 2009, was [TRANSLATION] "to compile an inventory of equipment and infrastructure in order to determine the extent, quality and condition of its property holdings." This objective is also in line with the good practices set out in the National Guide to Sustainable Municipal Infrastructure, 6 which states that "Pavement inventory is the key building block for pavement decision making." The first step in developing a road system inventory is to divide it into sections 7 that correspond to basic units. This inventory is completed by physical and status data. According to the National Guide to Sustainable Municipal Infrastructure, a road and sidewalk inventory should include: o The location of the road, roadway class, 8 length, width and surface area of the pavement section; o The date of the original construction and the date(s) of any subsequent rehabilitation treatments; o A description of the original pavement structure and the subsequent pavement preservation treatments; o Pavement condition (past and current); o Traffic data (e.g., estimated annual average daily traffic and the percentage of commercial vehicles). At the time of our audit, the DGAV had a computerized road management system (produced by the company CRCAC 9 ), which was put into operation in 2004 to maintain the arterial road system inventory. This system used a dedicated database containing physical inventory data as well as a pavement condition evaluation represented by the PPI. 10 It was also a geographic visual aid in the form of a map that showed all 8,500 sections of the arterial system, and it contained related data (e.g., test data, type of responses implemented) for each section. 6 7 8 9 10 InfraGuide. A section is a segment of a street, usually from one intersection to the next. For example, a thoroughfare, a collector street, a residential street. Centre de recherche et de contr?le appliqu? ? la construction. Pavement performance index. This index results from the combined analysis of three road condition indicators measured by taking readings on all sections. The indicators are the road surface condition, the degree of comfort in driving and the degree of rutting. Auditor General of the Ville de Montr?al 239 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Some physical inventory data, however, were not found in the database: o The date of original construction of a section and the date(s) of any subsequent rehabilitation treatments--according to the information obtained, data prior to 2003, which would nevertheless be useful for determining the useful life of the asset, were not in the system and could not be found in a hard-copy format; o A description of the original pavement structure and types of subsequent rehabilitations--data prior to 2003 had not been entered; o Traffic data. Tests conducted in 2006 were the source of data used to describe road conditions, or PPIs. A complete test of the system was conducted according to a four-year cycle. Within this period, status data were updated by the CRCAC system to account for investments made and aging roads and sidewalks. According to the information obtained, data were no longer considered reliable enough, after this four-year period, to be used as the main indicator for selecting projects to be implemented. Essentially, data compiled in the CRCAC application could be used to produce a list of sections based on their condition. At the time of our audit, test data were no longer up to date; nevertheless, Table 2 shows the last profile of the condition of the arterial road system, produced in 2010 by the DGAV. This profile is based on a field survey dating from 2006 and for which results were updated to take into account investments made since then and the aging of roads and sidewalks over time. Table 2 - Condition of the Arterial Road System - 2010 PPI 0 - 19 20 - 39 40 - 59 60 - 79 80 - 100 a Condition Very poor Poor Fair Good Very good Totala No. of km 46,7 118,5 230,8 249,0 171,7 816,7 % 6 15 28 30 21 100 As a result of the development of the IRP, it was determined that the length of the arterial road system is 845 km. In view of the urban agglomeration council's decision (August 2008) to include road assets in the water system response plan in order to establish the IRP for water, sewer and road systems, the DGAV has not made any efforts to keep the CRCAC application up to date. Auditor General of the Ville de Montr?al 240 2012 Annual Report 5.6. Implementation Plan for Arterial Road System Infrastructure Work (Service des infrastructures, du transport et de l'environnement - Direction des infrastructures) As part of the work of developing the IRP, a project managed by the RP project team of the Service de l'eau, the integration of road assets had to involve both the arterial and local systems. To achieve this, two contracts were awarded following a public tender process, so that new test data would be available for roads and sidewalks. An initial one-year contract of $1 million was awarded to a firm in 2009 to conduct tests on the local system. Later, in 2010, another one-year contract of $1.1 million was awarded to another outside firm. Under the terms of this second contract, tests were to be conducted throughout the arterial system and on collector and local streets in some boroughs. It should be mentioned that the RP project team of the Service de l'eau was also responsible for managing these contracts. After tests were conducted, all the data were transferred into databases used for the IDAS to produce the IRP. These centralized databases include the following features: o Site of the section and borough in which it is located; o Geometry (length, width); o Type of structure (e.g., concrete slab, type of material); o Year of construction; o Useful life (reconstruction, rehabilitation); o Rehabilitation and reconstruction work; o Costs (construction, rehabilitation, stop-gap maintenance); o Status active or not (identification of sections that are closed or redesigned); o Test data; o PPIs. When road system data were integrated in databases, a data validation plan was applied by the RP project team of the Service de l'eau. The validation procedures covered physical data, geomatic data and data describing the condition of roads and sidewalks. Thanks to the results of tests that were conducted, the data could be considered valid. The next step will be to maintain up-to-date data. To achieve this, it will be necessary to produce guides and procedures concerning updating, specifying the type of information required and how often it will be needed. One source of information that can be used to update physical data on the arterial system is provided by as-built (AB) plans, because they describe the composition of roads. AB plans are engineering documents produced following the construction, reconstruction or rehabilitation of an engineering or architectural work, and they take into account changes made to original construction plans. They confirm compliance with plans and specifications and are signed by the person in charge of monitoring the work. Auditor General of the Ville de Montr?al 241 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit According to the information obtained, to date, few AB plans for the arterial road system were scanned into databases. The consequence of this situation is particularly evident when the projects to be implemented are planned. In fact, the physical inventory data compiled in databases do not take into account certain types of information, such as the composition of materials of sections on which work was performed. To keep physical inventory data up to date, a process must be put in place to ensure that AB plans are taken into account in the relevant databases. For this purpose, in July 2011, the Direction g?n?rale approved a guideline called "Pr?paration et transmission des plans tels que construits / Plans TQC," which applies to all central departments and boroughs when they exercise a delegated power under a central authority. However, this guideline applies to water and sewer systems, but not road systems. The objective is as follows: [TRANSLATION] The purpose of this guideline is to optimize project costs by maintaining an up-to-date shared central data warehouse, through a spatially referenced geographic information system, of all underground public data, including subsoil on the territory, as well as to enact standards, terms and conditions to be complied with in data identification and the transmission of as-built plans (AB), response sketches and guarantees, in order to ensure data reliability, integrity and longevity. 11 According to this guideline, the Division de la g?omatique of the DI is responsible for developing terms, conditions and special procedures, for formulating appropriate recommendations to ensure access management and data updating for the spatial geographic information system (SGIS) and optimal implementation of these recommendations for all infrastructures, both existing and planned, of underground public land. One of the conditions set forth after this guideline was updated was that AB plans and response sketches must be delivered to the Division de la g?omatique within six months following provisional acceptance of the work by the manager (or authorized representative) of the unit responsible for execution of the work. The guideline also stipulates that the city manager can request accountability reports on its enforcement at any time from the Division de la g?omatique. 11 The guideline is Directive C-OG-SDO-D-11-001 entitled "Pr?paration et transmission des plans tels que construits / Plans TQC," July 15, 2011. The updated guideline came into force on October 15, 2012 (C-OGDG-D-12-011). Auditor General of the Ville de Montr?al 242 2012 Annual Report 5.6. Implementation Plan for Arterial Road System Infrastructure Work (Service des infrastructures, du transport et de l'environnement - Direction des infrastructures) According to the information obtained, the development of a guideline on the system is being studied. We think a decision should be made in this area. Another source of information that can be used for updates is test data. The test data entered in databases used for the IDAS (2010 and 2011 data) will need to be updated to be useful for decision-making. According to the information obtained, road and sidewalk tests are planned according to a three-year cycle. In closing, at the time of our audit, the DGAV had inventory data (CRCAC databases) that were no longer reliable because they were not kept up to date. As we noted, the CRCAC databases and the databases used by the IDAS contain more or less the same information. It appears, however, that important inventory data are still missing (e.g., AB plans, the history (date and type) of responses implemented prior to 2003 and traffic data). The difference lies more in the use of the databases. In fact, the IDAS is useful for identifying critical sections based on inventory data as well as for producing cost estimates for planned projects, for purposes of prioritization, planning or estimating investments required in the short, medium and long terms. At the end of our audit, specific IRPs had been submitted to the boroughs' public works directorate. However, according to the information obtained, an overall IRP should be submitted to authorities some time in 2013. Consequently, the results of this IRP can be used only for 2014 planning. According to the information obtained, another type of data not found in either the CRCAC databases or in databases used by the IDAS would be appropriate for planning purposes. That is socioeconomic data, which include characteristics of the sector served, by-laws in force, zoning, type of population, presence of institutional buildings associated with essential services and sensitive environmental areas. This relatively new concept can be used to fine-tune response planning, thereby alleviating inconveniences to the public. It would seem that most of this information could be collected from the city's business units. Finally, data on related assets, such as traffic lights and street lighting, are not part of inventory data, even if such assets are integrated into sidewalks, which are considered to be part of road systems. Such data would complete knowledge of road assets. In view of the replacement costs of these assets, it would be to the advantage of the DGAV to know this information at the time of project planning. Since inventory data are not taken into account, we think that the DGAV should produce a status report for the DI showing how this situation affects the selection of projects to be Auditor General of the Ville de Montr?al 243 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit implemented or response planning. If applicable, measures for making the necessary data available for proper response planning should be proposed. 3.1.B. Recommendation We recommend that the Division de la gestion des actifs de voirie produce a report for the Direction des infrastructures, showing: o A status report on the non-availability of inventory data during the annual planning stage; o The consequences of not having the data available; with a view to making the necessary arrangements to obtain these data and support the improvement of project planning. Business unit's response: [TRANSLATION] As part of the work of developing the IRP, all inventory data compiled were made available to all asset managers, including borough managers, by means of a cartographic display module that uses a (SGIS). This GIS was also used to enable access to all test (PPI) and photographic surveys by means of a "Viewer" application. The IRP project office also developed a guide concerning updates, specifying the type of information required and how often it will be needed. Funds were allocated to the Division de la g?omatique so that it could take on this responsibility. With respect to incomplete data, the DGAV will produce a report for the DI describing: o The current situation regarding inventory data that are useful and necessary for annual planning; o The impacts of non-availability of these data on the annual planning process and the accuracy of estimates; o Recommendations on measures that should be adopted to obtain these data, if applicable. (Planned completion: December 2013) 3.1.C. Recommendation We recommend that the Direction g?n?rale assess the appropriateness of broadening the scope of its guideline "Pr?paration et transmission des plans tels que construits / plans TQC," which came into force in July 2011 and was updated in October 2012 to include the arterial road system, so that it would place all physical inventory data at its disposal, thereby improving future responses planning. Auditor General of the Ville de Montr?al 244 2012 Annual Report 5.6. Implementation Plan for Arterial Road System Infrastructure Work (Service des infrastructures, du transport et de l'environnement - Direction des infrastructures) Business unit's response: [TRANSLATION] The Direction g?n?rale will assess the appropriateness of broadening the scope of its guideline "Pr?paration et transmission des plans tels que construits / Plans TQC," which came into force in July 2011 and was updated in October 2012, to include the arterial road system. If necessary, the Direction g?n?rale will have the guideline amended for this purpose. (Planned completion: October 2013) 3.2. Comprehensive Response Strategy 3.2.A. Background and Findings The determination of asset preservation needs consists in identifying responses considered necessary following tests, then classifying them according to their priority. In the case of the arterial road system, the types of possible responses are shown in Table 3. Table 3 - Types of Possible Responses on the Arterial Road System Response Preventive maintenance Objective and description o To prevent or slow the progressive degradation that can lead to premature deterioration. o Recurring or non-recurring responses. o For roads in good condition. o Strategy of optimal use of public funds. Budget Operating For example: sealing cracks. Current maintenance o To correct defects or deteriorations that can pose the risk of an accident or seriously compromise the comfort of users. o Minor corrective work. Operating For example: repairing potholes. o To maintain or improve the condition and avoid a much greater investment in the future when the asset has reached an advanced state of deterioration. o The appropriateness of a repair is based on its ability to extend the Rehabilitation useful life of the asset and on its cost. o Response methods and response times are generally chosen on the basis of the effectiveness and durability of the repair techniques. Capital For example: thin surfacing, planing or surfacing. Reconstruction o To restore to new condition or full functionality. Capital According to the National Guide to Sustainable Municipal Infrastructure (a reference tool for good practices), it is preferable to follow a comprehensive response strategy by reviewing all the assets of a given system. It is recommended that this strategy be followed for reconstruction, rehabilitation and maintenance responses alike. Auditor General of the Ville de Montr?al 245 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit The city's Politique des ?quipements et des infrastructures (which came into force in January 2009) runs along the same lines as these good practices: [TRANSLATION] In view of the condition of its assets, the Ville de Montr?al must assess the risk that components will break down or wear out prematurely. This evaluation will help determine the preventive and corrective measures needed in the short, medium and long terms. The analysis work must include a cost estimate, which is necessary for decision-making. This analysis should also make it possible to classify assets under one category or another, depending on whether the decision made is to: o Replace the asset; o Keep the asset, but carry out major short-term or long-term repairs; o Keep the asset and do routine maintenance on it; o Stop maintenance work on the asset, postpone repairs and conduct more indepth studies. The purpose of this classification is to make a decision for each asset as well as to set budget allocation priorities. The city's policy also recommends the [TRANSLATION] "design of a structured preventive maintenance program giving specific information on work that must be done to protect and maintain the quality of assets." There is partial compliance with the Politique des ?quipements et des infrastructures, because after tests are conducted, a score is assigned to each road and each sidewalk, providing planners with an index for the choice of responses to be implemented, in the areas of preventive or current maintenance (e.g., sealing cracks, repairing potholes), rehabilitation (e.g., planing and surfacing) or reconstruction (e.g., complete reconstruction) aimed at correcting the defects detected. However, contrary to the wording of the policy, there is no exhaustive classification of assets into categories based on the types of responses they require. Nor is there any structured preventive maintenance program as such, even if it is acknowledged as the optimal strategy for use of public funds. In fact, for assets that are in good condition, this prevents them from deteriorating prematurely and pushes back the time when more substantial investments will be necessary. With respect to the arterial road system, the fact that responsibilities are distributed among the DGAV and the 19 boroughs is not helpful in implementing a comprehensive response strategy. As we mentioned above, the DGAV is responsible for deciding on responses that can be considered investments (major repair and rebuilding projects), while the boroughs Auditor General of the Ville de Montr?al 246 2012 Annual Report 5.6. Implementation Plan for Arterial Road System Infrastructure Work (Service des infrastructures, du transport et de l'environnement - Direction des infrastructures) are independently responsible for anticipating requirements that are covered under the operating budget (current and preventive maintenance work, and certain minor repairs). The division of responsibilities, with boroughs in charge of operating budgets and the DGAV in charge of capital budgets, is not either conducive to integrated planning. It is important that a borough handle preventive maintenance through its operating budget, but this may be less obvious when the consequences of failure to do preventive maintenance are reflected in the DGAV's capital budget. The consequences are serious, since, according to the National Guide to Sustainable Municipal Infrastructure, $1 of timely prevention will delay the requirement to spend $5 of rehabilitation. In short, asset preservation needs and priority planning are not determined according to a comprehensive response strategy, contrary to the process recommended in: ? The National Guide to Sustainable Municipal Infrastructure; ? Management practices prescribed in the city's Politique des ?quipements et des infrastructures. In this context, planners (DGAV and boroughs) are not assured of selecting the right responses, on the right arterial road system assets, in a timely manner, and in such a way as to optimize public spending. By-law 08-055 adopted by city council in December 2008, which concerns the delegation to borough councils of certain powers relating to the arterial road system, could have promoted a more comprehensive approach to the planning process. We noted deficiencies in the areas of preventive maintenance execution and accountability reporting. First, the by-law refers in particular to maintenance activities that must be carried out in accordance with the prescriptions of an appended maintenance guide. While an example is cited in the area of preventive maintenance (sealing cracks), no activity of this type was performed by Le Plateau-Mont-Royal or Ville-Marie boroughs. In practice, regular maintenance work is occasionally considered as preventive maintenance work (e.g., filling holes in the road). The National Guide to Sustainable Municipal Infrastructure defines preventive maintenance as follows: "A treatment performed to prevent premature deterioration of the pavement or to retard the progression of pavement defects. The objective is to slow down the rate of pavement deterioration [...]." According to the information obtained, since the arterial road system inventory is managed by the DGAV, boroughs do not have the technical means to decide on appropriate types of Auditor General of the Ville de Montr?al 247 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit responses and materials or to determine favourable times for starting road preservation activities (sealing cracks, thin surfacing, etc.). Second, under section 6 of by-law 08-055, the borough council must provide the executive committee and the assistant general director of the infrastructures (designated as the senior manager of the SITE in the current structure) with a report on March 15 and another one on November 15 of every year. This report must review the execution of delegated activities and include technical information used to develop production indicators for management and maintenance of the arterial system. This section of the by-law appears never to have been enforced, as no report has been produced since it came into force. In our opinion, the DGAV could use this report as a management tool in its planning process, to keep track of the maintenance activities carried out by boroughs on the arterial system on their respective territories. 3.2.B. Recommendation We recommend that the Direction des infrastructures, in collaboration with the boroughs: o Develop a comprehensive response strategy for determining and prioritizing needs in the area of preservation of arterial road system assets that integrates maintenance (preventive and current), repair and major rebuilding work for the purpose of maintaining road assets in a desirable condition at the best possible cost; o Design, document and implement a structured preventive maintenance program for arterial road system assets in accordance with the Politique des ?quipements et des infrastructures, which came into force in January 2009, for the purpose of preventing premature deterioration. Business unit's response: [TRANSLATION] As part of the work of developing the IRP, the DGAV conducted tests on the arterial road system in 2011, in order to have updated data (PPI) for the purpose of establishing response priorities for the IRP (capital budgets). However, current and preventive maintenance responses (operating budgets) are left to the discretion of each borough. With respect to asset preservation needs, the DI will develop: o A comprehensive response strategy that will identify the types of responses required based on the PPI score. Since these data will be identified for all sections of the arterial road system, sections can be classified according to their categories, based on the types of responses they require; Auditor General of the Ville de Montr?al 248 2012 Annual Report 5.6. Implementation Plan for Arterial Road System Infrastructure Work (Service des infrastructures, du transport et de l'environnement - Direction des infrastructures) o A proposal to amend Schedule B of by-law 08-055 to be submitted to the Direction g?n?rale. This amendment to the "Voirie" section of the Technical Guide (available in French only) will define a structured preventive maintenance program that must be applied to all sections covered by the comprehensive response strategy. This program will be based chiefly on maintenance activities already identified in the Bilan d'?tat global des actifs de voirie 2011. (Planned completion: September 2014) Remark: The proposal to amend Schedule B of by-law 08-055 must take into account recommendations that could arise from the process of reviewing amounts allocated to boroughs in connection with possible performance criteria. 3.2.C. Recommendation We recommend that the Direction g?n?rale ensure that boroughs comply with the requirements of the By-law concerning the delegation to borough councils of certain powers relating to the arterial road system (08-055) in order to produce the information required for road system asset maintenance for the purpose of developing the comprehensive response strategy for the Direction des infrastructures. Business unit's response: [TRANSLATION] The Direction g?n?rale will assess the appropriateness of submitting to elected officials, for approval, the proposal to amend Schedule B of by-law 08055 to include implementation of a structured preventive maintenance program. The Direction g?n?rale will develop and implement the control measures required to ensure that boroughs comply with the requirements of by-law 08-055 in its present form or pursuant to the amendments that will be proposed. These control measures will be completed by: o Training that focuses on reviewing the objectives of the program; o A guide on gathering requested information; o A procedure for reminding boroughs. (Planned completion: September 2014) 3.3. Level of Service 3.3.A. Background and Findings According to the National Guide to Sustainable Municipal Infrastructure the desired level of service for the condition of roads and sidewalks must be established as a priority, before projects are selected and implementation is prioritized. However, the determination of a level of service is based on an assessment of the condition of the system and its use, the financial resources available, risks associated with the deterioration of the system, the consequences of underfunding and sources of funding. To do this, various scenarios must Auditor General of the Ville de Montr?al 249 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit be proposed so that informed decisions are made. For instance, in order to have x% of roads and sidewalks in good condition, investments of $y are needed. If all the resources required are to be channelled towards reaching this targeted condition, it is imperative that authorities make a clear commitment regarding a level of service. Whatever choice is made, there must be a strong correlation between the desired level of service and the level of long-term investment established. During our audit, we examined the extent to which the level of service had been determined for the arterial road system and whether it had been the subject of presentations made to authorities and their approval. First, in the June 2005 version of the Plan de transport, the SITE reported that an annual investment of $220 million was required to maintain the condition of road systems (local and arterial). Later, in a new version of its Plan de transport, developed in 2008, the Direction des transports outlined general orientations that it intended to adopt in the coming years. These were intended to [TRANSLATION] "restore and maintain the road system (arterial, local, structures and other components)." We think that the Direction des transports should have specified what is meant by "restore," in light of the previous version of the Plan de transport, in which it was concerned with maintaining road systems. Then, in 2010, the Direction g?n?rale launched a project for developing a 10-year investment plan. The mandate of this project, which targeted all business units, including the DGAV, was to establish a planning process for developing: o The RPs necessary for the maintenance and development of their assets over a 10year period; o Investment management policies, to ensure, among other things, maintenance of the condition of assets at an optimum level. To meet the demand, the Direction des transports produced a document entitled "Planification ? long terme des investissements," which it submitted to joint commissions of the Comit? sectoriel - Infrastructures routi?res in September 2010. The purpose of this presentation was to make elected officials aware of changes in the foreseeable condition of road infrastructures over a 10-year period according to various investment scenarios. All the investments were for the city's road systems (arterial and local), including road assets (roads and sidewalks), structures (bridges, viaducts and tunnels) and the operating system (traffic lights and street lighting). With respect to road assets, Figure 1 shows the impact of Auditor General of the Ville de Montr?al 250 2012 Annual Report 5.6. Implementation Plan for Arterial Road System Infrastructure Work (Service des infrastructures, du transport et de l'environnement - Direction des infrastructures) various levels of investment on the proportion of roads and sidewalks (arterial and local systems) in good condition over a 10-year period. Figure 1 - Impact of Different Levels of Investment on the Percentage of Roads and Sidewalks in Good Condition Levels of service Rehabilitation (68%) Maintenance (53%) Minimum responses required (41%) Borrowing limit proposed as a scenario (33%) Note: As a result of the development of the IRP, it was determined that the length of the arterial road system is 845 km in length. Source: "Planification ? long terme des investments," document submitted to the Comit? sectoriel - Infrastructures routi?res (September 2010). In September 2010 the Direction des transports estimated that annual investments of about $326 million would be required for the next 10 years to keep roads and sidewalks (arterial and local systems) in their 2010 condition. A comparison of this figure with projected investments that appeared in the 2005 Plan de transport ($220 million) five years earlier shows an annual increase of $106 million, or 48%. We were not able to assess the extent to which the variation in the investment ($326 million - $220 million) required to maintain roads and sidewalks was due to deterioration of the condition of roads and sidewalks over the past five years. Since our audit focused more specifically on the arterial road system, we reconciled the amounts invested in 2010 and 2011 with the levels of investment required to maintain the condition, to implement minimum responses or to align with the proposed borrowing limit. Table 4 shows the percentage of targets reached. Auditor General of the Ville de Montr?al 251 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Table 4 - Arterial Road System Percentage of Targets Reached According to Various Proposed Scenarios (in millions of dollars) Year 2010 2011 a b Total invested 19.5 39.8 2010 scenario - maintenance of conditiona % Target reached 101.1 19 101.1 39 2010 scenario - minimum responsesb % Target reached 61.1 32 61.1 65 Projected target according to the document "Planification ? long terme des investissements," September 2010, to maintain condition: $326 million, based on the assumption that the arterial road system accounts for 31% of the replacement value of the road system. Projected target according to the document "Planification ? long terme des investissements," September 2010, to implement the required short-term responses on roads and sidewalks in critical condition: $197 million, based on the assumption that the arterial road system accounts for 31% of the replacement value of the road system. We note that the amounts invested in 2010 and 2011 are not nearly enough to meet the needs established with the various targets. They are not sufficient to allow the minimum short-term responses required to be deployed on roads and sidewalks in critical condition, and even less sufficient to allow roads and sidewalks to be maintained in their current condition. Although these are not specific arterial system data, such a situation could cause the percentage of roads and sidewalks "in good condition" to drop from 53% to under 30%, which is well below the 2010 rate, between now and 2020. No matter what scenario is advanced, we note an underinvestment in the system, and this drives the investment deficit higher every year. The consequences of this underinvestment will be very serious in the years to come if the situation is not rectified, because the number of roads and sidewalks in poor condition will increase and more extensive work will be required, not just because of the types of work involved, but also because of the costs. Indeed, reconstruction is more costly than rehabilitation. What is more, this situation is likely to have major disruptive effects for the public. The low levels of investment that prevailed over the years are not aligned with the orientations of the 2008 Plan de transport, which is aimed at restoring the road system to good condition. Up to now, the funding of road and sidewalk work was dictated by budgetary constraints. To reverse the trend and slow down the deterioration of systems, authorities must approve a level of service and set long-term investment levels accordingly to enable officials to: Auditor General of the Ville de Montr?al 252 2012 Annual Report 5.6. Implementation Plan for Arterial Road System Infrastructure Work (Service des infrastructures, du transport et de l'environnement - Direction des infrastructures) o o Plan the implementation of priority responses in a timely manner as part of a comprehensive response strategy; Assess the attainment of objectives. Several references agree on the importance of making decisions in these areas: o According to the National Guide to Sustainable Municipal Infrastructure, city council must approve a level of service before formalizing it. o In June 2010, standing committees on finance and administration established 15 guidelines concerning the municipal administration's orientations and the city's financial framework for 2011. One of these is the need for clearly determining levels of service. o The Direction g?n?rale, in its Politique des ?quipements et des infrastructures, which came into force on January 30, 2009, pointed out the need for setting clear objectives regarding the condition in which property holdings must be kept. o As part of the work of developing a 10-year investment plan, the Service des finances planned to develop a new financial policy addressing financial goals to help determine a desirable investment level. Such a policy was to be approved in August 2010, but this did not occur. In conclusion, the level of service (desired condition of the system), like the level of longterm investment, was never confirmed clearly and specifically by municipal authorities, contrary to what key references have suggested. Without specific objectives, it is difficult to determine the needs to be met, to plan appropriate responses to be prioritized on the system with a long-term perspective and to reconcile all this with projected investments. 3.3.B. Recommendation We recommend that the Direction g?n?rale: o Express the Plan de transport orientations in terms of precise goals concerning level of service; o Set the required long-term investment level for each scenario; o Obtain approval from authorities concerning the targeted level of service and the corresponding long-term investment level and funding method; o Evaluate the results on a yearly basis; to enable it to plan and implement responses in a timely manner, slow the further deterioration of the road system and curb the investment deficit, thereby minimizing disruptive effects for citizens. Auditor General of the Ville de Montr?al 253 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Business unit's response: [TRANSLATION] The IRP proposes an approach based on knowledge of each asset of the road system. The investment needs analysis is therefore based on the actual condition of infrastructures at the time of the analysis. To determine investment needs, the IRP developed a decision tree based on response thresholds (level of service). The investment deficit and resulting investment plans will be calculated on the basis of these thresholds. Both the investment deficit and needs must be reevaluated periodically (five years), so that they take into account actual investments from the previous period. The Direction g?n?rale will submit to elected officials for approval the levels of service recommended for each category of assets and the resulting investment needs. Several proposed scenarios will show the interaction among levels of service and levels of investment at the same time. This discussion could be held in the broader context of the "road strategy" favoured by the Direction des transports. The Direction g?n?rale will keep elected officials informed of the results of investments through an annual investment follow-up report that will be developed by the DI. (Planned completion: March 2014) 3.4. Determination of Priorities 3.4.A. Background and Findings It is essential that care be exercised in the selection of road projects so that the projects implemented can reach the level of service that municipal authorities have set. This cannot be done effectively without first obtaining an overview of the condition of the system as a whole and without classifying responses according to their priority with a view to their implementation in the short, medium or long term. This task would be arduous without the use of specialized software. The process that the DGAV had established to select projects in 2010 and 2011 was described to us in the manner set out below. At the time of our audit, a list of the sections in the worst condition had been created. It consisted of 300 sections with a PPI below 40 (sections in poor and very poor condition). Sections were selected from this list according to an equitable distribution among the boroughs, based on both the number of kilometres in the arterial system and the proportion of the system that was in poor condition. The sections were grouped together to form projects and to create a bank of projects called a "project log." At the time of our audit, this project log constituted, in our opinion, all the response priorities established by the DGAV for the arterial system. Auditor General of the Ville de Montr?al 254 2012 Annual Report 5.6. Implementation Plan for Arterial Road System Infrastructure Work (Service des infrastructures, du transport et de l'environnement - Direction des infrastructures) When projects are selected for its annual planning, the DGAV takes into account projects listed in the project log to determine those appearing to be high-priority for the proper functioning of the road system. Before selecting the projects to include in annual planning, PPI scores and other observations made during tests (e.g., traffic volume, percentage of commercial vehicles) must be confirmed during visits by staff from the DGAV (engineer and technical officer). Laboratory analyses are also requested to specify the type of work to be carried out (e.g., rehabilitation, reconstruction). While this exercise is useful for targeting new projects, annual planning must still integrate: o Projects that compromise minimum safety standards; o Projects for which the implementation phase has already begun (call for tenders issued, contract already awarded or work under way); o Projects that had already been planned the previous year and had been postponed. All this information was used to create a preliminary list for planning for the year (2010 and 2011). The DGAV then took into account lists of projects selected by other business units (e.g., Service de l'eau, Service de la mise en valeur du territoire et du patrimoine [now the Service de la mise en valeur du territoire]), and modified its preliminary list of projects accordingly. A new list of projects was produced and submitted to business units so that they could integrate the projects retained in their planning and conduct any final inspections and analyses that might be necessary. According to the information obtained, annual planning was carried out six to nine months in advance. However, a final list of the projects to be implemented can be accessed only when budgets for the TCEP are confirmed. According to the information obtained, a high-priority project must involve sections with PPIs lower than 40 over most of their surface. During our audit, we wanted to assess the extent to which projects selected in planning for 2010 and 2011, on the one hand, were in line with the response priorities initially established by the DGAV and, on the other, involved sections with a PPI score below 40. If other selection criteria were used, we wanted proof that they had been documented to support the selection of projects. Using a sample of 12 projects appearing in 2010 or 2011 planning, we identified PPIs for each section covered by these projects. Our results show that only one project involved a section with a PPI below 40 over most of its surface. For the other projects in the sample, the breakdown of sections with a PPI below 40 is as follows: o For one project, there was no file backing the PPIs; o For five projects, fewer than 20% of the sections had a PPI below 40; o For five projects, 30% to 50% (in terms of surface area) of the sections covered by the project had a PPI below 40. Auditor General of the Ville de Montr?al 255 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit As a result, almost all the projects reviewed did not involve sections with PPIs below 40 over most of their surface. On first inspection, 11 files out of 12 did not involve projects considered to be in critical condition (very poor and poor). Since other criteria were considered, we looked for proof of documentation. We located a hard-copy file for each of the 12 projects reviewed. According to the information obtained, the creation of hard-copy files for projects included in planning began in 2010. First, in reviewing these hard-copy files, we were able to identify the PPI for the sections covered by projects and find lab reports supporting the type of work to be carried out, as well as correspondence with project stakeholders. However, we did not track down any documents that justified the use of the other criteria used to select these projects (e.g., results of on-site visits, tests to determine traffic volume and vehicle categories). Consequently, for 11 out of 12 tests, we did not obtain proof that the sections selected for the purposes of our audit complied with the criteria used. Moreover, since some of the communication between stakeholders and project managers (engineers) was done electronically, we noted that this information was no longer accessible when resources left the DGAV. At the time of our audit, no procedure was in place for specifying the information that must be archived (hard-copy files and electronic files). In our opinion, such a procedure would help standardize methods and provide evidence of the choices made. At the time of our audit, the IRP was still being developed and it had not been used for planning purposes in 2010 and 2011. A specific IRP was submitted to each borough in 2012. According to the information obtained, the overall IRP should be submitted to authorities in 2013, so that it will be used by planners in 2014. Regarding the integration of road systems (arterial and local) into the IRP, the latter's new planned functions will raise each administrative unit's awareness of the need to take into account the priorities of other systems when managing its own system (local and arterial road systems, water and sewers systems). The review of the IRP first reveals a classification of roads and sidewalks based on their function (local or arterial road system). The threshold criteria used in identifying sections were reviewed. In the IRP are tables listing sections that are critical in "one, two or three" systems. Three tables include one for each system (water, sewer and road) as well as an integrated table of the three systems. The data in this table will be used to select projects to Auditor General of the Ville de Montr?al 256 2012 Annual Report 5.6. Implementation Plan for Arterial Road System Infrastructure Work (Service des infrastructures, du transport et de l'environnement - Direction des infrastructures) be reviewed for the purpose of establishing annual planning for 2014. The information presented covers: o The designation of the section (location) and its length; o The condition of assets (water, sewer and road). Unlike the first RP, the IRP did not provide an overall condition score, the condition being described instead, if applicable, as "critical"; o The type of work to be done (rehabilitation or reconstruction) as well as the costs involved, for this section and for each of these assets. The IRP, which consists of a list of "critical" sections, in a sense becomes the list used for developing the project log. For the DGAV, the IRP is similar to the list of sections with a PPI below 40 produced from the CRCAC database. After that, the planning process remains the same. The only difference is that, from the outset, annual planning (projects launched in 2014 and subsequent years) will take into account the priorities of the Service de l'eau. According to the two methods, the use of the IRP will still require that resources from the DGAV be used to conduct an analysis of sections in order to assess the appropriateness of projects. It should be pointed out that the type of work to be carried out and with the costs involved that are found in the IRP are used to produce an estimate of investments that will be required over the next few years. Before entering a project in a project log, a pre-project analysis must be conducted to confirm or complete the status and functional data, and to specify the type of work to be carried out. In closing, since not all projects appearing in the project log are analyzed to determine their level of priority, we believe that the DGAV is unable to show that projects retained in its annual planning are those with the highest priority for the arterial road system. Furthermore, during our audit, since projects selected by other business units compromised the implementation of projects appearing in the project log, we were not given any assurance that work required on roads and sidewalks in poor or very poor condition appearing in this log would be performed shortly or within a limited period. Nor will the IRP give us this degree of assurance. Considering the large number of potential projects that appear in the project log, we think it is imperative that they be classified according to their priority over a time line of a few years. To do this, this classification must be based on objective criteria, including those outlined in a cost-benefit report for the project, taking into account a favourable time for carrying out the work. For example: o Compromises between less costly responses, which must be paid for now, and more expensive responses, which must be paid for later, are not evaluated; Auditor General of the Ville de Montr?al 257 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit o The effects of accelerating or postponing a response on related cost estimates are not evaluated. This financial information would be useful to planners for evaluating the most favourable time to implement responses. Furthermore, classification of projects according to their priority could also incorporate the concept of socioeconomic costs, or the costs incurred by citizens. In fact, as a result of greater population density on the territory and the greater number of projects of all types aimed at eliminating infrastructure investment deficits, road systems are afflicted with a great deal of traffic congestion. The negative impacts of these factors has received a great deal of media coverage. The SITE could undertake a process of reflection on this subject to assess whether it is appropriate to integrate the socioeconomic factor into selection criteria. Finally, a reliable project classification would be useful in the production of an implementation plan to guide annual project planning. Of course, there should be follow-up on this implementation plan to ensure compliance. If work not appearing in the implementation plan needs to be performed, reasons should be provided. 3.4.B. Recommendation We recommend that the Direction des infrastructures issue guidelines on creating files (hard-copy and electronic) for projects retained in the project log or in the integrated response plan for water, sewer and road systems so that it can show priorities used in decision-making. Business unit's response: [TRANSLATION] The first guideline on the method for saving emails in electronic files was issued in March 2009. This method requires the software Adobe Acrobat Professional, which was installed on all DGAV work stations. A new guideline will be issued on the creation of hard-copy and electronic files specifying the type of information that must be found in the file for each project in the project log or the IRP. The implementation of this guideline could take the form of a project information sheet. (Planned completion: July 2013) Auditor General of the Ville de Montr?al 258 2012 Annual Report 5.6. Implementation Plan for Arterial Road System Infrastructure Work (Service des infrastructures, du transport et de l'environnement - Direction des infrastructures) 3.4.C. Recommendation We recommend that the Direction des infrastructures integrate, in an implementation plan covering a definite timeframe, the responses to be implemented following analysis of the priorities established in the project log or integrated response plan for water, sewer and road systems to guide the selection of projects. Business unit's response: [TRANSLATION] The projects found in the DGAV's annual plan are those with the highest priority; their selection takes into consideration all assets found in the public right-of-way rather than just pavement condition. The inclusion of a project in the annual plan also depends on the ability of each asset or program manager to submit a description of its needs and concepts before the prescribed deadline so that it can be included in the pre-project phase. The DI will establish a five-year planning process with respect to needs identified in the IRP and the project log, failing which stop-gap responses will be considered. This planning will be done concurrently with integrated project planning undertaken by the Direction des transports. (Planned completion: October 2013) 3.4.D. Recommendation We recommend that the Division de la gestion des actifs de voirie document, during the planning stage, projects selected for implementation so that it can justify the decisions made. Business unit's response: [TRANSLATION] A new guideline will be issued on the creation of hard-copy and electronic files, with strong emphasis on the importance of keeping records of any decisions made affecting the selection, priority and timeline for completion of each project. (Planned completion: July 2013) 3.5. Allocation and Use of Resources 3.5.A. Background and Findings At the time of our audit, the DGAV was establishing its priorities and then submitting them when it requested its capital budget. The budget allocated would help set the limits for projects planned for the coming year. We assessed the extent to which planned projects were implemented. First, we wanted detailed information on projects that the DGAV planned to implement using the budget Auditor General of the Ville de Montr?al 259 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit allocated. For 2010, unexpended balances for postponed projects in years prior to 2009 amounted to $15.4 million, or close to 80% of the budget ($19.5 million) (see Tables 5 and 6). The difference of $4.1 million ($19.5 million - $15.4 million) was available to launch new projects, but it was not nearly enough to cover road asset maintenance. It should be mentioned that the year 2010 was marked by the freeze on all budgets allocated to construction projects, including those involving road repair. In 2011, the budget allocated ($39.8 million) also had to account for postponed projects for 2010. The amount left over was $8.4 million, or 21% of the budget allocated, which left slightly more leeway for undertaking new projects ($31.4 million). According to budget documents consulted, for 2010 and 2011, 58% and 31% of the total budgets available were spent, respectively. The budgets allocated and expenditures made in 2010 and 2011 are shown in Table 5. Over a two-year period, an average of only 40% of the amounts budgeted was used. During our audit, the manager in charge of the DGAV confirmed that the authorized budget could not be used in its entirety. Table 5 - Budgets Allocated and Amounts Spent Road Repair Program (TCEP) (in millions of dollars) 2010 Budget allocated 2011 Amount spent Amount not spent Budget allocated Amount spent Amount not spent 19.5 11.3 8.2 39.8 12.4 27.4 100% 58% 42% 100% 31% 69% Table 6 - Amounts Carried Over Road Repair Projects (in millions of dollars) 2009 to 2010 Central city Agglomeration Total 2010 to 2011 2011 to 2012 14.6 6.0 13.2 0.8 2.4 0.7 15.4 8.4 13.9 In our audit, we were able to identify several large projects involving roads and sidewalks of the arterial road system for which implementation was planned in 2011 but was deferred until 2012. According to a table produced in July 2011 by the DGAV, postponed projects accounted for $13.9 million. Auditor General of the Ville de Montr?al 260 2012 Annual Report 5.6. Implementation Plan for Arterial Road System Infrastructure Work (Service des infrastructures, du transport et de l'environnement - Direction des infrastructures) Generally, the consequences of postponing such responses are serious. Here are a few of them: o The necessity, in some cases, for follow-up on projects that are under way for longer periods in order to ensure the safety of users until the situation is rectified, which generates additional costs; o The deterioration of the general condition of roads and sidewalks, which could lead to more costly responses in the future; o The additional maintenance activities that need to be carried out by boroughs. In Tables 7 and 8, we reconciled actual expenditures in 2010 and in 2011 with investment scenarios that the Direction des transports submitted to standing committees on finance and administration in September 2010, as part of the work of developing a 10-year investment plan. These scenarios were framed so as to show the cost of maintaining the condition of roads and sidewalks at the 2010 level ($326 million) or implementing minimum responses required ($197 million). It can clearly be seen that an investment deficit exists for those two years. For the purposes of this comparison, we assumed that 31% of the targets involved the arterial road system (according to the replacement value percentage). Table 7 - Investment Deficit Based on the Projected Level of Investment Maintenance of the Proportion of Roads and Sidewalks in Good Condition (in millions of dollars) Year 2010 2011 a According to the 2010 scenario Investment Budget Amount required allocated spent a 101.1 19.5 11.3 101.1 39.8 12.4 Total Investment deficit 89.8 88.7 178.5 31% of the projected $326 million in investments, according to the 2010 presentation given to standing committees on finance and administration. Table 8 - Investment Deficit Based on Projected Level of Investment Minimum Responses Required (in millions of dollars) Year 2010 2011 a According to the 2010 scenario Investment Budget Amount required allocated spent a 61.1 19.5 11.3 61.1 39.8 12.4 Total Investment deficit 49.8 48.7 98.5 31% of the projected $197 million in investments, according to the September 2010 presentation given to standing committees on finance and administration. Auditor General of the Ville de Montr?al 261 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit According to the two scenarios retained, the investment deficit is between $98.5 million and $178.5 million for 2010 and 2011 alone. This amount would be greater if budgets allocated for years prior to 2010 were taken into account because those budgets were also lower than the investments required according to the investment scenarios. Also, since the scenarios were produced in 2010, the estimates will probably need to be reviewed and updated, which could affect the investment deficit amount. The consequences of underinvestment will be very serious in coming years if the situation is not rectified, because a growing proportion of roads and sidewalks will be in poor condition and the number of emergency responses on the system will also rise. Furthermore, over the years, more work will be required, not only because of the type of work, but because of the costs involved as well. In fact, reconstruction responses are more costly than rehabilitation responses. What is more, this situation could cause major disruptive effects for the public. During our audit, we also wanted to obtain proof of documented project implementation follow-up to determine the degree of progress of projects and evaluate, in a timely manner, the percentage of postponed projects. We also would have liked to find explanations for the postponement of projects included in planning and locate information on the types of actions taken to improve the situation. At the time of our audit, the DGAV, which at that time reported to the Direction des transports, and the Direction des travaux publics (now the DI) held meetings periodically on the progress status of the work that was contracted. However, this exercise was not intended to allow a structured evaluation of the costs and consequences of project postponements. This finding led us to review the information submitted by the DGAV within the framework of the TCEP approval process. These documents make no mention of the amounts of work carried over from previous years. In our opinion, such a situation allows elected officials to believe that a greater volume of work is carried out, whereas only part of the budgets is actually used (average of 40% for 2010 and 2011). In our opinion, an assessment of the costs of project postponements provides important management information that supports accountability reporting on project implementation. It could help provide information periodically to the Direction g?n?rale and elected officials on the consequences of these postponements, thereby promoting informed decision-making. The accountability reporting should also show the extent of the investment deficit in light of the level of service decided upon by authorities. Scenarios for recovering this investment deficit should be proposed in order to slow the deterioration of the arterial road system. Auditor General of the Ville de Montr?al 262 2012 Annual Report 5.6. Implementation Plan for Arterial Road System Infrastructure Work (Service des infrastructures, du transport et de l'environnement - Direction des infrastructures) 3.5.B. Recommendation We recommend that the Direction des infrastructures document monitoring of the progress status of planned investment projects in terms of work performed and provide reasons for postponements in order to find solutions to irritants that delay project implementation. Business unit's response: [TRANSLATION] The DI will overhaul its main project monitoring tool: the control panel. It will be upgraded to include the planning stage (pre-project), thereby giving an overview of all stages of a project. The DI will issue a guideline on the frequency of updating timelines for completion and on documenting events that have led to postponements. (Planned completion: March 2014) 3.5.C. Recommendation We recommend that the Direction des infrastructures report periodically to the Direction g?n?rale and to authorities on the current situation with respect to management of arterial road system assets. In particular, this accountability reporting should: o Focus on the degree of implementation of investment projects that were planned originally, including costs incurred; o Describe the extent of responses considered high-priority that will be deferred and the reasons for their deferral; o Show the future consequences and costs of these project postponements; o Demonstrate how the condition of the arterial road system has changed after tests are conducted and responses are implemented; o Clearly show any changes in the investment deficit, taking into account the level of service approved by authorities to the arterial road system; o Propose scenarios for recovering this investment deficit if applicable; so that informed decisions can be made with respect to expected results. Business unit's response: [TRANSLATION] Before 2012, the DGAV reported to elected officials on the progress of programs, submitted the planning for the coming year and requested authorization to issue calls for tenders. The last such meeting was held on December 14, 2011, regarding 2012 investments. Since 2012, it has no longer been necessary to request authorization for calls for tenders for programs included in the TCEP, with the result that these submissions stopped. Auditor General of the Ville de Montr?al 263 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit The DI will prepare a follow-up report on annual investments and submit it to the Direction g?n?rale, which will decide on the appropriateness of forwarding it to elected officials (executive committee, corporate program committees, municipal committees on finances, etc.). This report would be submitted in March of every year, in anticipation of the adoption of the TCEP in the coming year. (Planned completion: March 2014) Auditor General of the Ville de Montr?al 264 2012 Annual Report Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) 5.7 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) Table of Contents 1. Introduction ................................................................................................................... 269 2. Audit Scope................................................................................................................... 270 3. Findings and Recommendations .................................................................................. 270 3.1. Project Planning .................................................................................................... 271 3.2. Design and Approval of the Tender Documents ................................................... 287 3.3. Compliance of the Contract Awarding Process..................................................... 290 3.4. Approval of Contract Awarding and Other Information Related to Decision-Making.................................................................................................... 298 3.5. Accountability ........................................................................................................ 314 4. Appendices ................................................................................................................... 318 4.1. Sample of Contracts Examined............................................................................. 318 4.2. Summary of the Contracts Examined that Received Additional Funds................. 319 4.3. Difference Between the Amount of the Awarded Contracts and the Cost Estimates Presented in the Decision-Making Summary and Minutes of the Proceedings of the Soci?t? du parc Jean-Drapeau ........................................ 320 Auditor General of the Ville de Montr?al 267 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit List of Acronyms CTA Cities and Towns Act SEAO electronic tendering system DSTI Direction des strat?gies et transactions immobili?res SPJD Soci?t? du parc Jean-Drapeau TCEP FTP functional and technical program three-year capital expenditures program Auditor General of the Ville de Montr?al 268 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) 1. Introduction The Soci?t? du parc Jean-Drapeau (SPJD) is a paramunicipal corporation incorporated by letters patent under the Charter of Ville de Montr?al (the Charter). Its mission is to manage, administer, operate, develop, maintain and plan activities for the Parc Jean-Drapeau site, which comprises Sainte-H?l?ne and Notre-Dame islands. The members of its board of directors are appointed by the executive committee of the Ville de Montr?al (the city), and its budgets (operating and three-year capital expenditures program [TCEP]) are approved by the city's urban agglomeration council given that Parc Jean-Drapeau falls under the category of facilities and infrastructures of public interest. From a territorial standpoint, Sainte-H?l?ne and Notre-Dame islands are part of the territory under the authority of the Ville-Marie borough, in accordance with Schedule B of the Charter. The city makes the buildings, facilities, amenities, infrastructures, public works of art and equipment found on the site available to the SPJD. All of these assets are recorded in the city's books. It was through a memorandum of understanding concluded with the city in 1996 that the SPJD was delegated the responsibilities for: o conservation, protection and enhancement of green and blue spaces; o management of the site with regard to receiving the public, safety and signage; o traffic, access to the islands and parking; o the visual identity and image of Parc Jean-Drapeau; o activities for the general public; o management and conservation of buildings, facilities and works of art on the site; o relationships with partners of the park. In carrying out the responsibilities conferred on it, the SPJD may grant contracts for the construction, renovation, redevelopment and maintenance of any building, facility, amenity or equipment under its jurisdiction. More specifically in this regard, pursuant to article 231.1, Chapter V of the Charter, the SPJD is subject to the regulations of section 573 of the Cities and Towns Act (CTA) regarding the awarding of contracts. In terms of the budget, the memorandum of understanding concluded between the SPJD and the city states that the latter must, as part of the TCEP, provide funding for protection Auditor General of the Ville de Montr?al 269 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit and development work at Parc Jean-Drapeau. Thus, for 2011 and 2012, the SPJD was assigned the management of a budget (modified) within the TCEP of $14.5 million and $12.4 million respectively. As for the operating budget, the SPJD was given $16.5 million and $15.2 million for the same years, including a contribution by the city of $10.7 million and $8.7 million respectively. 2. Audit Scope The purpose of our audit was to ensure that the awarding and management of contracts at the SPJD are done through a rigorous process that complies with the laws, regulations and internal policies in effect. More specifically, we examined aspects surrounding project planning, the design and approval of tender documents, the compliance of the contract awarding process, adherence to the delegation of authority rules for contract approval, the sharing of roles and responsibilities as well as the accountability mechanisms put in place to track the progress of work related to awarded contracts. Our audit focused on a sample of ten contracts valued at over $100,000 awarded following a public call for tenders during 2010 and 2011 as well as the first four months of 2012 (see Appendix 4.1). These contracts concerned the supply of materials, the execution of work or the acquisition of professional services. Our on-site audit work was mainly carried out over the months of May and June 2012, but we also took into account information sent and communicated to us up until October 26, 2012. 3. Findings and Recommendations The SPJD can award various contracts under the responsibilities assigned to it. To illustrate the scope of this activity, the most recent annual report of activities produced by the SPJD reveals that, in 2011, they issued close to 80 public and invited calls for tenders. In order to evaluate the compliance of the contract awarding and management process within the SPJD, especially given the coming into force of new legal obligations in this regard (e.g., the adoption of a contract management policy in December 2010, the tightening of rules regarding the publication of contracts in the electronic tendering system [SEAO] as of April 2011), we assembled our sample using contracts awarded in 2010, 2011 and 2012. Other criteria also guided our selection, namely: Auditor General of the Ville de Montr?al 270 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) o o the type of contract and its financial magnitude. It had to be a contract related to the execution of work or professional services (or a combination of both) valued at over $100,000; the financial magnitude of the project with which the awarded contract was associated. Taking into account these criteria, the ten contracts in our sample were distributed as follows: two contracts awarded in 2010, seven in 2011 and one during the first four months of 2012. It should also be noted that five of the selected contracts are related to the H?l?nede-Champlain building renovation and expansion project, for which the work begun in 2010 was expected to continue at the time of our audit. This major project caught our attention due to the fact that it required considerable investments and, up to now, has required the awarding of several sizeable contracts. As for the five other contracts that made up our sample, they were awarded for the implementation of special projects separate from one another (see Appendix 4.1). Having made these clarifications, the following sections of this audit report will discuss our observations regarding each of the main aspects to be considered in order to ensure, among other things, that the contract awarding and management process adheres to all the established directives (e.g., the level of authorization required), but also that it is carried out in compliance with the laws and regulations in effect and promotes adherence to sound management practices. 3.1. Project Planning Planning a project prior to issuing a call for tenders for the awarding of a contract is a fundamental step in defining its scale and scope and ensuring that it runs smoothly and successfully. This planning phase, which should be properly documented for reference purposes, is one in which solutions are brought and analyzed in order to meet the expressed needs. At this stage, the objectives and expected results related to the expressed needs should be clearly defined, taking into account the available budget allowance, expected completion, risks inherent to the project to be implemented and the need, if applicable, to conduct certain preliminary studies beforehand (e.g., feasibility studies, characterization studies) or obtain required permits. Of course, this step should also take into consideration the obtaining of the various approvals required prior to project implementation. Auditor General of the Ville de Montr?al 271 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit In this regard, given that five of the ten contracts examined are related to the H?l?ne-deChamplain building renovation and expansion project, we felt it was appropriate to make it a separate section from the other projects examined, allowing us to more clearly expose the facts. 3.1.1. H?l?ne-de-Champlain Building Renovation and Expansion Project 3.1.1.A. Background and Findings To put the project into context, we will begin by presenting the history of the main events surrounding its implementation. Thus: o In 2007, the former tenant operating the H?l?ne-de-Champlain restaurant in this building informed the SPJD that he would not be renewing his lease. The building, constructed in the 1930s on the Sainte-H?l?ne island heritage site, was in need of major renovations. o In December 2009, the SPJD launched a call for proposals to find a new operator for the H?l?ne-de-Champlain building. It wanted to receive projects to repair the building as well as give it and the site a new purpose. o In January 2010, the H?l?ne-de-Champlain restaurant closed its doors permanently. o At the end of March 2010, the call for proposals came to a close. The SPJD received three proposals, which were assessed by an evaluation committee made up of one director, three members of management and one SPJD employee. We examined the evaluation process, and it appears to us to have been carried out fairly and in accordance with the pre-established criteria, along with a weighting assigned to each of these. o In April 2010, the SPJD board of directors approved the evaluation committee's recommendation to retain the proposal aimed at turning the H?l?ne-de-Champlain building into a high-end restaurant complex meeting the criteria for "Grand Chef Relais & Ch?teaux" accreditation. The retained developer's proposal planned for the building to house a restaurant, a bar, a bistro, a wine cellar, rental rooms, a cooking school as well as the headquarters and library of the Fondation Relais & Ch?teaux. To accomplish this, the proposed project involved expanding the current building as well as renovating it, modifying it and bringing it up to current standards. o Between June and October 2010, steps were taken to obtain the various approvals required to ensure the project's compliance with municipal regulations and rules regarding heritage conservation, including submitting a request for a special project to Auditor General of the Ville de Montr?al 272 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) o o o the Ville-Marie borough, presenting it to the urban planning advisory committee, applying for permits, and so on. On October 12, 2010, the SPJD board of directors approved the main parameters of the agreement with the developer and authorized management to conclude an agreement largely based on these. It also authorized the chair and vice-chair of its board to sign a lease with the developer as well as the financing documents listed in the agreement, the parameters of which are presented below. These main parameters, which we found in an internal memo presented to the members of the SPJD board of directors during the October 12, 2010, working session, are essentially as follows: - Obligations of the SPJD: act as main coordinating manager for the work to expand and bring the building up to standards, fund the expansion work up to $1 million in return for additional rent equivalent to the repayment of principal and interest, cover the costs of major work--base building, deliver the rented premises to the tenant on June 1, 2011; - Obligations of the retained developer (tenant): ensure that the three partners in the submitted project retain their roles for three years, cover the costs of expansion and construction work required by the tenant, provide the kitchen equipment and contents of the wine cellar as collateral, pay the rent: payment of a progressive percentage rent from year one to year five, corresponding to 4.5% of gross sales up to $10 million and 5.5% above $10 million in gross sales, applicable guaranteed minimum rent, repay the $1 million in funding provided by the SPJD for the expansion work: payment of the principal and interest over 20 years (SPJD rates, plus an additional 2% premium), obtain prior authorization from the SPJD for any work. On October 28, 2010, the SPJD board of directors approved the awarding of a contract for expansion work to be carried out on the H?l?ne-de-Champlain building in the amount of $506,200, before taxes ($571,373, taxes included). On November 24, 2010, the city's executive committee was informed of the agreement parameters (the same as those presented earlier) and authorized the SPJD to sign a long-term lease (35 years) with the retained developer (resolution CE10 1842). The decision-making summary submitted for approval by the SPJD stated estimated costs for the work of $7.3 million, $5.3 million of which was to be funded by the city and $2 million by the future operator for the expansion and certain renovations planned. Auditor General of the Ville de Montr?al 273 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit o o o o o o o On November 29, 2010, the renovation permit requested by the SPJD in regard to the planned expansion work was issued by the Ville-Marie borough. On December 15, 2010, the lease was signed before the notary who took part in drafting the document. This notary was employed by the city's Service des affaires juridiques et de l'?valuation fonci?re. On reading the lease, we noted that an additional obligation that was not originally reflected in the documents presented for approval by authorities had been added to those of the tenant. The lease stipulates (clause 5.2.2) that the tenant shall be responsible for all professional fees, including, but not limited to, those for the architects and engineers associated with the work stated in the lease, with the exception of the fees for overseeing this work, which are the responsibility of the SPJD. In February 2011, the SPJD realized that it was facing various problems, particularly with regard to modifications made to the drawings and specifications, thus making it more difficult to control costs and stay on schedule. A call for tenders was therefore launched to retain the professional services of a firm to provide management support-- construction supervision, for the H?l?ne-de-Champlain building expansion and restoration project. The contract was finally approved by the SPJD board of directors on April 27, 2011, in the amount of $231,120 before taxes ($263,303 taxes included). On June 9, 2011, the SPJD board of directors approved the awarding of the contract to carry out the interior demolition of the H?l?ne-de-Champlain building in the amount of $439,978 taxes included. Shortly after it was hired, the firm responsible for overseeing construction re-evaluated the project and revised its detailed cost estimate, bringing the cost from $7.3 million to $16.4 million (a difference of $9.1 million), $12.4 million of which was to be assumed by the city and $4 million by the future operator. According to the information obtained by the SPJD, a portion of this increase (approximately $2.6 million) was attributable to the unforeseen work related to the discovery of larger amounts of asbestos than expected during the interior demolition of the building. In July 2011, talks were begun between the SPJD and the developer to amend the lease to include, among other things, the expected costs for the work as well as their distribution between the SPJD and the developer. Moreover, because of several construction delays, the delivery date for the rented premises was pushed back to March 1, 2012. On November 9, 2011, the city's executive committee (resolution CE11 1799) authorized the SPJD's decision to modify the lease for the H?l?ne-de-Champlain building and approved the additional funding required to complete the project, revising the SPJD's TCEP to include an additional $5 million for 2011. This plan for a modified lease, which would never be concluded, stipulated that the SPJD would provide Auditor General of the Ville de Montr?al 274 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) o o financing for the developer in the amount of $2 million instead of $1 million, as had initially been established. In the month of November 2011, the documentation in the file reveals a disagreement between the SPJD and the developer with regard to the latter's contractual obligation (lease) to ensure the three partners in the submitted project retain their roles for three years. Finally, on January 18, 2012, even though the renovation and expansion work on the building was not yet completed, the SPJD board of directors decided to terminate the contract (lease) binding the SPJD and the developer. The actual reasons leading to the termination of the lease are unknown to us because they are subject to a confidentiality agreement signed by both parties. According to the information obtained, it would appear that various difficulties encountered throughout the different phases of the work to renovate the H?l?ne-de-Champlain building changed the terms under which the project was being carried out, making it difficult to continue based on the contract (lease) concluded. Consequently, in compensation for the costs incurred by the developer and by certain suppliers that had not been paid, the SPJD was forced to pay close to $2 million. It is therefore based on these events and a review of the five contracts selected for this project that we sought to assess the adequacy of the measures taken at the planning stage of this project. Our audit revealed certain oversights that could, at least in part, explain the hiccups experienced in the project. From the output, we noted, in light of the information gathered from the manager and employee responsible for the project at the SPJD, that no functional and technical program (FTP) was set out prior to the start of this major project. At the planning stage of a project, documenting an FTP would have been a fundamental step since it would have allowed the parties involved, i.e., the SPJD and the developer, to come to an agreement beforehand as to: o the complete determination and definition of needs and requirements in terms of the functional, operational and physical organization of the premises given, among other things, the planned use, the construction standards to adhere to and the quality criteria for obtaining "Grand Chef Relais & Ch?teaux" accreditation; o the costs of the project to be carried out given the available budget allowance; o the planned completion given the scale and specifics of the project to be carried out. What's more, on reading the terms of the lease concluded between the parties, we noted an inappropriate distribution of roles and responsibilities. As we briefly discussed above, clause 5 of the lease clearly stipulated that all aspects related to the design of the project, Auditor General of the Ville de Montr?al 275 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit i.e., preparing the drawings and specifications, was left entirely up to the developer, who, in order to do so, had to select and hire firms of professionals (e.g., engineers, architects) and assume the costs of the associated professional fees. For its part, the SPJD was responsible for launching the calls for tenders for the construction work to be completed and assuming all the related costs (apart from those for the building expansion, which were the responsibility of the developer). According to the information found in the files assembled by the SPJD, this way of proceeding was chosen due to the fact that the developer was not governed by the contract awarding regulations (section 573 of the CTA); the parties were therefore free to entrust the continued design work to the architect firm of their choice. In our opinion, this division of roles and responsibilities, combined with the absence of an FTP, undeniably exposed the SPJD to the risk of losing control over the project, both in terms of staying within the planned budget allowance and on schedule as well as the nature of the expected deliverables, especially since the lease did not stipulate any time frame for the developer to submit the final drawings and specifications to the SPJD. The individuals we met with at the SPJD stated that they did, indeed, face constant delays in the production of drawings and specifications by the professionals hired by the developer, delays which would have been caused by the numerous changes requested to the drawings and specifications and by the presence of several design specialists working for the developer. In fact, since the professionals hired by the developer had no contractual connection to the SPJD, they continued to receive their instructions from the developer, which, according to the information obtained, made it difficult to manage the project. Given these circumstances, and with the aim of delivering the completed building by the date initially stipulated in the lease (June 1, 2011), the SPJD was forced to divide the project into several construction packages, making its coordination all the more complex. Moreover, the individuals met with stated that, for certain construction packages, the SPJD was also forced to launch calls for tenders for work to be carried out based on partially completed drawings and specifications, thus further exposing it to the risks of cost increases and delays. In fact, at the time of our audit, the review of a sample of five contracts awarded in this project revealed significant cost overruns in three of them. For these three contracts, the SPJD board of directors granted additional funds ranging from 94% to 130% above the original cost of the awarded contract (see Appendix 4.2, Table B). These facts observed with regard to the division of roles and responsibilities lead us to question whether enough information was communicated to the various authorities involved Auditor General of the Ville de Montr?al 276 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) (the SPJD board of directors and the city's executive committee) for them to authorize the signing of the lease. In fact, we noted that this aspect of the lease, regarding the developer overseeing the steps related to the project design, was not included anywhere in the main parameters presented to the members of the SPJD board of directors in October 2010 and to the members of the city's executive committee in November 2010 for them to authorize the signing of a lease (see the history of the main events presented earlier). Therefore, while this solution may have been discussed, at least on the part of the SPJD, we feel that all relevant information should have been communicated in a transparent manner in order to allow the decisionmakers to react at the appropriate time and make an informed decision. What's more, in reviewing the information presented by the SPJD in the decision-making summary attached to the November 24, 2010, resolution of the city's executive committee (CE10 1842) to inform the latter of the parameters of the agreement and receive its authorization to sign a long-term lease with the developer, we noted that it shows an estimated cost for the project of $7.3 million (before taxes and contingencies), $5.3 million of which was to be funded by the city and $2 million by the future operator. In this regard, we understand, from reading a presentation made by the SPJD on November 9, 2011, to the city's executive committee, that the portion of this cost estimate that was to be funded by the city ($5.3 million) was based on an evaluation commissioned by the SPJD from a consulting firm a little over four years prior (February 2006). To carry out this evaluation, the firm in question stated in its report that it used various engineering analyses conducted on the building, some of which dated back to 2002, 2003 and 2004. In our opinion, given that these costs were not recent, it was very risky to use them without any indexation and without considering the new function that the building was going to have as part of the project submitted by the developer. This is especially true given the fact that, when the SPJD presented this project, it had in its possession another, more recent evaluation showing that the cost of the project to renovate and expand the building could reach more than $7.3 million. The SPJD had in its possession a cost estimate dating back to 2010, which had been commissioned to a specialized firm by the city's Direction des strat?gies et transactions immobili?res 1 (DSTI) as part of steps taken to assess the investment deficit of municipal buildings. At that time, the cost estimate obviously only pertained to the existing building and not to any other development elements such as the expansion or renovations required due to specific aspects of the project (e.g., the standards to be met for "Relais & Ch?teaux"1 In 2010, this directorate was under the Service des immeubles et des syst?mes d'information. Auditor General of the Ville de Montr?al 277 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit type buildings). The firm therefore evaluated the costs for the work to be $4.7 million; however, it specified that this value was before taxes and other considerations, and it suggested increasing this by 73% to take into account factors such as construction contingencies, professional fees, administration, the contractor's profits and taxes. Consequently, the estimated costs amounted to $8.1 million, not including the costs involved in the expansion planned as part of the project. Furthermore, the retained developer had also attached a budget of $10.7 million to the project, including contingencies for the renovations and modification as well as the expansion of the building. Given this information, we find it hard to understand the reasoning behind using an evaluation from 2006 to support the cost estimates for such a project. In addition, given that the H?l?ne-de-Champlain building was built in the 1930s and then renovated for Expo 67, it was to be expected that it might be insulated with asbestos. In fact, the building evaluation produced in 2006 clearly states that the building contained a large amount of toxic asbestos, requiring a high-level decontamination. These costs alone were estimated at approximately $250,000. Therefore, it would have been relevant to commission the appropriate additional analyses at the planning stage of the project in order to further determine the estimated costs of decontamination as well as those related to the reconstruction of the premises following this procedure (e.g., demolition of walls and ceilings). In this regard, we see that the SPJD did, in fact, hire a specialized firm to determine which materials within the building were likely to contain asbestos. However, it appears that the results of this report, dated December 17, 2010, arrived a bit too late in the process since, at that date, the project had already begun; the terms of the project and the signing of the lease had already been approved by the SPJD board of directors (on October 12, 2010) and, shortly after, by the city's executive committee (on November 24, 2010). Moreover, the actual lease had been signed before the notary on December 15, 2010, two days before the date the report was produced. We tried to obtain the detailed cost estimate for the decontamination related to this characterization analysis from the SPJD staff, but it could not be located. Nevertheless, it is clear that the results of this characterization study, which confirmed the contamination of several components of the building, did not contribute to determining the estimated costs of carrying out the project. Yet, ultimately, the information obtained from the SPJD revealed that a portion of the cost overruns of the project could be partially attributed to the unforeseen work related to the discovery of larger amounts of Auditor General of the Ville de Montr?al 278 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) asbestos than expected during the interior demolition of the building, in the range of $2.6 million. We were, however, unable to reconcile this amount precisely. In sum, it appears to us that the cost estimate was not performed as rigorously as it should have been given the scale and complexity of the project. On another note, it should be stated that, from a financial perspective, the lease stipulated that the rent to be paid by the tenant would be equivalent to the sum of the amounts determined in points A and B below: A. The highest of: o a minimum guaranteed rent, the amount of which was determined in the lease for a period of 35 years; or o a rent based on the gross revenue for each period of the lease multiplied by a percentage established in the lease. B. An additional rent essentially equivalent to the repayment (principal and interest) of the loan made by the SPJD to provide the developer with the funds to expand the building. However, the decision-making summary presented to the city's executive committee to obtain its authorization to sign a long-term lease (35 years) with the retained developer indicates that this lease had to enable the SPJD to benefit from rental revenues of over $15 million. In light of the information obtained from the individuals we met with, it is our understanding that this $15 million corresponded to the variable portion of the rent (see point A above), which was calculated using the estimated gross revenues for the duration of the lease. While this may have been a valid way of proceeding, we nonetheless feel that it would have been wiser to have had a market study conducted beforehand to confirm that the expected revenues were realistic, something which, according to the individuals we met with at the SPJD, was not done. In this regard, we also came across a report entitled "Analyses et commentaires sur les d?passements de co?t du H?l?ne-de-Champlain" (analyses and comments on H?l?ne-de-Champlain cost overruns) produced on December 15, 2011, by the DSTI, a directorate of the Service de la concertation des arrondissements et des ressources mat?rielles, which indicated that the DSTI questioned whether the expected rental income of $15 million was realistic. On the same note, while the city's urban agglomeration council approved the SPJD's guarantee for a $1 million loan, which it took out with Caisse centrale Desjardins to provide financing to the developer for the expansion of the building, the information gathered reveals that no assessment of the developer's solvency was requested beforehand by the SPJD. In consideration of the loan made to the developer, guaranties were, however, stipulated in the lease with regard to kitchen equipment and the content of the wine cellar. Auditor General of the Ville de Montr?al 279 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Furthermore, in the process of examining the events surrounding the implementation of this project, we were surprised to learn that on October 28, 2010, the SPJD board of directors approved the awarding of a contract ($571,373, taxes included) for work to expand the H?l?ne-de-Champlain building, as planned in the project. This contract, under which the work was begun in early November 2010, was in fact awarded: o before the city's executive committee authorized the SPJD to sign the lease (November 24, 2010, resolution CE10 1842); o before the renovation permit for the expansion work requested by the SPJD was issued by the Ville-Marie borough on November 29, 2010; and o even before the official signing of the lease on December 15, 2010. In this regard, it should be noted that the planning phase of the project should have also taken into account the various approvals required prior to its implementation. More specifically, by rushing the start of the construction work to expand the building, work which, as you will recall, was to be the responsibility of the developer according to the parameters of the agreement, the SPJD was exposing itself to the risks of having to assume the costs on its own if the executive committee did not approve the signing of the lease or if the regulatory requirements for the Ville-Marie borough to issue the permit were not met. According to the information obtained from the individuals we met with at the SPJD, the tight deadline for delivery of the completed building, i.e., June 1, 2011, was one of the main motivating factors justifying this way of proceeding. At the time of our audit, the total budget allocations granted to the firm awarded the contract for expansion of the H?l?ne-deChamplain building totalled $1,174,457, taxes included, which is 106% more than the contract originally awarded (see Appendix 4.2, Table B), and still other amounts had to be invested to complete the work. In sum, we found that several important steps in the planning of this project were not given the full attention they deserved. In particular, we observed the following: o the absence of an FTP aimed at clearly defining the scale and scope of the project needs; o improper division of roles and responsibilities between the SPJD and the developer, wherein the SPJD was not entirely in control of the project, as had initially been stated to the authorities to obtain authorization to sign the lease; o the cost estimate for carrying out the project was based on historical data that was not up to date; o additional analyses to further specify the costs of the project were not obtained at the proper time (e.g., determination of the materials likely to contain asbestos), while other analyses were not requested (e.g., market study to confirm the expected revenues for the project, assessment of the developer's solvency); Auditor General of the Ville de Montr?al 280 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) o certain crucial approvals were obtained after the start of the project. Since January 18, 2012, the date on which the SPJD board of directors decided to terminate the lease concluded with the developer originally retained, the following steps have been taken to relaunch the project under new terms: o production of an FTP and new drawings and specifications aimed at completing the construction work and delivering a building that is up to standards to a future operator; o launch of a search for a new operator for the H?l?ne-de-Champlain building; o launch of a call for tenders to award a contract for completion of the remaining work needed to bring the building up to standards and expand it. According to the most recent information obtained from the SPJD dated October 26, 2012, the actual costs incurred to date for this project to renovate and expand the building add up to $8.7 million. Bids were received in August 2012 following the call for tenders launched by the SPJD in July 2012 to finalize the renovation and expansion work. The proposal from the lowest compliant bidder was valued at $7.5 million. On October 26, 2012, the awarding of the contract was pending approval of city authorities. On the date this audit report was validated, November 8, 2012, the SPJD expected the work to be completed in fall 2013. Given the substantial costs related to this project, we currently question the possibility of finding an operator whose proposal will make the facilities profitable. In conclusion, considering that improving the infrastructures of Parc Jean-Drapeau is part of the priorities of the municipal administration with regard to the development Montr?al, which will celebrate the 375th anniversary of its founding in 2017, it should be expected that the SPJD will have many other large-scale projects to manage. In fact, this motivation on the part of the municipal administration is reflected in the TCEP budget tabled for 2013-2015, in which a sum of $110 million has been confirmed for the redevelopment of Parc JeanDrapeau assets: $32.4 million in 2013, $42.3 million in 2014 and $35.3 million in 2015. Given this considerable increase in the budget allowance granted to the SPJD compared to the 2011-2013 TCEP ($14.5 million, $10 million and $10 million respectively) for the conservation of the site's buildings equipment and works of art, we believe it would be appropriate for it to re-evaluate its practices so that the projects to be implemented benefit from all the appropriate structuring and monitoring at the planning stage. Auditor General of the Ville de Montr?al 281 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.1.1.B. Recommendation We recommend that the Soci?t? du parc Jean-Drapeau take all the necessary steps to complete the H?l?ne-de-Champlain building renovation and expansion project, ensuring that: o all the decisions to be made are controlled to guarantee that the work is in line with the set objectives based on the functional and technical program developed; o the costs to be incurred going forward are rigorously monitored so that the project is completed within the provided budget allowance; o the impact of any other potential risk related to a future operator's proposal for this building is identified and assessed. Business unit's response: [TRANSLATION] Have the cost of the remaining work evaluated by an independent firm. (Completed, November 2012) Implement the required procurement and authorization process to award the contract to a general contractor for the work to be completed. (Completed, December 2012) Report on the progress of the work and evolution of the costs of the project at each meeting of the audit and TCEP committee and of the SPJD board of directors. (Completed, periodically beginning in March 2013) Launch a search aimed at a wide range of bidders in order to choose the one offering the project with the best revenue potential and minimal risks (business plan, solvency, clause for withdrawal/penalty/guarantees). (Completed, March 2013) 3.1.1.C. Recommendation We also recommend, if other major projects are to be implemented, given the size of the three-year capital expenditure program budget granted, that the Soci?t? du parc Jean-Drapeau: o re-evaluate its practices so that these projects benefit from all the appropriate structuring and monitoring at the planning stage; o ensure that all the information needed to make an informed decision is communicated to the authorities at the appropriate time. Business unit's response: [TRANSLATION] Develop more detailed project management standards and procedures to ensure consistent practices at each stage of a project: prefeasability, Auditor General of the Ville de Montr?al 282 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) estimate, contract awarding, accountability and project closure. (Planned completion: May 2013) Standardize the information and its presentation to facilitate comprehension of the files when decisions are to be made by the executive director, the audit and TCEP committee and the board of directors. "Standard decision-making summary." (Planned completion: May 2013) 3.1.2. All Projects in Our Sample 3.1.2.A. Background and Findings As we mentioned earlier, while five of the ten contracts examined were related to the H?l?ne-de-Champlain building renovation and expansion project, we also reviewed five other contracts that were awarded for the implementation of the following special projects: 1. Construction of a glassed-in staircase and an elevator at the Fort de l'?le Sainte-H?l?ne arsenal-Stewart Museum; 2. Supply and installation of signage elements-phase 3; 3. Repair of the masonry in the powder magazine; 4. Design and construction of a new service courtyard; 5. Expansion of the Jamaica pavilion. This section will therefore include our observations regarding the planning of these five other projects as well as our observations regarding certain elements of planning stemming from a more in-depth review of the documents underlying all the contracts in our selection. On one hand, our review of the documentation assembled to launch the call for tenders for the awarding of the contract (e.g., the specifications, the reference terms and the project manual) allowed us to assess the measures taken by the SPJD to define and circumscribe the work to be done. We were thus able to determine the result of the steps taken prior to project implementation for aspects such as: o the project description and nature of the work required; o the functional requirements (e.g., specification of needs in usable area) and the technical requirements (e.g., specification of needs with regard to electrical or mechanical features); o the planned completion schedule. On the other hand, our review of the documentation for each of the contracts in our sample allowed us to uncover potential areas of improvement with regard to aspects that should be given particular attention at the planning stage. These aspects are discussed below. Auditor General of the Ville de Montr?al 283 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Cost Estimate One of our observations pertains to the cost estimate for the work involved in the contract to be awarded. We reiterate that establishing a detailed cost estimate is an important step in planning a project. This cost estimate makes it possible to both determine the market solicitation method to use (e.g., an invited or public call for tenders) and assess the funds required to carry out a project in order to then evaluate the reasonableness of the prices submitted by the bidders and make an informed decision as to whether or not to award a contract. In this perspective, it is important that all the efforts required to document this estimate be put forth and that recorded in the file so as to allow the individuals involved to consult them. However, at the time of our audit, finding these cost estimates proved to be quite difficult since none of them were recorded in the file created by the SPJD for the contracts reviewed (100% of cases). Following our requests to obtain them from the project managers (one of which was an external consultant), the documentation was located for six of them (six cases out of ten, see Appendix 4.1--Nos. 1, 3, 5, 7, 8 and 10). As for the four other cases (see Appendix 4.1--Nos. 2, 4, 6 and 9), while the estimates had been prepared prior to launching the call for tenders according to the managers we met with, documentation for these could not be located. These four contracts were associated with the H?l?ne-deChamplain building renovation and expansion project, specifically the expansion work, construction management, interior demolition work and the supply and installation of windows and doors. Among the six detailed estimates that were found, we noticed that one of them, related to the design and construction of a new service courtyard, was not dated or identified with the name of the person who prepared it. While, according to the information obtained, this estimate was prepared by the project manager responsible for the file, who was employed by the SPJD, it seems to us that the lack of a date and identification of the person in charge, which would have made the document more official, could cause questions to remain as to the precise moment this estimate was prepared or the value of the estimate prepared, given the expertise of the person assigned to this task. In these circumstances, in order to allow for a reliable value to be assigned to them, we feel that the SPJD should take the necessary measures to ensure that all cost estimates bear the date on which they were prepared and the name of the person or firm that prepared them. Moreover, given their importance in the awarding process, we feel that cost estimates should be properly documented and kept with all the other documents assembled for the call for tenders. This way, it will be easier to follow-up on a file if a different person is Auditor General of the Ville de Montr?al 284 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) assigned to it and to demonstrate the reasonableness or unreasonableness of bids received in order to make an informed decision. Our audit also revealed the existence of different cost estimate amounts for the same contract. In fact, we found that the cost estimate amount presented to the board of directors for the awarding of the contract did not always match the amount appearing in the documented cost estimate (two out of ten cases). These two cases are detailed in Table 1 below. Table 1 - Contracts Presenting a Discrepancy Between the Documented Cost Estimate and that Presented to the Authorities Purpose of the contract Amount appearing on the documented cost estimate (including taxes and contingencies) Cost estimate amount presented in the decision-making summary and related minutes (including taxes and contingencies) Design and construction of a new service courtyard $9,123,057 $8,500,000 Landscaping and civil engineering (H?l?ne-de-Champlain building renovation and expansion project) $836,454 $800,000 a Difference $623,057 a $36,454 This amount was found by performing a reconciliation since it was included in an overall estimate of $1,525,000 presented in the minutes of the November 9, 2011, meeting of the SPJD and is attributable to the approval by the SPJD board of directors of the awarding of a batch of five contracts for which the overall cost quoted by the lowest bidders was $1,590,268. Likewise for the cost estimate presented in the SEAO following the awarding of the contract sometimes differed from the amount appearing in the estimate documented by the SPJD (one out of ten cases). Table 2 presents the information related to this case. Table 2 - Contract Presenting a Discrepancy Between the Documented Cost Estimate and that Presented in the SEAO Purpose of the contract Amount appearing on the documented cost estimate (including taxes and contingencies) Design and construction of a new service courtyard Auditor General of the Ville de Montr?al $9,123,057 285 Cost estimate amount presented in the SEAO Difference (including taxes and contingencies) $8,000,000 $1,123,057 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit With regard to the aspects related to the information to be published in the SEAO, note that they will be discussed further in section 3.3 of this audit report dealing with the compliance of the contract awarding process. Moreover, we reviewed the lists of contracts held by the SPJD, entitled "Liste des appels d'offres inscrits dans le syst?me ?lectronique d'appel d'offres" (list of call for tenders entered in the electronic tendering system), for 2011 and 2012. We found that these lists contained a range of information pertaining to contracts to be awarded, including the estimated value of the work involved. However, for certain contracts examined, we also observed a different amount from that in the cost estimate. In these circumstances, the confusions observed with regard to the cost estimates presented lead us to question the thoroughness with which they were prepared and the accuracy of the information communicated to the members of the SPJD board of directors to obtain their approval for the awarding of the contracts. 3.1.2.B. Recommendation We recommend that the Soci?t? du parc Jean-Drapeau take the necessary measures to: o reiterate to the personnel in charge the importance of documenting the detailed cost estimates and keeping these in the file containing all of the documents assembled for the launch of a call for tenders; o reiterate to the personnel in charge the importance of making sure that the cost estimates prepared are dated and identified with the name of the person or firm of experts assigned to this task; o ensure consistency in the information presented to the directors of the Soci?t? du parc Jean-Drapeau, in the electronic tendering system and in any other work document prepared, with regard to established cost estimates; to ensure that any person assigned to a file can easily consult the detailed cost estimate, and so that this estimate may also be used in deciding whether or not to award a contract. Business unit's response: [TRANSLATION] Determine the essential documents required, in particular the estimates, for every step of a project, ensure they are produced in a compliant manner and followed-up on, and lastly, centralize where they are kept (archives). (Completed, March 2013) Auditor General of the Ville de Montr?al 286 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) Standardize the content and presentation of the estimates for investment projects to ensure that they are signed by the designated person. Confirm where and when they were produced. (Completed, March 2013) Have the various contributors to the file sign it to attest to the accuracy of the information and to ensure that it is presented in a consistent manner at every step in the process. (Planned completion: May 2013) 3.2. Design and Approval of the Tender Documents Subsequent to planning the work to be carried out or the professional services required, tender documents must be prepared, based on the magnitude of the sums, to proceed to market solicitation. The design of these documents, which include the instructions for bidders, the tender schedule, the drawings and specifications, the general and specific clauses and the addenda, requires close monitoring to ensure that their content provides not only equal treatment to all bidders but also all the clauses enabling the SPJD to control the conduct of the work in terms of planned timelines, payment methods and the progress of the work or services rendered. In this regard, all approvals from the appropriate hierarchical levels should be obtained prior to public distribution (launch) of such tender documents. It is therefore in this perspective that we examined the tender documents for each of the ten contracts in our selection. We wanted to ensure that they had been designed to promote competitive bidding, that they had received the required approvals prior to their launch and that they included clauses that were specific enough to allow the SPJD to control the conduct of the work with regard to the aspects mentioned above. 3.2.1. Design of the Tender Documents with Regard to Competitive Bidding 3.2.1.A. Background and Findings The documents appear to us to have been designed to promote competitive bidding. In addition, all the bidders received the same tender documents, the addenda were sent to all at the appropriate time when required, and a single person, specifically the purchasing manager, was clearly designated to receive the requests for information from bidders interested in the call for tenders. Auditor General of the Ville de Montr?al 287 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.2.2. Approvals Prior to the Launch of Tender Documents 3.2.2.A. Background and Findings With regard to the approval of tender documents prior to their publication in the SEAO, it is important that this responsibility be exercised by a manager with the required authorization level according to the authority delegated within the SPJD. In this regard, it is our understanding, after reading a presentation document prepared by the administration directorate of the SPJD on April 5, 2011, entitled "Approvisionnement, normes et proc?dures" (procurement, standards and procedures), that the internal rules set out by the SPJD state that the requester of a call for tenders (e.g., the project manager) has the following responsibilities, among others: o obtain, prior to starting the process leading to the launch of a call for tenders (e.g., determining the needs, preparing the drawings and specifications, drafting a detailed cost estimate for the project), authorization from the person with the appropriate level of authority according to the SPJD's authority delegation rules; o prepare the specification, if this is not done by the external consultants; o perform an expense estimate (if this is not done by external consultants) and ensure the budget is available; o inform the procurement advisor (purchasing manager) about the file in progress and determine the appropriate awarding method (e.g., invited or public call for tenders). The procurement advisor must, among other things: o ensure compliance of the retained procurement process (e.g., the awarding method, the choice of supplier); o analyze the tender documents prepared by the project managers or external consultants, whatever the case may be, adapt the administrative clauses and make recommendations as to the modifications deemed necessary; o publish the call for tenders in the SEAO. Therefore, the internal standards and procedures set out by the SPJD provide that the requester of a call for tenders must obtain prior authorization from a person with the appropriate level of authority. However, with regard to approval of the content of the tender documents and their subsequent publication, the involvement of a person with a higher level of authority is in no way mentioned. Moreover, in conducting our audit, we found, with regard to all the tender documents related to the contracts examined, whether they were prepared internally or by external consultants, that there was no evidence of: Auditor General of the Ville de Montr?al 288 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) o o prior approval authorizing the start of the process leading to the launch of a call for tenders, as set out in the standards and procedures established by the SPJD; the review and approval of the content of the tender documents, as well as their publication in the SEAO. According to the information obtained from the individuals we met with, these approvals are given on more of a verbal basis, without any record of them necessarily being included in the file. More specifically with regard to tender documents prepared by a firm of external consultants, it should be noted that payment of fees should not be interpreted as an admission that the work and services are fully or partially complete, satisfactory or consistent with the contract. We therefore feel that proof of approval of the documents prepared by external consultants should also be applicable. Thus, in order to ensure greater transparency in the decisions made and approvals obtained, we feel that the internal standards and procedures set out by the SPJD with regard to procurement should be improved by including the required approvals at the steps in which they are missing. In addition, we feel that a mechanism should be introduced within the organization to make it possible to demonstrate proof (leave a paper trail) of the approvals obtained from superiors for authorizations related to starting the process to launch a call for tenders, the content of tender documents after revisions have been made, and publication of the call for tenders in the SEAO. For example, this mechanism could consist of a form included in the files assembled for the awarded contracts that contains fields to indicate information such as: o the purpose of the contract; o the name of the requester; o the amount of the cost estimate for the work or services required; o the awarding method retained; o a brief description of the main aspects of the tender documents that have been revised; o the signature of the authorized person(s) designated to provide the necessary approvals. Auditor General of the Ville de Montr?al 289 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.2.2.B. Recommendation To ensure greater transparency in the decisions made, we recommend that the Soci?t? du parc Jean-Drapeau: o review its internal standards and procedures regarding procurement so that the document entitled "Approvisionnement, normes et proc?dures" (procurement, standards and procedures) includes the prior approvals required from superiors at the steps in which they are missing, namely when the content of the tender documents are approved following verifications and when they are published in the electronic tendering system; o take the necessary measures to develop a mechanism to keep a written record of approvals granted with regard to: - the start of the process leading to the launch of a call for tenders by a requester as already set out in the existing internal standards and procedures, - the review of the content of tender documents, whether prepared internally or by external consultants, prior to their publication, - the publication of these tender documents in the electronic tendering system. Business unit's response: [TRANSLATION] The "project sheet" will have to be completed by the various individuals in charge and signed by the designated authorities in accordance with the delegation of authority rules. Once they are validated and signed, the sheets will be submitted to the audit and TCEP committee as well as the board of directors for decision-making and project follow-up. (Completed, March 2013) 3.3. Compliance of the Contract Awarding Process 3.3.A. Background and Findings The SPJD is subject to the general rules applicable with regard to the awarding of contracts by municipal bodies stipulated in section 573 of the CTA, pursuant to article 231.1, Chapter V of the Charter. As a paramunicipal corporation, the SPJD acts through its board of directors. Therefore, when the CTA mentions the "council" in a context where this applies to a paramunicipal corporation, this refers to the board of directors of the corporation. According to these rules, any contract related to the supply of goods and services exceeding $100,000 can only be awarded after a public call for tenders by way of Auditor General of the Ville de Montr?al 290 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) advertisement in a newspaper distributed on the territory of the municipality. The call for tenders must also be published in the electronic tendering system approved by the Government of Qu?bec. The time limit given to bidders for receipt of their bid must be no less than 15 days. Subsequently, the bids received must be opened publicly in the presence of at least two witnesses at the date, time and location stated in the call for tenders. More specifically, with regard to professional service contracts valued at $25,000 or more for which a municipal body must proceed to a public or invited call for tenders public, the general rule requires the use of the two-step tender evaluation and weighting system. The quality of the service offer is evaluated first according to pre-established criteria in order to eliminate, if need be, offers that do not receive a passing grade (at least 70%). The price of the offers will only be considered in the second step. In addition, a selection committee must be appointed to evaluate the offers received. The purpose of our audit was to verify, among other things, whether contracts were being awarded through a process that complies with the laws, regulations and internal policies in effect. Our review of the ten contracts in our sample led us to conclude that the contract awarding process was carried out in accordance with the rules stipulated in section 573 of the CTA, with the exception of the following aspects discussed below. Specifically with regard to the contracts related to the supply of professional services, the CTA states the following: Where a contract for professional services is to be awarded, the council must use a system of bid weighting and evaluating whose establishment and operation are consistent with the following rules: (1) the system must have a minimum of four evaluation criteria in addition to price; (2) the system must provide for the maximum number of points that may be assigned to a tender for each of the criteria other than price; that number may not be greater than 30 out of a total of 100 points that may be assigned to a tender for all the criteria; (3) the council shall establish a selection committee consisting of at least three members, other than council members, which must (a) evaluate each tender without knowing the price; (b) assign a number of points to the tender for each criterion; (c) establish an interim score for each tender by adding the points obtained for all the criteria; (d) as regards the envelopes containing the proposed price, open only those envelopes from persons whose tender has obtained an interim score of at Auditor General of the Ville de Montr?al 291 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit least 70, and return the other envelopes unopened to the senders 2 . (Our emphasis) Thus, according to these provisions of the CTA, it is the responsibility of the SPJD board of directors, prior to the launch of the call for tenders, to approve both the bid evaluation grid that will be used to evaluate the offers, i.e., the criteria and points assigned to each of them, and the composition of the selection committee that will be responsible for evaluating the offers received. However, for the two contracts in our selection related to the supply of professional services (see Appendix 4.1--Nos. 4 and 7), while we found that an evaluation grid had been used and that a selection committee had been put together, all in accordance with the specifications stipulated in the CTA, we were unable to find any evidence of these two aspects having received prior approval from the SPJD board of directors. Also, more specifically with regard to approval of the composition of the selection committee, the CTA states that "The council may adopt a by-law delegating the power to establish a selection committee to an officer or employee of the municipality and setting the conditions and procedures for the exercise of the delegated power 3 ." The information obtained from the SPJD personnel confirms that the board has adopted no such resolution. While, according to the opinion obtained from its legal advisors, the SPJD does not have this obligation, we feel that, in order to provide greater transparency, it would be appropriate for it to apply stricter rules, following the example of the city. Similarly, the excerpt from section 573 cited earlier stipulates that for bids that did not obtain an interim score of at least 70, the envelopes containing the proposed price must be returned to their sender unopened. However, for one of the two professional service contracts examined (see Appendix 4.1--No. 4), we found, on examining the file created for this purpose, that the envelopes containing the price of the bidders that did not obtain a passing score were still in it. While they did not appear to have been opened, these envelopes should have nonetheless been returned to their senders so as to comply with the provisions of the CTA. 2 3 CTA, section 573.1.0.1. CTA, section 573.1.0.1.1. Auditor General of the Ville de Montr?al 292 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) 3.3.B. Recommendation In order to provide greater transparency, we recommend that the Soci?t? du parc Jean-Drapeau take the necessary measures to: o ensure that the bid evaluation grid that will be used to evaluate the offers as well as the composition of the selection committee responsible for evaluating the offers received be approved by the Soci?t? du parc Jean-Drapeau board of directors prior to the launch of the call for tenders or that this authority be delegated where permitted by the Cities and Towns Act; o reiterate to the personnel involved the obligation to return the price envelopes unopened to the bidders who did not obtain a passing score; in order to ensure compliance with the provisions of section 573 of the Cities and Towns Act pertaining to the awarding of professional service contracts. Business unit's response: [TRANSLATION] Calls for tenders involving a qualitative analysis component will be evaluated according to the standard evaluation grid adopted through a resolution of the SPJD board of directors. (Completed, March 2013) The SPJD authority delegation rules will have to be adjusted to reflect the delegation of authority for the composition of selection committees to the SPJD management (adoption also required by the city's executive committee according to current letters patent). (Planned completion: December 2013 [if executive committee]) The designated person will have to confirm, by way of signature, that envelopes have been returned unopened to bidders who did not obtain a passing score. (Completed, January 2013) 3.3.1. Additional Budget Allocations Granted 3.3.1.A. Background and Findings As part of our review of the contracts in our sample, we also paid close attention to the additional budget allocations granted by the SPJD to the same successful bidder above the contract amount originally submitted by the latter. At the time of our audit, four out of the ten contracts in our sample had received approval for additional funds (see Appendix 4.2). For three out of these, associated with the H?l?nede-Champlain renovation and expansion project, we found, as mentioned earlier, that a considerable amount of additional funds had been granted above the original cost of the awarded contract, i.e., 94% to 130% more. Clearly, it is understood that these additional Auditor General of the Ville de Montr?al 293 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit funds were systematically granted to the same successful bidder (by mutual agreement), without repeating the call for tenders procedures. In this regard, the CTA states the following: "A municipality may not amend a contract awarded following a call for tenders unless the amendment is accessory and does not change the nature of the contract. 4 " In this regard, we found, in reading the minutes of the proceedings, that the SPJD generally justified increasing the value of the contract originally awarded to the retained firm by claiming that these were accessory amendments. We are aware that a contract may be modified in the midst of being carried out due to unforeseen events or constraints that can occur. We also recognize that, depending on the nature of the work or services to be provided, it could be more practical and sometimes even more cost-effective to allow the same firm to continue the work. However, in the cases cited, it seems, given their magnitude, that there may be some question as to whether the additional funds qualify as an accessory cost in the meaning of the CTA. We understand that every situation is unique and requires a specific review since case law does not provide any guidelines as to the definition of the term "accessory," other than that the accessory should generally be less than the principal. In this perspective, we feel that the SPJD should have obtained the opinion of a legal advisor specialized in the matter in order to validate the legality of entrusting further work to the same successful bidder by private agreement. In this regard, the managers in charge stated to us that a legal opinion had been requested in every case to confirm that the additional funds to be granted could qualify as accessory costs in the meaning of the CTA. We therefore asked to obtain a copy of these opinions since none of these could be found in the file created for the contract in question. After a search carried out by these same managers, it would appear that the said legal opinions were obtained verbally over the phone, without any official documented proof to support the result of the opinion requested from the legal advisor being sent to the SPJD. In this perspective, in order to demonstrate the measures taken to ensure the compliance of the contract awarding process, we feel that it would be in the SPJD's best interest to require that documented legal opinions be sent to it so that these can be included in the file related to the contract in question. 4 CTA, section 573.3.0.4. Auditor General of the Ville de Montr?al 294 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) 3.3.1.B. Recommendation We recommend that the Soci?t? du parc Jean-Drapeau: o require its legal advisors to send it the result of requested legal opinions in writing; o keep the documentation of these legal opinions in the file created for the contract in question; in order to demonstrate the measures taken to ensure the compliance of the contract awarding process with section 573 of the Cities and Towns Act when additional funds are required to continue the work planned in an initial contract. Business unit's response: [TRANSLATION] When a request for additional funds is deemed necessary by the project manager, a written request will have to be produced in order to obtain the legal advice required for authorization from the applicable body (Direction g?n?rale, board of directors or executive committee). The related request and advice will be kept in the project file in the archives. (Completed, January 2013) 3.3.2. Publication of Contract Information in the Electronic Tendering System 3.3.2.A. Background and Findings As we briefly discussed earlier, on April 1, 2011, new legislation was also introduced in the CTA (sections 477.4 to 477.6) following the coming into force of Bill 76, entitled An Act to amend various legislative provisions principally with regard to the awarding process for contracts made by municipal bodies. These new provisions mainly pertain to the obligation that all municipal bodies have to publish in the SEAO, approved by the Government of Qu?bec, the list of all contracts concluded that involve an expenditure of $25,000 or more (excluding employment contracts). This list must be updated at least once a month and must include the following information for each contract: o if the contract involves an expenditure of $100,000 or more, the price of the contract as estimated by the body (entered after the opening of the bids or when the contract is awarded); o the price of the awarded contract and the name of the bidder with which it was concluded; o the purpose of the contract; Auditor General of the Ville de Montr?al 295 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit o the total amount of the expenditure actually incurred (this amount must include, where applicable, contingencies and expenses related to duly authorized additional work). For contracts concluded following a call for tenders process, the list must also contain the following information: o the name of each bidder; o the amount of each bid; o identification of any bid that was lower than the one chosen and deemed to be noncompliant. Thus, for each of the contracts in our selection for which the new publication rules applied due to the fact that they were awarded after April 1, 2011 (eight out of ten cases, see Appendix 4.1--Nos. 3 to 10), we verified whether the information had been entered in the SEAO as required. Our audit revealed the following findings: o With regard to the estimated contract price to be entered in the SEAO: - In five out of eight cases, no amount was entered (see Appendix 4.1--Nos. 3, 4, 5, 6 and 8); - In one out of eight cases, the estimated price entered matched the detailed estimate prepared by the SPJD (see Appendix 4.1--No. 10), whereas in the two remaining cases, the estimated price did not match (see Appendix 4.1--Nos. 7 and 9). o With regard to the total amount of the actual expenditure to be entered once the work is complete (the work was complete for five out of eight contracts): - In four out of five cases, no amount was entered in the SEAO under the heading "Montant total de la d?pense effectivement faite" (total amount of the actual expenditure) (see Appendix 4.1--Nos. 3, 5, 6 and 10); - In one out of five cases, the amount indicated matched the original cost of the contract and did not include the additional funds subsequently granted (see Appendix 4.1--No. 4). 3.3.2.B. Recommendation We recommend that the Soci?t? du parc Jean-Drapeau implement the mechanisms necessary to ensure that all the information related to each of the contracts granted is properly entered in the electronic tendering system, in order to comply with the provisions of sections 477.4 to 477.6 of the Cities and Towns Act. Business unit's response: [TRANSLATION] Segment the task "'information entry" and "information validation" in the SEAO. The person in charge of validating the information will have to sign the Auditor General of the Ville de Montr?al 296 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) validation action on the "project sheet." The documents supporting this validation will also be kept in the project file, in the archives. (Completed, January 2013) 3.3.3. Adoption of a Contract Management Policy 3.3.3.A. Background and Findings In conclusion, we should mention that, in accordance with the legislation in effect as of January 1, 2011 (section 573.3.1.2 of the CTA), on December 14, 2010, the SPJD board of directors adopted its contract management policy aimed at reinforcing the principles of sound competition, efficiency, ethics, transparency and fairness applicable to all contracts. According to this new obligation incorporated in the CTA, the SPJD had to include in its contract management policy measures aimed, among other things, at the following: 1. ensuring that no bidder or representative of a bidder has contacted or attempted to contact a member of the selection committee with regard to the call for tenders for which he or she has presented a bid in an attempt to influence them; 2. combatting bid rigging; 3. promoting compliance with the Lobbying Transparency and Ethics Act; 4. preventing acts of intimidation, influence peddling and corruption; 5. preventing conflicts of interest; 6. preventing any other situation liable to compromise the impartiality and objectivity of the tendering process and resulting management; 7. structuring all decision-making related to the authorization to modify a contract. We reviewed the content of the contract management policy adopted by the SPJD and concluded that its content complies with provisions set out in section 573.3.1.2 of the CTA. Moreover, for the contracts in our selection that were awarded after the coming into force of this new contract management policy (January 1, 2011), we found that the various measures stated in the SPJD's contract management policy had been incorporated in the tender documents. For example: o designation by the SPJD of a person to contact for questions; o solemn declarations of the bidder; o solemn declarations of the members of the selection committee, where applicable ; o abstention from bidding of any supplier who has participated in preparing the tender documents. Our audit of the contracts did not reveal any irregularity with regard to compliance with these provisions. Auditor General of the Ville de Montr?al 297 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.4. Approval of Contract Awarding and Other Information Related to Decision-Making Approval of contract awarding must not only be conducted in accordance with the authority delegated to various managers or employees in this matter, taking into account the financial magnitude of the goods or services requisitioned, but it must also be based on information that is sufficiently complete and accurate to allow for informed decision-making. At the same time, the minutes of SPJD proceedings should be an accurate and complete reflection of the subjects discussed and decisions made by the directors so that they can be referred to later on to ensure follow-up on and implementation, as well as to serve as proof in the event of disputes or litigation. At this stage in the process, our audit consisted in ensuring that the contracts in our selection had received authorization in accordance with the approval levels (delegation of authority) in effect within the SPJD and that this delegation of authority had been done in compliance with the provisions contained in the letters patent. We also paid close attention to the relevance and sufficiency of the information communicated to the directors through internal memos or decision-making summaries 5 prepared to help them fully understand all the contextual elements they need to consider in deciding whether or not to approve the awarding of contracts. In addition, the documentation of the SPJD's minutes of proceedings was examined to assess whether they were precise and complete enough with regard to the decisions made. To do this, we read all of the minutes of the board of directors and the reports of the audit committee for the years 2010 and 2011 as well as for the first six months of 2012. 3.4.1. Delegation of Authority Rules for Contract Approval 3.4.1.A. Background and Findings The SPJD's procurement policy includes the approval levels required by the various stakeholders involved according to the amount of the goods and services requisitioned. These approval levels, which were reviewed and recognized by a resolution of the SPJD board of directors during the November 9, 2011, working session, are presented in Table 3. 5 We found that the SPJD uses two types of documents for contract approval, the "internal memo" and the "decision-making summary." Since these two documents contain essentially the same information, we will use the term "decision-making summary" in this audit report. Auditor General of the Ville de Montr?al 298 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) Table 3 - Approval Levels Required Based on the Amounts of the Goods and Services Amount of the goods and services (taxes included) less than $10,000 $10,000 to $25,000 $25,000 to $100,000 $100,000 to $2,000,000 $2,000,000 and over Stakeholders and approval levels Director Director and senior director Senior director and executive director Board of directors or audit committee, on recommendation of the executive director, to be ratified by the board of directors City's executive committee, on recommendations of the board of directors For information purposes, this procurement policy also states that: [TRANSLATION] Senior management is authorized to make unforeseen additional expenditures not exceeding 10% of the contracts and mandates awarded. These additional expenses must not, however, exceed $100,000 per awarded contract or mandate. If this percentage or amount is exceeded, any additional or unforeseen amount will have to be authorized by the SPJD board of directors 6 . First, we noted that the SPJD board of directors established the approval levels for the authorities delegated to the various stakeholders involved through a resolution and not a regulation as is set out in the letters patent issued by the Government of Qu?bec when the SPJD was constituted. Article 4 of the supplementary letters patent registered on July 7, 1995 stipulates, among other things, that the SPJD is administered by a board of directors (article 4.a). Also, when the latter seeks to delegate some of its responsibilities for any matter that is useful or necessary to the internal control of the SPJD, it must do so by way of a regulation, which will come into force on approval of the city's executive committee (article 4.j). Given these circumstances, we feel that all delegated authorities, particularly those set out in the November 9, 2011, resolution of the board of directors and the delegation of authority granted to the executive director through the procurement policy, regarding the unforeseen additional expenditures not exceeding 10% of the cost of the awarded contracts and mandates, are aspects related to the internal control of the SPJD. They must therefore be recognized by an official regulation that will be submitted to the city's executive committee for approval, in accordance with what is set out in the letters patent in effect. 6 Procurement policy, Soci?t? du parc Jean-Drapeau, p. 6. Auditor General of the Ville de Montr?al 299 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Moreover, with regard specifically to the delegation of authority granted to the executive director for approval of unforeseen additional expenses, we also feel that regular reporting to the SPJD board of directors should be performed to ensure transparency in the decisions made. We also noticed inconsistencies in the information contained in the various source documents used within the SPJD regarding the aspects related to the delegation of authority (approval levels based on the amounts of goods and services). For one, the SPJD procurement policy has not been updated to reflect the November 9, 2011, approval by its board of directors of the revised approval levels. For another, in regard to the delegation of authority granted to the executive director for the unforeseen additional expenses, the procurement policy refers to a maximum of 10%, while the document entitled "Approvisionnement, normes et proc?dures" (procurement, standards and procedures) states a maximum of 20%. In our opinion, it would be appropriate for the existing reference documents at the SPJD to be updated and standardized to make them more useful and avoid any possible confusion on the part of the various stakeholders involved in the contract awarding process within the SPJD. We also noted that the SPJD audit committee was added as an approving officer for contracts valued between $100,000 and $2,000,000. More specifically, for the goods and services valued within this range, the approval rule regarding the stakeholders authorized to approve the expenditure is as follows: [TRANSLATION] "Board of directors or audit committee, on recommendation of the executive director, to be ratified by the board of directors." (Our emphasis) According to the November 9, 2011, minutes and considering the information obtained from the individuals we met with, we understand that the possible involvement of the audit committee was added to allow for greater flexibility at the time contracts are awarded in cases where there is a tight deadline and there is no working session of the board of directors scheduled in the near future. However, this procedure is not in compliance with the memorandum of understanding (agreement) entered into between the SPJD and the city on November 6, 1996. The memorandum of understanding states the following: [TRANSLATION] "The SPJD must put together an audit committee whose composition, operating rules, authorities, duties and attributions appear in Appendix 1 7 ." On reading the content of Appendix 1 of the memorandum of understanding pertaining to the audit committee, we found that there is no provision whatsoever that this body can 7 Agreement between the SPJD and the Ville de Montr?al, November 6, 1996, article 5.3.5, p. 8. Auditor General of the Ville de Montr?al 300 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) award contracts. In this perspective, the November 9, 2011, resolution of the SPJD board of directors does not comply with the provisions of the memorandum of understanding. In fact, the very purpose of the delegation of authority to the audit committee is incongruous. The SPJD cannot act through two "authorities" for a decision of the same nature. In this sense, a decision (e.g., the awarding of a contract) that has to be ratified by the board of directors, as currently set out in the SPJD's rules for authority delegation, cannot become effective before the latter intervenes. Yet, in the case under discussion, the work related to the expansion of the Jamaica pavilion began even before the audit committee's decision (on March 14, 2012) was authorized by the SPJD board of directors. In fact, we found that the purchase order for this contract had been created on March 16, 2012, and that the work began around March 19, 2012, according to the information found in the file supporting this contract, while the file was presented to the members of the board of directors during a working session held on March 29, 2012. In addition, the very possibility of allowing the audit committee to award contracts can be called into question due to the fact that, by nature, this type of committee is mainly responsible for monitoring the processes related to the quality of the financial information produced or internal controls. Contract awarding therefore does not fall under its authorities, duties or attributions. We would like to point out that our understanding of the facts related to the non-compliance of the two aspects that we just discussed with regard to the delegation of authority implemented within the SPJD (need for a regulation regarding the delegation of authority and non-compliance with the memorandum of understanding pertaining to the audit committee) was confirmed through a legal opinion obtained by the city's Service des affaires juridiques et de l'?valuation fonci?re. This said, while the delegation of authority should be ratified through a regulation and not a simple resolution, the current situation is more a question of compliance than unlawfulness since it does not invalidate the decisions made by the individuals involved based on this resolution. Therefore, our examination of whether there was compliance with the delegation of authority rules established for approval of the contracts in our sample was based on those existing at the SPJD. Our audit therefore revealed that, in general, the delegation of authority rules in effect within the SPJD were respected, with the exception of the situation discussed below, which involved a contract in our selection. Auditor General of the Ville de Montr?al 301 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit On reading the minutes of the SPJD board of directors, we found that at the November 9, 2011, working session, the directors present ratified the decision made by the chair of the board of directors eight weeks earlier to award 12 contracts totalling $3.7 million. Among these contracts was the one from our selection regarding the landscaping and civil engineering on the site of the H?l?ne-de-Champlain building (see Appendix 4.1--No. 8). According to the information obtained from an email message (dated September 14, 2011) discussing the awarding of these contracts, it appears that the decision made by the Chair of the board of directors was motivated by the critical timelines for carrying out the work. We recognize that certain situations may require quick decision-making; however, in these circumstances, it is clear that the other members of the board of directors were presented with a fait accompli and were not able to exercise their decision-making authority, as is set out in the delegation of authority rules implemented within the SPJD. Therefore, in order to prevent this type of situation happening again, we feel that the SPJD should evaluate, based on the urgency of the situation, the relevance of holding a special meeting of the board of directors. 3.4.1.B. Recommendation In order to provide greater transparency, we recommend that the Soci?t? du parc Jean-Drapeau take the necessary measures to ensure that all the authorities delegated by its board of directors are ratified through a regulation, which should be duly approved by the city's executive committee in order to comply with the provisions set out in the letters patent in effect at the Soci?t? du parc Jean-Drapeau. Business unit's response: [TRANSLATION] It should be noted that the proposed letters patent developed in 2011 would modify this recommendation. If the letters patent are not modified: present the SPJD's authority delegation regulation to the executive committee for adoption. (Planned completion: December 2013 - executive committee). We wish to remind you that a draft agreement with the city, a draft internal governance regulation and a regulation on Parc Jean-Drapeau were also developed in 2011. Auditor General of the Ville de Montr?al 302 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) 3.4.1.C. Recommendation We recommend that the Soci?t? du parc Jean-Drapeau revise its delegation of authority rules to remove the audit committee from its list of authorized stakeholders for the approval of expenditures in order to comply with the provisions of Appendix 1 of the memorandum of understanding concluded between the Soci?t? du parc JeanDrapeau and the Ville de Montr?al with regard to the responsibilities assigned to this committee. Business unit's response: [TRANSLATION] It should be noted that the proposed letters patent developed in 2011 would modify this recommendation. If the letters patent are not modified: present the SPJD's authority delegation regulation to the executive committee for adoption. (Planned completion: December 2013 - executive committee). We wish to remind you that a draft agreement with the city, a draft internal governance regulation and a regulation on Parc Jean-Drapeau were also developed in 2011. 3.4.1.D. Recommendation To ensure transparency in decision-making, we recommend that the Soci?t? du parc Jean-Drapeau plan to implement a mechanism to periodically report to the members of the board of directors regarding unforeseen additional expenses approved by the executive director for an awarded contract or mandate, given the delegation of authority granted to him in this regard. Business unit's response: [TRANSLATION] The executive director can currently authorize expenditures according to SPJD standards and procedures. Currently, he can also authorize an addition to the contract not exceeding 20% of the value of the initial contract (the 20% must be less than $100,000). When the initial contract is awarded by the board of directors, the executive director will have to specify the addition on the "project sheet." (Completed, January 2013) Auditor General of the Ville de Montr?al 303 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.4.1.E. Recommendation We recommend that the Soci?t? du parc Jean-Drapeau update and standardize its reference documents (procurement policy, standards and procedures) in particular so that they accurately reflect the delegation of authority rules approved by the authorities, thus making them more useful and avoiding any possible confusion on the part of the various stakeholders involved in the contract awarding process. Business unit's response: [TRANSLATION] Following adoption of the SPJD's delegation of authority regulation, the procurement policy, the standards and procedures document as well as the "project sheet" and its instructions for use will be grouped together in one reference document. (Planned completion: December 2013) The contract management policy, which is required by law, will, however, remain a separate document. 3.4.1.F. Recommendation We also recommend that the Soci?t? du parc Jean-Drapeau plan to hold a special meeting of the board of directors whenever the urgency of the situation justifies it in order to comply with the delegation of authority rules approved by the Soci?t? du parc Jean-Drapeau. Business unit's response: [TRANSLATION] The chair of the SPJD board of directors and the chair of the audit and TCEP committee, on the recommendation of the executive director, will decide on the need for special meetings of the board of directors. (Completed, January 2013) 3.4.2. Sufficiency of the Information Presented for Decision-Making Purposes (Decision-Making Summary) 3.4.2.A. Background and Findings To enable directors to quickly become familiar with all the contextual aspects underlying the need to award a contract, it is customary for a document entitled "decision-making summary" to be prepared and submitted to them during a board working session. If these decision-making summaries are to be a useful management tool, they must be prepared in such a way as to include information that is sufficiently accurate and complete to allow the directors to make an informed decision. These decision-making summaries should include: Auditor General of the Ville de Montr?al 304 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) o o o o o o o a presentation of the contextual information justifying the decision to authorize the awarding of a contract for the required work or services to be performed (e.g., objectives to achieve, description of the work required, related past decisions, issues); the estimated cost for the work or services required, including contingencies, if applicable; the retained awarding method, the names of the invited firms or, if it is a public call for tenders, the number of companies that have obtained the specifications as well as the number of bidders in contention; the results of the interim scoring of each bidder and the final score, for contracts awarded through a two-step evaluation and weighting system in accordance with the CTA; an analysis of the differences between the prices submitted by the various bidders in contention (e.g., comparison between the average cost of the compliant bids received and the lowest compliant bid, between the price of the highest and lowest compliant bid as well as between the lowest and second lowest compliant bid); an explanation of the difference (based on a given threshold) between the cost estimate prepared by the SPJD prior to the launch of the call for tenders and the cost of the bid of the lowest compliant bidder; the price of the bid as well as the name of the recommended bidder following the tendering process. As part of our audit, we examined the decision-making summaries prepared for the ten contracts in our selection in order to determine whether the information provided to the directors was sufficient for them to make a decision. Our audit revealed that certain information was missing that would help improve the usefulness of decision-making summaries in making an informed decision. More specifically, we found that the cost estimate for the work to be carried out, a key element in decision-making, was not always specifically indicated in the summary. This situation was observed in three out of the ten contracts examined. One of these was related to the contract awarded for the construction of a glassed-in staircase and elevator in the arsenal of the Fort de l'?le Sainte-H?l?ne-Stewart Museum (see Appendix 4.1--No. 1), for which an evaluation of the cost estimate for the work to be carried out was missing from both the decision-making summary and the minutes (May 11, 2010, meeting of the board of directors). The two other cases observed pertain to the contract for the landscaping and civil engineering as well as the contract for the supply and installation of windows and doors Auditor General of the Ville de Montr?al 305 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit (see Appendix 4.1--Nos. 8 et 9), both awarded as part of the H?l?ne-de-Champlain building renovation and expansion project. For these two cases, we found that the related decision-making summaries pertained to the approval of batches of contracts that included the two contracts in our sample. The contract related to the installation of windows and doors is part of a batch of 11 contracts, while the contract related to the landscaping and civil engineering is part of a batch of 5 contracts. Both batches were to be approved by the SPJD board of directors. In terms of the contract related to the supply and installation of windows and doors included in a batch of 11 contracts, we observed that the decision-making summary as well as the minutes (November 9, 2011, working session) both completely lacked any evaluation of the estimated costs for each of these 11 contracts, the price of the lowest compliant bid individually submitted for each of the contracts and the identification of these firms. The resolution of the board of directors stipulates the following: [TRANSLATION] Considering that the work related to these contracts includes, but is not limited to: o the interior carpentry and cabinetwork; o the wrought iron work related to the interior design; o the supply and installation of exterior wood windows and doors; o the plumbing and natural gas; o the heating and ventilation; o the supply and installation of regulation and control systems; o the supply and installation of fire protection systems; o the prepurchase of five ventilation systems for the building; o the prepurchase of ventilation hoods for the kitchens; o the prepurchase of hot-water boilers for the building; o the prepurchase of heat exchangers for the building. It is resolved: to award 11 contracts to various firms for a range of work to be performed at the H?l?ne-de-Champlain pavilion at the prices and under the conditions of the submitted bids, in accordance with the results of the public calls for tenders, and authorize a total expenditure for this purpose of $5,370,516.01, including all taxes 8 . (Our emphasis) Lastly, with regard to the contract related to the landscaping and civil engineering on the H?l?ne-de-Champlain site, our examination of the decision-making summary prepared by the SPJD, which contained more or less the same information as that presented in the minutes of the November 9, 2011, meeting of the board of directors, revealed the following stipulation: 8 Minutes, meeting of the Soci?t? du parc Jean-Drapeau's board of directors, November 9, 2011, p. 7 and 8. Auditor General of the Ville de Montr?al 306 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) [TRANSLATION] [...] five (5) batches [contracts] of work were tendered out through public calls for tenders to allow the work on the site to continue. These were the following: Batch 7.3. Prepurchases--Chillers Batch 7.4. Prepurchases--Evaporative condenser Batch 7.7. Prepurchases-- power generator and automatic change-over valves Batch 8. Passenger elevator and freight elevator Batches 24. and 35. Landscaping and civil engineering 9 . (Our emphasis) Under the heading "Aspects financiers" (financial aspects), it is stated: [TRANLATION] Prior to the call for tenders, the cost of the work related to the batches was estimated by project professionals to be valued at $1,525,000, including taxes. All of the lowest bids submitted for each of the batches total $1,590,267.68, including taxes 10 . We found, as with the previous case (batch of 11 contracts), that the decision-making summary as well as the minutes did not include anything with regard to the evaluation of the costs estimated for each of the five contracts as well as with regard to the price of the lowest compliant bid individually submitted for each of them and the identity of the firms retained. We found, however, that while there was mention of a cost estimate the amount submitted to the SPJD directors for approval of the awarding of the contracts was a cumulative amount for the five contracts in the batch. Based on the information contained in the list of contracts kept by the SPJD, entitled "Liste des appels d'offres inscrits dans le syst?me ?lectronique d'appel d'offres" (list of calls for tenders entered in the electronic tendering system), we were able to piece together the details of the overall estimate ($1,525,000) presented in the decision-making summary and minutes. This reconciliation revealed that the cost estimate presented for the contract in our selection related to the landscaping and civil engineering of the H?l?ne-de-Champlain site amounted to $800,000. As we mentioned earlier in section 3.1.2, while the difference is not that great, we found, for one, that this amount does not correspond precisely to the amount appearing in the documented detailed estimate found for this contract ($836,454). In addition, our examination also revealed that, taken separately, the five contracts that make up this batch display differences between the cost estimate used by the SPJD (according to the breakdown of the $1,525,000) and the price of the lowest compliant bid 11 ranging from - 9 10 11 Decision-making summary, Soci?t? du parc Jean-Drapeau's board of directors, September 2, 2011, p. 1. Decision-making summary, Soci?t? du parc Jean-Drapeau's board of directors, September 2, 2011, p. 2. According to the minutes of the public opening of the bids received. Auditor General of the Ville de Montr?al 307 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 66.9% to + 36% (see Table 4), compared to a total difference of 4.3%, as suggested by the information presented in the decision-making summary. Table 4 - Details Regarding the Amounts of the Cost Estimates used for the Five Contracts Included in the Batch Purpose of the contract Costs of the estimate used Cost of the lowest compliant bid Difference Batch 7.3. Prepurchases - chillers $125,000 $75,706.16 (39.4%) Batch 7.4. Prepurchases - evaporative condensers $125,000 $41,525.66 (66.8%) Batch 7.7. Prepurchases - power generator and automatic change-over valves $250,000 $82,698.16 (66.9%) Batch 8. Passenger elevator and freight elevator $225,000 $302,544.93 34.5% Batches 24. and 35. Landscaping and civil engineering $800,000 $1,087,795.77 36.0% $1,525,000 $1,590,267.68 4.3% Total In examining the content of the decision-making summaries, we also found that a mention of the contingencies available, in cases where this would apply (eight out of ten cases, see Appendix 4.1--all contracts except Nos. 4 and 9), was only present in three of the contracts examined: the project to build a glassed-in staircase and elevator at the arsenal of the Fort de l'?le Sainte-H?l?ne--Stewart Museum, the project to repair the masonry in the powder magazine and the project to design and build a new service courtyard (see Appendix 4.1-- Nos. 1, 5 and 7). For the five other contracts out of the eight, we found that: o for two of them (see Appendix 4.1--Nos. 8 and 10), a provision for contingencies had been set out in the tender schedule included in the tender documents, but no mention was made of it in the decision-making summary or the minutes. In our opinion, the magnitude of the provision for contingencies is part of the relevant information that should systematically be communicated to the directors to allow them to make an informed decision; o for the three other contracts (see Appendix 4.1--Nos. 2, 3 and 6), we found that no provision for contingencies was mentioned in the tender schedule or the decisionmaking summaries and that no budget in this regard had been approved by the board of directors at the time the contract was awarded. We feel that the SPJD would have every interest in setting out a budget provision for contingencies at the time contracts are awarded to deal with unforeseen circumstances that may arise during the execution and thus: Auditor General of the Ville de Montr?al 308 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) - - avoid delaying work completion due to having to wait for the next working session of the board of directors to obtain approval for the additional funds required; limit the risks of further work being carried out without the prior approval of the SPJD directors. Moreover, our audit of the decision-making summaries underlying the contracts in our selection revealed the absence of the following information: o A comparative analysis of the differences between the prices submitted by the various bidders in contention was generally not documented in the decision-making summary (nine cases out of ten). Only a brief mention that the prices of the first ten bids received were within an 8% range was found in the decision-making summary related to the contract awarding for the project to build a glassed-in staircase and elevator at the arsenal of the Fort de l'?le Sainte-H?l?ne--Stewart Museum (see Appendix 4.1--No. 1). o An explanation justifying, if applicable, the difference between the cost estimate prepared for awarding of the contract and the cost of the bid from the recommended bidder was never documented in them. In this regard, the information obtained from the people we met with confirmed that the SPJD has not established any internal directive stipulating the threshold at which explanations must be provided in the decision-making summary presented to directors. For example, this directive could state that differences greater than 10% (plus or minus) between the prepared cost estimate and the price of the recommended bidder must be systematically explained and documented in the decision-making summary. We therefore proceeded to carry out this comparative analysis for the contracts in our selection. To do this, we compared the cost of the awarded contract with that of the cost estimate presented to the SPJD directors in a decision-making summary, which was also fully reflected in the minutes of the corresponding board meeting, for the cases in which this information was provided. We were therefore able to perform the comparative analysis for eight out of the ten contracts in our selection (see Appendix 4.3). The results of this analysis showed that the differences calculated ranged from - 42.8% to + 73.4%. Among these eight contracts, only two displayed a difference between the cost estimate and the price of the lowest compliant that fell under an acceptable threshold, i.e., ? 10%. In our opinion, disclosing such large differences as the ones mentioned above in the decision-making summaries may have certainly raised questions on the part of board Auditor General of the Ville de Montr?al 309 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit members, and in the case of significant overruns, may have even led them to rethink their decision to award the contract. Given this, we feel that the decision-making summaries presented to the directors should be improved and standardized with respect to the information contained in them so that they provide all the relevant information needed for making an informed decision. Accordingly, we feel that the SPJD should re-evaluate its practices and develop a guide or directive to provide a better framework for the preparation of decision-making summaries in terms of the information to be contained in them. 3.4.2.B. Recommendation In order for its directors to benefit from a useful and relevant tool to help them make informed decisions, we recommend that the Soci?t? du parc Jean-Drapeau reevaluate its practices in the short-term and develop a guide to provide a better framework for the preparation of decision-making summaries in terms of the information to be contained in them so that their content is more consistent and improved through the incorporation of information such as: o disclosure of the amount of the cost estimate prepared prior to the awarding of a contract, the price of the retained bid and the identity of the bidder; o disclosure of the contingency funds provided; o a comparative analysis of the differences between the prices submitted by the various bidders; o an explanation, based on a threshold that is deemed acceptable (e.g., ? 10%), of the differences found between the cost estimate prepared for the contract awarding and the cost of the bid from the recommended bidder; and to distribute this guide to all the appropriate employees within the Soci?t? du parc Jean-Drapeau. Business unit's response: [TRANSLATION] The decision-making summary presented for purposes of decisionmaking must be validated by more than one person, in accordance with the information stated in the "project sheet" signed by the designated authorities. (Completed, March 2013) Develop more detailed project management standards and procedures to ensure consistent practices at each stage of a project: prefeasability, estimate, contract awarding, accountability and project closure. (Planned completion: May 2013) Standardize the information and its presentation to facilitate comprehension of the files when decisions are to be made by the executive director, the audit and TCEP Auditor General of the Ville de Montr?al 310 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) committee and the board of directors. "Standard decision-making summary." (Planned completion: May 2013) Standardize the content and presentation of investment project cost estimates for purposes of signature by the designated authority. (Note that the design and construction contingencies are not the same and also vary based on the type of site, the complexity of the design and the precision of the FTP.) (Completed, March 2013) The "Octroi de contrat" (contract awarding) section details the difference between the cost of the contract and the estimate as well as the applicable justification if necessary. (Completed, March 2013) 3.4.2.C. Recommendation We recommend that the Soci?t? du parc Jean-Drapeau set out a budget provision for contingencies at the time contracts are awarded to avoid delaying work completion due to having to wait for the next working session of the board of directors to obtain approval for the additional funds required as well as to limit the risks of further work being carried out without the prior approval of the Soci?t? du parc Jean-Drapeau directors. Business unit's response: [TRANSLATION] "Standard decision-making summary." The project will include, according to the nature of the contract, a contingency percentage. The contract budget will relate to the overall budget for the project. (Completed, March 2013) 3.4.3. Documentation of Minutes of Proceedings 3.4.3.A. Background and Findings The minutes is an official document detailing what was said, debated, done or decided during a working session of the board of directors. It confirms the conduct of a session according to the items on the agenda. In essence, it is the corporate memory and serves as a reference when needed. Its purpose is to: o relate past or more recent events to those not present; o record the decisions made to facilitate their follow-up and application; o maintain the continuity of debates; o serve as proof in case of disputes or lawsuits, or a source of information for the various stakeholders interested. Auditor General of the Ville de Montr?al 311 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Given that directors sitting on boards of directors must assume great legal responsibilities, they have the obligation to act with caution and diligence. It is therefore important that the minutes prepared following a board's working sessions be sufficiently detailed and transparent with regard to the decisions made or directions chosen to demonstrate that the directors properly carried out the responsibilities incumbent on them. As part of our audit, in addition to examining the minutes related to the approval of the ten contracts included in our sample, we also looked at all the SPJD minutes related to the working sessions held between February 9, 2010 and June 21, 2012. As we briefly discussed in the previous section, this examination revealed that the information identified as being missing in the decision-making summaries prepared for the contracts in our sample (e.g., disclosure of the amount of the cost estimate prepared, disclosure of the contingency funds available, the comparative analysis of the differences in the prices submitted by the various bidders, the explanation of the differences found between the cost estimate prepared and the price of the bid from the recommended bidder) was also missing in the SPJD minutes. In fact, we found that the information appearing in the decision-making summaries had been transcribed almost word for word in the minutes, without any other additions. Furthermore, on reading the minutes, we found that they were incomplete in certain regards since certain relevant information was not necessarily included in them. More specifically, we observed the following: o When professional service contracts are awarded on the basis of an hourly rate, the information regarding the maximum amount of the authorized expenditure is not always indicated, nor is the hourly rate of the lowest compliant bidder. The awarding of the following contracts can be cited as an example: - Electromechanical system maintenance (resolution CA2010-23); - Civil works (resolution CA2010-24); - General construction work (resolution CA2010-25). o The decisions made or discussions regarding the various items on the agenda of the board of directors working sessions are not fully reflected in the minutes. For example, we observed the following lapses: - No evidence of the adoption of the agenda since 2011. We therefore do not know if items were added or if modifications were requested; - No mention of documents that may have been tabled for information purposes, nor any related discussions. On reading the minutes, we therefore cannot be sure that the members had at their disposal all the documents that were supposed to be presented to them to allow them to make an informed decision; Auditor General of the Ville de Montr?al 312 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) - - - - - No mention of questions from Board members; Lack of details regarding the conclusion of a decision. While the decisions are recorded in the minutes, we do not know if the decision was reached unanimously or by the majority or if there were any dissenting votes; Absence of information regarding certain items on the agenda. We do not know if these were deferred to a later meeting or cancelled for various reasons. Consequently, it becomes difficult to track the progress of files over time; Information sometimes missing in the minutes regarding cancellation of the awarding of contracts approved during a previous working session (e.g., certain contracts awarded as part of the H?l?ne-de-Champlain building renovation and expansion project were subsequently cancelled without this being clearly reflected in the minutes); Missing information regarding the amount of the fees that the SPJD has agreed to pay out. For example, as part of the H?l?ne-de-Champlain building renovation and expansion project, the resolution (CA2012-09) of the SPJD board of directors working session (February 22, 2012) makes no mention of the amount of the compensation to be paid (close to $2 million) to the developer following the termination of the contractual agreement (lease) concluded with the SPJD. Given these circumstances, although we have not excluded the possibility that other complementary information documents may have been presented to the directors in addition to those indicated in the minutes, and that these may have been discussed during the working sessions, we have no way of knowing for sure whether this was the case. Consequently, for purposes of providing greater transparency as well as protecting the interests of the organization and the individuals who are held accountable for the decisions made, we feel that the minutes should be improved to include more details regarding the underlying information elements of decisions made, as well as any facts related to the topics discussed and directions chosen. Accordingly, we feel that it is in the SPJD's best interest to review its practices, basing itself on the best management practices in the matter, to then develop a guide to provide a framework for the preparation of the minutes with regard to their structure (e.g., an "Agenda" section, a "Contract Awarding" section), as well as the nature of the information to appear in it and the level of detail expected. In our opinion, the SPJD minutes should be well documented, such that they report, with total transparency, on all the information elements considered to support the decisions made. Auditor General of the Ville de Montr?al 313 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.4.3.B. Recommendation In order to provide greater transparency as well as to protect the interests of the organization and the individuals who are held accountable for the decisions made, we recommend that the Soci?t? du parc Jean-Drapeau: o review its practices regarding documentation of the minutes of the board of directors so that they more precisely and consistently incorporate all of the information elements underlying the decisions made and directions chosen as well as the topics discussed; o develop a guide based on the best practices in the matter to provide a framework for the preparation of the minutes with regard to aspects such as their structure, the nature of the information that should appear in them and the level of detail expected. Business unit's response: [TRANSLATION] Review the practices related to the drafting of the minutes in accordance with the recommendations of a legal advisor. Train and inform the individuals in charge of drafting the minutes. (Completed, March 2013) 3.5. Accountability 3.5.A. Background and Findings Ultimately, this monitoring discussed earlier regarding the progress of work underlying awarded contracts must also provide for regular reporting mechanisms aimed at providing sufficient relevant information to managers and SPJD directors to allow them to judge the progress of work and, if applicable, to make the necessary decisions with regard to the types of corrective measures that need to be taken. In this regard, our audit revealed that few to no management reports were produced by project managers to regularly inform management and the SPJD board of directors of the progress of the work related to awarded contracts. These management reports should, in our opinion, contain a sufficient amount of relevant information about the progress of the project with regard to both costs and timelines as well as expected deliverables to allow for assessment of the attainment of set objectives and, if necessary, react in a timely manner to remedy the situation. Auditor General of the Ville de Montr?al 314 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) Among the projects covered by the contracts in our selection, we found that progress reports were produced for only two of them, the one regarding the H?l?ne-de-Champlain building renovation and expansion project (grouping together five contracts in our sample) and the one pertaining to the project to design and build a new service courtyard. At the time of our audit, two progress reports had been produced for each of the two aforementioned projects, one in February 2012 and the other in May 2012. These reports were presented to the SPJD directors during a board working session. According to the information obtained from the project manager in charge, the production of such reports is a new practice. It would appear that, in general, the information related to the progress of the work underlying the awarded contracts is communicated mostly verbally during regular meetings between the various stakeholders involved. On closer examination of the content of said progress reports, we found that these were more narrative in nature and briefly discussed the following elements: o a brief reminder of the set objective for the project being carried out; o the status of the work (e.g., what is completed, what is under way and what has not yet begun); o the planned completion dates based on the progress of the work, without relating new dates to the initial ones determined; o the amounts of the actual costs incurred to date or a note indicating that the project is within the initial budget and that contingency funds have been dipped into. However, a comparison between the planned budget and the actual costs is not always clearly presented. We feel that this type of report is a relevant management and decision-making tool for managers as well as for the members of the board of directors since it can provide an overview of the progress of the work underlying the awarded contracts. Moreover, we feel that SPJD project managers should, at certain intervals to be determined, produce such progress reports for each of the projects under their responsibility. However, in order to make these reports more useful, we feel that their content should be improved and made more specific through the addition of information aimed at: o illustrating the comparison between the actual costs and the established budget in order to promptly identify foreseeable cost overruns, if any exist; o illustrating the comparison between actual timelines and those planned and revised, if applicable, in order to determine whether the project is delayed or on schedule; o explaining the reason for differences that may have been noted in the magnitude of the costs or adherence to the timelines mentioned above; Auditor General of the Ville de Montr?al 315 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit o o o presenting the percentage of progress of the work; reporting on the anticipated problems, risks or major issues that could have negative repercussions on the budget, the schedule and the expected deliverables, as well as proposed alternatives to mitigate the effect of the problematic causes detected; describing the assessment of the overall status of the project (e.g., "under control," "ahead of schedule" or "critical"). To do this, we feel that SPJD management would benefit from producing an internal directive aimed at communicating its directions regarding the nature and frequency of the progress reports to be produced by its project managers as well as the work tools that could be used for these purposes (e.g., use of the MS Project software). 3.5.B. Recommendation To facilitate informed decision-making in a timely manner, we recommend that the Soci?t? du parc Jean-Drapeau produce an internal directive describing it directions in order to: o require progress reports to be produced regularly by its project managers for each of the projects under their responsibility; o specify the nature and frequency of production of these progress reports, which could include, among other things, information aimed at: - illustrating the comparison between the actual costs and the established budget in order to promptly identify foreseeable cost overruns, if any exist, - illustrating the comparison between actual timelines and those planned and revised, if applicable, in order to determine whether the project is delayed or on schedule, - explaining the reason for differences that may have been noted in the magnitude of the costs or adherence to the timelines mentioned above, - presenting the percentage of progress of the work, - reporting on the anticipated problems, risks or major issues that could have negative repercussions on the budget, the schedule and the expected deliverables, as well as proposed alternatives to mitigate the effect of the problematic causes detected, - describing the assessment of the overall status of the project (e.g., "under control," "ahead of schedule" or "critical"); o establish the work tool to be used to produce these management reports illustrating the status of projects under way. Auditor General of the Ville de Montr?al 316 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) Business unit's response: [TRANSLATION] The "project sheet" developed will have to indicate the level and frequency of reporting taking into account the project in question. (Completed, March 2013) Development of standards and procedures specifying the objectives and accountability reporting required and adapted to the various authorities (management committee, audit and TCEP committee, board of directors, executive committee). (Planned completion: May 2013) Auditor General of the Ville de Montr?al 317 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 4. Appendices 4.1. Sample of Contracts Examined Table A - Details Regarding the Contracts in our Sample Contract No. Purpose of the contract Date of approval by the BODa Type of contract $1,404,391 2010-05-11 Execution of work $571,373 2010-10-28 Execution of work $474,221 2011-04-13 Supply of materials and execution of work $263,303 2011-04-27 Professional services $396,459 2011-04-27 Execution of work $439,978 2011-06-09 Execution of work $9,197,746 2011-10-12 Professional services and execution of work $1,087,796 2011-11-09 Execution of work $1,065,199 2011-11-09 Supply of materials and execution of work 2012-03-29 Execution of work Amount (taxes included) 2010 1 2 Construction of a glassed-in staircase and elevator at the arsenal of the Fort de l'?le Sainte-H?l?ne- Stewart Museum Expansion work (H?l?ne-de-Champlain building renovation and expansion project) 2011 3 4 5 6 7 8 9 Supply and installation of signage elements - phase 3 Construction management (H?l?ne-de-Champlain building renovation and expansion project) Repair of the masonry in the powder magazine Interior demolition work (H?l?ne-de-Champlain building renovation and expansion project) Design and construction of a new service courtyard Landscaping and civil engineering (H?l?ne-de-Champlain building renovation and expansion project) Supply and installation of windows and doors (H?l?ne-de-Champlain building renovation and expansion project) 2012 10 a Expansion work - Jamaica pavilion $168,638 Board of directors. Auditor General of the Ville de Montr?al 318 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) 4.2. Summary of the Contracts Examined that Received Additional Funds Table B - Contracts Related to the H?l?ne-de-Champlain Renovation and Expansion Project Contract No. according to Appendix 4.1 Expansion work Original cost of the awarded contract (taxes included) 2010-10-28 2 Purpose of the contract Date of approval by the BODa $571,373 2010-12-02 $451,854 $571,373 Difference Construction management 4 2011-04-27 $341,775 $263,303 $341,775 $605,078 (130% more) 2011-06-09 $439,978 2012-03-29 Total $413,390 $439,978 Difference a $1,174,457 $263,303 2011-11-09 Difference Interior demolition work $603,084 (106% more) Total 6 Grand total (taxes included) $151,230 2011-06-09 Total Additional funds granted by the BODa (taxes included) $413,390 $853,368 (94% more) Board of directors. Table C - Contracts Related to the Repair of the Masonry in the Powder Magazine Contract No. according to Appendix 4.1 5 Purpose of the contract Repair of the masonry in the powder magazine Date of approval by the BODa Original cost of the awarded contract (taxes included) 2011-04-27 $396,459 2011-04-27 $20,100 $396,459 $79,568 (20% more) $476,027 Board of directors. Note 1: 12 Grand total (taxes included) $59,468 (Note 1) Total Difference a Additional funds granted by the BODa (taxes included) 5% in additional funds granted by the executive director, in accordance with the purchase requisition approval levels in the SPJD procurement policy, which stipulates: [TRANSLATION] "Senior management is authorized to proceed with unforeseen additional expenses not exceeding 10% of the awarded contracts and mandates. These additional 12 expenses must not, however, exceed $100,000 per contract or per mandated awarded ." Procurement policy, Soci?t? du parc Jean-Drapeau, p. 6. Auditor General of the Ville de Montr?al 319 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 4.3. Difference Between the Amount of the Awarded Contracts 13 and the Cost Estimates Presented in the Decision-Making Summary and Minutes of the Proceedings of the Soci?t? du parc Jean-Drapeau Table D - Contracts for which the Difference Between the Amount of the Awarded Contracts and the Cost Estimate is Less than ? 10% Contract No. according to Appendix 4.1 Purpose of the contract 10 Expansion work - Jamaica pavilion 7 Design and construction of a new service courtyard Amount of the awarded contract (taxes included) Cost estimate in the decision-making summary and minutes (taxes included) $168,638 $186,678 $9,197,746 $8,500,000 Difference ($18,040) (9.7%) $697,746 8.2% Table E - Contracts for which the Difference Between the Amount of the Awarded Contracts and the Cost Estimate is More than ? 10% Purpose of the contract 2 Expansion work (H?l?ne-de-Champlain building renovation and expansion project) $571,373 $998,944 ($427,571) (42.8%) 5 Repair of the masonry in the powder magazine $396,459 $626,588 ($230,129) (36.7%) 4 Construction management (H?l?ne-de-Champlain building renovation and expansion project) $263,303 $227,850 $35,453 15.6% 6 Interior demolition work (H?l?ne-de-Champlain building renovation and expansion project) $439,978 $368,000 $71,978 19.6% 8 Landscaping and civil engineering (H?l?ne-de-Champlain building renovation and expansion project) $1,087,796 $800,000 $287,796 36.0% 3 Supply and installation of signage elements-phase 3 $474,221 $273,420 $200,801 73.4% a 13 Amount of the awarded contract (taxes included) Cost estimate in the decision-making summary and minutes (taxes included) Contract No. according to Appendix 4.1 a Difference This amount was found by performing a reconciliation since it was included in an overall estimate of $1,525,000 presented in the minutes of the November 9, 2011, meeting of the SPJD and is attributable to the approval by the SPJD board of directors of the awarding of a batch of five contracts for which the overall cost quoted by the lowest bidders was $1,590,268. This cost corresponds to the original amount of the retained bid and does not take into account the additional funds awarded. Auditor General of the Ville de Montr?al 320 2012 Annual Report 5.7. Contract Awarding and Management Process (Soci?t? du parc Jean-Drapeau) Table F - Contracts Whose Cost Estimate is not Specifically Indicated in the Decision-Making Summary and Minutes Purpose of the contract Amount of the awarded contract (taxes included) Cost estimate in the decision-making summary and minutes (taxes included) 1 Construction of a glassed-in staircase and elevator at the arsenal of the Fort de l'?le SainteH?l?ne-Stewart Museum $1,404,391 n/a n/a n/a 9 Supply and installation of windows and doors (H?l?ne-de-Champlain building renovation and expansion project) $1,065,199 n/a n/a n/a Contract No. according to Appendix 4.1 Difference N/a: not available. Auditor General of the Ville de Montr?al 321 2012 Annual Report Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 Residential Swimming Pool Safety Regulations 5.8 5.8. Residential Swimming Pool Safety Regulations Table of Contents 1. Introduction ................................................................................................................... 327 2. Audit Scope................................................................................................................... 328 3. Findings and Recommendations .................................................................................. 328 3.1. Compliance of the Permit-Issuing Process and Enforcement of Regulatory Provisions ........................................................................................... 328 3.2. Mechanisms for Identifying Non-Compliant Installations....................................... 345 3.3. Follow-Up on Notices of Non-Compliance ............................................................ 347 3.4. Mechanisms for Raising Public Awareness .......................................................... 356 3.5. Accountability ........................................................................................................ 362 Auditor General of the Ville de Montr?al 325 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit List of Acronyms DAUSE Direction de l'am?nagement urbain et des services aux entreprises MELS Minist?re de l'?ducation, du Loisir et du Sport Auditor General of the Ville de Montr?al SGIS 326 spatial geographic information system 2012 Annual Report 5.8. Residential Swimming Pool Safety Regulations 5.8. Residential Swimming Pool Safety Regulations 1. Introduction A 2010 report 1 produced by the Minist?re de l'?ducation, du Loisir et du Sport (MELS) indicates that between 2000 and 2008 there were 80 residential swimming pool drownings in Qu?bec, or an average of nine per year. This type of drowning represents 12% of all water-related deaths and does not include the many near-drownings requiring hospitalization that occur every year. Statistics put the number of residential swimming pools in the province at close to 300,000 during this period. According to a count done by the Ville de Montr?al (the city) in 2004, there were nearly 33,000 residential swimming pools in Montr?al, which was made up of 27 boroughs at the time. Nearly a third of residential swimming pool drowning victims are children under 5 years of age. The main cause of these deaths is the lack of appropriate safety equipment around the pool. This is exacerbated by the fact that, in most of these cases, children are left unattended near the water while the person supervising them is momentarily absent or distracted. In an attempt to reduce the number of residential swimming pool drownings, the Government of Qu?bec passed the Residential Swimming Pool Safety Act 2 (the Act) on October 25, 2007. Pursuant to its powers under this new Act, the Government then adopted the Residential Swimming Pool Safety Regulation 3 (the Provincial Regulation) on June 23, 2010. It took effect on July 22, 2010. This Provincial Regulation sets forth a series of standards aimed at increasing the safety of these installations, including the pool itself (e.g., minimum height of pool walls, presence of ladders equipped with a safety gate), the enclosure to restrict pool access and the features and location of all devices used in pool operations. Since the Provincial Regulation came into force on July 22, 2010, all Montr?al residents have been required to obtain a municipal permit to build, install or replace a swimming pool or to erect a construction allowing or preventing access to a swimming pool (section 9 of the Provincial Regulation). Under the Act, municipalities are responsible for ensuring compliance with this regulation. In Montr?al, this responsibility falls on the boroughs, which 1 2 3 Faits saillants sur les noyades et les autres d?c?s li?s ? l'eau au Qu?bec de 2000 ? 2008 - ?dition 2010, Trois-Rivi?res, MELS - Direction de la promotion de la s?curit?, 2010, 19 pages. RSQ, chapter S-3.1.02. Gazette officielle du Qu?bec. Part 2, Vol. 142, No. 27, July 7, 2010, pp. 2805-7. Auditor General of the Ville de Montr?al 327 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit under section 131 of the Charter of Ville de Montr?al 4 hold the authority to issue certificates of authorization and permits related to swimming pools. 2. Audit Scope The purpose of our audit was to review the measures undertaken by the city business units responsible for ensuring compliance with the regulatory and legislative provisions governing residential swimming pool safety. Furthermore, we also sought to examine the preventive measures adopted by these business units to raise residential pool owners' awareness about the safety standards prescribed in the Provincial Regulation. The audit, which began in the fall of 2012, dealt primarily with 2011 and 2012, but it also took into account information sent and communicated to us up until February 22, 2013. For certain aspects, data from previous years were also considered. Our audit focused on three boroughs: o Ahuntsic-Cartierville borough: Direction du d?veloppement du territoire; o Pierrefonds-Roxboro borough: Direction de l'am?nagement urbain et des services aux entreprises (DAUSE); o Rivi?re-des-Prairies-Pointe-aux-Trembles borough: Direction de l'am?nagement urbain et des services aux entreprises (DAUSE). 3. Findings and Recommendations 3.1. Compliance of the Permit-Issuing Process and Enforcement of Regulatory Provisions To comply with the standards outlined in the Provincial Regulation, residents must apply to their borough for a permit 5 to build, install or replace a swimming pool or to erect a construction allowing or preventing access to a swimming pool. The Provincial Regulation concerns any permanent or temporary artificially constructed outdoor basin intended for swimming that has a water depth equal to or greater than 60 centimetres and to which the Regulation respecting safety in public baths 6 does not 4 5 6 RSQ, chapter 11.4. For the purposes of this audit report, we will use the term "permit" as specified in the Provincial Regulation, although Pierrefonds-Roxboro and Rivi?re-des-Prairies-Pointe-aux-Trembles boroughs use the term "certificate of authorization." RRQ, chapter S-3, r. 3. Auditor General of the Ville de Montr?al 328 2012 Annual Report 5.8. Residential Swimming Pool Safety Regulations apply. It excludes whirlpools and hot tubs whose capacity does not exceed 2,000 litres. This covers the following categories: o Aboveground pool: a hard-sided swimming pool installed permanently on the ground surface; o Portable pool: a soft-sided swimming pool, inflatable or not, designed to be installed on a temporary basis; o Inground or semi-inground pool: an artificially constructed outdoor basin intended for swimming that is partially or completely buried under the ground surface. The Provincial Regulation does not apply to pools installed before the date it came into force (July 22, 2010) or pools acquired but not installed before this date, provided that they were installed no later than October 31, 2010. When any such pool is replaced, the new pool must adhere to the standards in the Provincial Regulation. The Provincial Regulation also establishes fines for swimming pool owners in breach of a provision. The safety standards set out in the Provincial Regulation aim to control and protect access to swimming pools. They include the following: o All inground and semi-inground pools must be equipped with a ladder or steps to enter and exit the water; o All swimming pools must be surrounded by an enclosure to restrict access. This enclosure must: - prevent the passage of a spherical object 10 centimetres in diameter, - be at least 1.2 metres high, - have no fixture, projection or open parts enabling it to be climbed; o All gates that form part of an enclosure must be equipped with a self-closing and selflatching passive security device installed on the inside of the enclosure in the upper part of the gate; o Aboveground pools must have a wall height of at least 1.2 metres from the ground at any point, and portable pools must have a wall height of 1.4 metres or more; o Any device linked to the operation of a swimming pool must be installed more than 1 metre away from the pool wall. Furthermore, the pipes linking the device to the swimming pool must be flexible and not be installed in a way that facilitates the climbing of the pool wall or the enclosure, as the case may be. As a result, prior to issuing a permit, borough authorities must obtain all the information that confirms that an installation will be in compliance with the provisions of the Provincial Regulation or a stricter borough standard, if one is in place. Well before new legislative and regulatory provisions concerning residential swimming pool safety were implemented, a number of boroughs had already adopted certain safety measures, many of which were Auditor General of the Ville de Montr?al 329 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit covered in borough by-laws on fences. When the new Provincial Regulation came into force in 2010, it contributed to enhancing the safety of residential swimming pool installations, but section 3 of the Act does not prevent a borough from introducing standards that are even more stringent. The following pages contain the results of our audit with respect to compliance with the permit-issuing process. For each of the boroughs we visited, our observations are based on a review of a sample of four swimming pool permits issued in 2011 and 2012 (two permits each year). We deliberately chose two permits issued for inground pools and two permits for aboveground pools to better assess the degree of compliance with the regulatory requirements specific to each. 3.1.1. Ahuntsic-Cartierville Borough 3.1.1.1. Processing of Permit Applications and Existing By-Laws 3.1.1.1.A. Background and Findings It is important to point out that Ahuntsic-Cartierville borough has not modified its by-laws for issuing permits and that the term "permit" has been retained. This borough adheres to the provisions of the By-law concerning the construction and conversion of buildings. 7 More specifically, article 34 of this by-law indicates the information and documentation that must be submitted with a permit application (e.g., description of the planned work, estimated value of the work, drawings and location certificate). Accordingly, in this borough, residents who wish to install a swimming pool must first submit a permit application to the borough office, along with all the required documents. A permit clerk with the Division urbanisme, permis et inspection of the Direction du d?veloppement du territoire is responsible for administering the permit-issuing process by: o Ensuring that all the required documents are included with the application; o Reviewing the permit application to confirm whether the proposed installations adhere to the Residential Swimming Pool Safety Regulation or other stricter standards contained in a by-law (e.g., the fence by-law); o Entering the application in the computer system used for this purpose, the Gestion du territoire - Permis system; 7 City council, by-law 11-018, October 24, 2011. Auditor General of the Ville de Montr?al 330 2012 Annual Report 5.8. Residential Swimming Pool Safety Regulations o o Determining the cost of the permit, in accordance with the list of rates in effect; Issuing the permit once the file is in order and the corresponding fees have been paid. Once the permit has been issued, an inspector is assigned to the file and dispatched to visit the premises to ensure that the work related to the installation of the swimming pool is compliant. The inspector then enters this information into the Gestion du territoire - Permis computer system. If everything is found to be acceptable, the file is closed. However, if any of the components do not meet regulatory requirements, the inspector must: o Provide a notice of non-compliance, informing the swimming pool owner of the corrective measures to implement to become compliant within a set timeframe; o Follow up to ensure the corrective action stipulated in the notice of non-compliance is carried out; o Initiate the procedures involved in issuing a statement of violation in the case of an uncooperative owner. In addition to the Provincial Regulation, the borough must enforce the provisions of its bylaw on fences as they apply to swimming pool safety. It is therefore more laborious to ensure adherence to both sets of provisions in their entirety. Our audit revealed that most of the provisions in the Provincial Regulation have been integrated into the borough's fence by-law. Incidentally, the fall 2011/winter 2012 edition of the borough's newsletter 8 published a report on the new fence by-law, 9 which was adopted in response to the new provincial standards for residential swimming pool safety. Although this is a legitimate decision, we observed when comparing the Provincial Regulation and the borough's new fence by-law that some provisions of the former had not been incorporated into the latter. For one, section 2 of the Provincial Regulation, which specifies that all "inground and semi-inground pools must be equipped with a ladder or steps used to enter or exit the water," is left out of the fence by-law. The same applies to section 8 of the Provincial Regulation: "Every installation intended to allow or prevent access to the swimming pool must be kept in good working order." Based on the information obtained from the people with whom we met, this seems to be an oversight. Consequently, in order to avoid any potential confusion for borough employees and residents alike on matters related to the regulations in effect, we believe that corrective action is called for. 8 9 [TRANSLATION] Ahuntsic-Cartierville Newsletter, Fall 2011/Winter 2012, Vol. 8, No. 2, p. 25. Borough council, by-law RCA11 09008 entitled By-law modifying the by-law concerning fences (Consolidated by-laws of the Ville de Montr?al, chapter C-5) concerning the borough of Ahuntsic-Cartierville, October 3, 2011. Auditor General of the Ville de Montr?al 331 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.1.1.1.B. Recommendation We recommend, should the Direction du d?veloppement du territoire in AhuntsicCartierville borough still wish to use a single set of provisions on residential swimming pool safety, that it take the necessary measures to ensure the borough's fence by-law incorporates all of the provisions contained in the Residential Swimming Pool Safety Regulation to avoid any confusion or oversight in terms of compliance. Business unit's response: [TRANSLATION] The [TRANSLATION] Urban planning by-law for Ahuntsic-Cartierville borough 10 is currently being updated (first reading and adoption by the borough council expected in May 2013) to incorporate a provision on devices that allow users to enter and exit swimming pools, which could not be incorporated into the already amended fence by-law because the devices are not fences per se. The provision dealing with the maintenance of pool components (including fences) is currently covered in the By-law concerning the construction and conversion of buildings, which gives our inspectors a wide berth for taking action in circumstances where insufficient maintenance poses a public safety threat. All of the aspects of the Provincial Regulation will therefore be covered at the municipal level once the amended [TRANSLATION] Urban planning by-law for Ahuntsic-Cartierville borough is adopted. It is important to note that, in the meantime, since our inspectors are authorized to enforce the Provincial Regulation, public safety has never been compromised. (Planned completion: May 2013) 3.1.1.2. Enforcement of Regulatory Provisions 3.1.1.2.A. Background and Findings Regarding the process leading up to the issuance of a permit by the permit clerk, our review of the files on the four permits in our sample shows that the information was not always compiled in a way that makes it possible to trace the evidence showing that compliance with the safety standards outlined in the Provincial Regulation was checked. In fact, only one of the four files was complete in this regard. Given that a permit is issued before swimming pool installation work is done, it is important to obtain all the documentation and information necessary for the permit clerk to analyze 10 Borough council, by-law 01-274, August 31, 2012. Auditor General of the Ville de Montr?al 332 2012 Annual Report 5.8. Residential Swimming Pool Safety Regulations the file and carry out the required checks to ensure compliance with the applicable regulations. These checks are all the more important in that they make it possible to notify residents in a timely manner of any violations observed before work gets under way and thus save them the unnecessary expense of subsequent corrective measures. Accordingly, we feel it is advisable to compile the documentation in a way that supports all of the regulatory aspects that have been checked before the permit is issued. In this regard, we observed in two of the four files we examined that a regulatory analysis sheet had been used by one of the clerks to assess the application. The title on the sheet reads [TRANSLATION] "POOLS 01-274 Section 347." It features a checklist with the various regulatory requirements related to aspects such as the pool location (e.g., presence of overhead wires, easements, 1-metre buffer between the side of the pool and the fence), fences (e.g., 1.2-metre minimum height) and devices (e.g., location of a heat pump in relation to the lot line). We nevertheless noticed that the sheet did not bear the borough logo and was not signed by the employee who filled it out. Moreover, it was not necessarily up to date when the corresponding permits were issued. In particular, we noted that certain fence-related items were based on outdated standards. According to the information obtained from the people with whom we met, it appears that this analysis sheet is a personal work tool developed by one of the employees and in no way represents a formal practice in place at the borough's Direction du d?veloppement du territoire. For the two other files in our sample, although the permit clerk created a file using the Gestion du territoire - Permis computer system, as is the case for all issued permits, this file did not contain any evidence or supporting documentation that confirmed that the necessary checks had been done and that the application was complete and in order. Although the analysis sheet in question was not necessarily comprehensive and up to date in terms of the regulatory standards to be checked, it is still a very useful work tool that would enable clerks to ensure they have reviewed all of the applicable regulatory provisions and show the work done before the requested permit is issued. The sheet could even be enhanced with the addition of a list of the required supporting documents. We believe that the borough would benefit from adopting an official regulatory analysis sheet covering all of the standards related to the safety of residential swimming pools and having all of the employees involved in the process use it. In addition, since the inspector, like the permit clerk, must refer to the content of the regulatory provisions in effect to ensure the compliance of on-site installations, we feel that an analysis sheet (a checklist) would be just as useful and relevant, enabling the inspector Auditor General of the Ville de Montr?al 333 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit to document and record the evidence of the various items inspected, as well as conclusions as to whether or not installations are in compliance. At this stage in the process, we also audited the fees charged for obtaining a permit to determine whether they matched the rates in effect and to ensure that they had been collected before a permit was issued. The borough calculates the corresponding amounts based on the city's by-law on fees. 11 We observed that these fees were properly collected on the same day as the permit applications were submitted and that they were consistent with the list of rates in effect. 3.1.1.2.B. Recommendation We recommend that the Direction du d?veloppement du territoire in AhuntsicCartierville borough take the necessary measures to officially implement a regulatory analysis sheet (checklist) that: o Lists all the applicable standards related to residential swimming pool safety and possibly the supporting documents required for the permit application; o Is used by all the employees involved in the permit application process and systematically kept on file; so that permit clerks and inspectors can document and keep on file evidence of the various aspects that have been checked or inspected, as well as their conclusions as to whether or not installations are compliant. Business unit's response: [TRANSLATION] A checklist has been produced by an architectural technician. It is currently being reviewed by the senior permit clerk and the head of the Section de l'inspection. The layout will be finalized and the checklist incorporated into the standard practices of permit clerks and inspectors by May 1, 2013. After an inspection is carried out, the completed checklist will be scanned and incorporated into the electronic permit system record. (Planned completion: May 2013) 11 In 2011: By-law concerning fees (Fiscal 2011) (10-029), city council, December 20, 2010. In 2012: By-law concerning fees (Fiscal 2012) (11-037), city council, December 19, 2011. Auditor General of the Ville de Montr?al 334 2012 Annual Report 5.8. Residential Swimming Pool Safety Regulations 3.1.1.3. Installation Inspection 3.1.1.3.A. Background and Findings After the permit has been issued, the next step in the process is to assign an inspector to visit the premises and ensure that the swimming pool installation is compliant. We therefore extended the scope of our audit of the four permits in our sample to include the information in the records contained in the computer system (Gestion du territoire - Permis) concerning this step. The results of our analysis show that, for two of the four selected files, the first inspection was conducted between 26 and 58 days following the date the permit was issued. However, for the two other files, we noted that, in one case, it took 135 days (nearly 4.5 months) before the first inspection was carried out. In the other case, as of the date of our audit (November 28, 2012), there was no evidence in the file that the inspector had visited the premises for a project for which a permit had been issued on May 23, 2013 (189 days or a little over 6 months before). In addition, although it is possible to enter the estimated start and end date of the pool installation work in the computer system, we noticed that this information was not included in any of the files. This makes it more difficult to determine a suitable moment for the on-site inspection. We are aware that various circumstances, sometimes out of the borough's control, can lead to delays (e.g., when a resident does not proceed with an installation right away). However, we consider that following up on the work after the permit has been issued is an important step that should be carried out as quickly as possible to ensure that installations are safe and comply with the necessary regulatory provisions, with a view to minimizing drowning risks. Accordingly, we feel it would be advisable for the administration to use a tool it deems appropriate to track the progress of required inspections once a permit has been issued so that these activities are not overlooked and non-compliant installations are identified in a timely manner. 3.1.1.3.B. Recommendation We recommend that the Direction du d?veloppement du territoire in AhuntsicCartierville borough determine the tool to be used to keep track of the inspections that need to be carried out once a permit has been issued so that these activities are not overlooked and non-compliant installations can be identified in a timely manner. Auditor General of the Ville de Montr?al 335 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Business unit's response: The checklist mentioned in the response to the [TRANSLATION] recommendation 3.1.2.2.B will ensure compliance of all installations throughout the permit issuance and on-site inspection process. (Planned completion: May 2013) 3.1.2. Pierrefonds-Roxboro Borough 3.1.2.1. Processing of Permit Applications and Existing By-Laws 3.1.2.1.A. Background and Findings Pierrefonds-Roxboro borough residents who wish to install a residential swimming pool must apply to the borough office for a permit. This application is processed by a permit clerk working for the Division de la construction et de l'occupation of the DAUSE. The clerk must: o Ensure all the required application documents have been provided by the applicant; o Enter the application in the computer system used for this purpose (Gestion du territoire - Permis). Once the file is considered complete, it is sent to a DAUSE inspector whose conclusions will also be incorporated into the Gestion du territoire - Permis system. The inspector is responsible for: o Reviewing the permit application and confirming whether the proposed installations adhere to the Residential Swimming Pool Safety Regulation or other standards outlined in a borough by-law (e.g., the Zoning by-law 12 ); o Determining the cost of the permit, in accordance with the list of rates in effect; o Issuing the permit once the file is in order and the corresponding fees have been paid by the applicant; o Inspecting the premises, after the permit is issued, to ensure that the installation work is compliant; o Closing the file once everything is in order; otherwise: - issuing a notice of non-compliance, informing the swimming pool owner of the corrective action to be taken within an established timeframe, - following up to ensure the corrective action stipulated in the notice of noncompliance is carried out, - initiating, as necessary, the procedures involved in issuing a statement of violation to any uncooperative owners. 12 Borough council, by-law CA29 0040, June 7, 2010. Auditor General of the Ville de Montr?al 336 2012 Annual Report 5.8. Residential Swimming Pool Safety Regulations In addition to the Provincial Regulation, there are specific provisions contained in the borough's Zoning by-law 13 on such aspects as fence height and the location of mechanical equipment (filters, heaters, etc.). Regarding the documentation and information to be provided when applying for a permit to install a residential swimming pool, there are two by-laws to take into account. If the person filing for the permit is a resident of Pierrefonds, the By-law concerning permits and certificates 14 applies. If the person is a resident of Roxboro, the process is subject to the city's By-law concerning the construction and conversion of buildings. 15 Although there is a By-law concerning permits and certificates 16 in place in Roxboro, it refers to the city by-law (No. 11-018) for all applications involving a swimming pool. In comparing these by-laws, we noted that in addition to the usual information (e.g., description of the planned work, estimated value of the work, drawings), the Pierrefonds by-law (No. 1051, article 42) is more demanding in terms of the information to be provided by the pool owner. For example, article 42, paragraph 4 indicates that applications must be accompanied by the following: o the type of filtration and chlorination, the pipes as well as the manufacturer's recommendations; o the diameter and location of wastewater draining pipes. Under these by-laws, there are therefore two sets of procedures at play for residents of the same borough. Therefore, in order to avoid any potential confusion in the enforcement of these provisions and ensure that all residents of the borough receive equal treatment, we believe that the permits and certificates by-laws for Pierrefonds (No. 1051) and Roxboro (No. 93-558) should be harmonized. 3.1.2.1.B. Recommendation We recommend that the Direction de l'am?nagement urbain et des services aux entreprises in Pierrefonds-Roxboro borough undertake the necessary steps to harmonize the provisions in the permits and certificates by-laws currently in effect in Pierrefonds (No. 1051) and Roxboro (No. 93-558) in order to avoid potential confusion in their enforcement and ensure that all borough residents receive equal treatment. 13 14 15 16 Borough council, by-law CA29 0040, June 7, 2010, articles 138, 143, 145 and 146. Borough council, by-law 1051, October 10, 1989, article 42. City council, by-law 11-018, October 24, 2011, article 34. Borough council, by-law 93-558, April 5, 1993. Auditor General of the Ville de Montr?al 337 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Business unit's response: [TRANSLATION] Harmonization of Pierrefonds and Roxboro's permits and certificates by-laws: they will be repealed and replaced by the administration by-law, so that implementation rules are identical across the borough. (Planned completion: August 2013) 3.1.2.2. Enforcement of Regulatory Provisions 3.1.2.2.A. Background and Findings In an attempt to make it easier to administer the regulatory provisions related to residential swimming pool safety, we observed that the DAUSE required the use of an analysis sheet bearing the title [TRANSLATION] "Pool Inspection," featuring a checklist for each of the applicable items derived from the Provincial Regulation as well as the borough's Zoning bylaw. Based on the information we obtained and our analysis, we determined that this sheet is being used primarily when inspectors visit the premises, i.e., after a permit has been issued, to check that installations are in order. The information on this sheet includes the owner's address, type of swimming pool, name of the inspector, date of inspection and whether or not a notice of non-compliance has been issued and, if so, the amount of time granted to the owner to carry out the required corrective action. This analysis sheet strikes us as being a very relevant and useful tool. During our audit of the four files in our sample, however, we were not always able to find evidence that it was being used by the inspectors assigned to the files. In fact, in three out of the four files we examined, the analysis sheet was not used. Furthermore, in order to be able to assess the comprehensiveness of the regulatory standards that appear in the analysis sheet to be used, we compared these standards with those contained in the Provincial Regulation as well as those in the borough's Zoning bylaw. This analysis revealed that all of the standards had been integrated into the checklist with the exception of two aspects of the Provincial Regulation: o Article 7, paragraph 2: "The pipes linking the device to the swimming pool must be flexible and not be installed in a way that facilitates the climbing of the pool wall or, as the case may be, the enclosure." o Article 9, paragraph 3: "During the time of the work, the person to whom the permit [...] is issued must take any temporary measures to control access to the pool. [...]" Auditor General of the Ville de Montr?al 338 2012 Annual Report 5.8. Residential Swimming Pool Safety Regulations Furthermore, in reviewing the files, we observed that there was not always evidence that the information and documentation that were supposed to be analyzed had all been received before the permit was issued. We feel it is important that all the documentation and information required to analyze and confirm whether installations comply with requirements be received before a permit is issued. Accordingly, we believe it may be useful to remind those involved in the process about the importance of using the [TRANSLATION] "Pool Inspection" analysis sheet and improving it further to include a section containing a list of all the information and documentation (drawings, description of the planned work, estimated value of the work, manufacturer's documentation) that must accompany the permit application. This would enable the permit clerk to ensure from the start of the process that the file is complete so that the inspector can then assess the application with a view to issuing the permit and, as appropriate, promptly notify residents of any violations observed before work gets under way and thus save them the unnecessary expense of subsequent corrective measures. 3.1.2.2.B. Recommendation We recommend that the Direction de l'am?nagement urbain et des services aux entreprises in Pierrefonds-Roxboro borough take the necessary measures to ensure the [TRANSLATION] "Pool Inspection" analysis sheet is: o Reviewed to incorporate all of the provisions in the Residential Swimming Pool Safety Regulation; o Improved, by inserting a section containing a list of all the information and documentation that must accompany a permit application; o used by all of the employees involved in the process and systematically kept in the corresponding file; so that employees involved in the swimming pool permit application and issuance process can keep track of what has been checked and note their findings as to whether or not the designated installations are in compliance. Business unit's response: [TRANSLATION] The standards outlined in article 7, paragraph 2, and article 9, paragraph 3, of the Residential Swimming Pool Safety Regulation will be added to the checklist used by inspectors. (Planned completion: May 2013) Once the administration by-law has been adopted, the documentation requirements will be reviewed and harmonized on a borough-wide basis. A section will also be added to the analysis sheet to incorporate the required information. (Planned completion: August 2013) Auditor General of the Ville de Montr?al 339 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Inspectors will also use and fill out the analysis sheet as they review the application for a certificate of authorization in order to ensure project compliance. We will continue to fill out an analysis sheet for each inspection. (Planned completion: May 2013) 3.1.2.3. Installation Inspection 3.1.2.3.A. Background and Findings The final stage in the process in the on-site inspection, during which an inspector ensures that the work involved in installing the swimming pool is in order after the permit is issued. Our audit of the selected files in this matter revealed the following: o For one of the four files, for which a permit was issued on April 25, 2012, we did not find evidence of any kind of on-site inspection, even though our audit took place on February 12, 2013 (i.e., 293 days, or almost 10 months, later); o For the three other files we analyzed, we noted that the inspection was performed within a timeframe varying between 36 and 137 days (4.5 months). In addition, although the computer system being used is programmed to record the planned start and end date for the swimming pool installation work, we noted that this feature was not used in two out of the four files. As a result, it is more difficult to plan an appropriate time for the inspector to visit the premises. We are aware that various circumstances, sometimes out of the borough's control, can lead to delays (e.g., when a resident does not proceed with an installation right away). However, we consider that following up on the work after the permit has been issued is an important step that should be carried out as quickly as possible to ensure that installations are safe and comply with the necessary regulatory provisions, but mostly with a view to minimizing drowning risks. Accordingly, we feel it would be advisable for the DAUSE to use a tool it deems appropriate to track the required inspections once a permit has been issued so that these activities are not overlooked and non-compliant installations are identified in a timely manner. Finally, we also reviewed the fees for obtaining permits to ensure that they were consistent with the list of rates in effect and that they were collected before the corresponding permits were issued. We used the borough's by-law on price determination 17 for this purpose. Our 17 For 2011: By-law on the price determination of various goods, activities and municipal services for the fiscal year 2011, CA 29 0050, borough council, December 6, 2010. For 2012: By-law on the price determination of various goods, activities and municipal services for the fiscal year 2012, CA 29 0058, borough council, December 5, 2011. Auditor General of the Ville de Montr?al 340 2012 Annual Report 5.8. Residential Swimming Pool Safety Regulations audit showed that all fees had been paid on the day the applications were submitted and matched the amounts indicated in the list of rates. 3.1.2.3.B. Recommendation We recommend that the Direction de l'am?nagement urbain et des services aux entreprises in Pierrefonds-Roxboro borough determine the tool to be used to keep track of inspections that need to be carried out after a permit is issued to ensure these activities are not overlooked and non-compliant installations are detected in a timely manner. Business unit's response: [TRANSLATION] The Oracle permit system is the tool of choice for tracking certificates of authorization issued and for producing reports. The Excel spreadsheet allows us to produce a detailed report to follow up on the status of statements of violation in circulation and to be issued. (Planned completion: May 2013) 3.1.3. Rivi?re-des-Prairies-Pointe-aux-Trembles Borough 3.1.3.1. Processing of Permit Applications and Existing By-Laws 3.1.3.1.A. Background and Findings One of the borough's by-laws, namely the [TRANSLATION] By-law concerning certificates of occupancy and certain permits in the borough of Rivi?re-des-Prairies-Pointe-auxTrembles, 18 specifies that residents who wish to build or install a swimming pool must submit a permit application, along with the required information and documentation, to the borough office. A permit clerk with the DAUSE's Division des permis et des inspections is responsible for processing the application, reviewing the file and issuing the permit. This entails: o Ensuring all the required application paperwork is received; o Reviewing the permit application and confirming whether the proposed installations adhere to regulations concerning residential swimming pool safety or other provisions set forth in the [TRANSLATION] Zoning by-law for the borough of Rivi?re-des-Prairies- Pointe-aux-Trembles 19 ; o Entering the application into the Gestion du territoire - Permis computer system; 18 19 Borough council, by-law RCA09-C01, October 1, 2009. Borough council, by-law RCA09-Z01, September 4, 2009. Auditor General of the Ville de Montr?al 341 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit o o Determining the cost of the permit, in accordance with the list of rates in effect; Issuing the permit once the file is in order and the corresponding fees have been paid. Once the permit has been issued, an inspector with the DAUSE is assigned to visit the premises and check whether or not the swimming pool installation work is in order. The inspector then enters this information into the Gestion du territoire - Permis system. If everything is found to be acceptable, the file is closed. If any of the components do not meet regulatory requirements, the inspector must: o Provide a notice of non-compliance, informing the swimming pool owner of the corrective action to be taken within a set timeframe; o Follow up to ensure the corrective action stipulated in the notice of non-compliance is carried out; o Initiate the procedures involved in issuing a statement of violation in the case of an uncooperative owner. 3.1.3.2. Enforcement of Regulatory Provisions 3.1.3.2.A. Background and Findings We observed that the DAUSE required the use of an analysis sheet entitled [TRANSLATION] "Regulatory Analysis Sheet - Review/Inspection" to make it easier to track compliance with the regulatory provisions related to residential swimming pool safety. This sheet includes a section featuring a list of information and documentation (e.g., drawings, location certificate, description of planned work) that must be submitted with a permit application. This makes it easier for the permit clerk to make sure the file is complete so that it can be reviewed and, ultimately, a permit can be issued. It also features a checklist with most of the standards covered in the borough's zoning by-law and the Provincial Regulation, with the exception of the following: o Section 8: "Every installation intended to allow or prevent access to the swimming pool must be kept in good working order." o Section 9, paragraph 3: "During the time of the work, the person to whom the permit [...] is issued must take any temporary measures to control access to the pool [...]" This sheet is also used by the inspector assigned to a file to assess the compliance of the swimming pool installation work and is designed to accommodate the inspector's input. We feel that this sheet is a very relevant and useful tool. We ascertained that it was used in all four of the files we reviewed, although in one case it was used by the inspector for an Auditor General of the Ville de Montr?al 342 2012 Annual Report 5.8. Residential Swimming Pool Safety Regulations on-site visit but not by the permit clerk who processed the initial application and issued the permit. Apart from that file, for which we saw no evidence that that the permit clerk had checked the required information before issuing the permit, the three other files were found to be complete. We then reviewed the permit application fees to ensure that the amounts charged were the same as the amounts indicated in the list of rates in effect and that they had been collected before the permits were issued. The borough's by-law on rates 20 was used to validate this information. Our audit showed that all fees were collected the day permit applications were submitted and were consistent with the list of rates in effect. 3.1.3.2.B. Recommendation We recommend that the Direction de l'am?nagement urbain et des services aux entreprises in Rivi?re-des-Prairies-Pointe-aux-Trembles borough take the necessary measures so that employees involved in the permit application process can document what they have checked and enter their conclusions as to whether or not installations are in compliance by ensuring that the [TRANSLATION] "Regulatory Analysis Sheet - Review/Inspection" is: o Revised to incorporate all of the requirements contained in the Residential Swimming Pool Safety Regulation; o Used by all employees involved in the permit application and inspection process. Business unit's response: [TRANSLATION] The [TRANSLATION] "Regulatory Analysis Sheet - Review/Inspection" has been modified to incorporate provisions related to the temporary measures that may be required while work is under way (article 3, paragraph 9, of the Provincial Regulation). The [TRANSLATION] "Regulatory Analysis Sheet - Review/Inspection" applies to new installations. Since article 8 of the Provincial Regulation requires existing installations to be kept in good working order, we have created an inspection sheet specifically for this purpose. (Completed, March 2013) 20 For 2011: [TRANSLATION] By-law on rates for the borough of Rivi?re-des-Prairies-Pointe-aux-Trembles (fiscal 2011), RCA10-30038, borough council, December 7, 2010. For 2012: [TRANSLATION] By-law on rates for the borough of Rivi?re-des-Prairies-Pointe-aux-Trembles (fiscal 2012), RCA11-30043, borough council, December 6, 2011. Auditor General of the Ville de Montr?al 343 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.1.3.3. Installation Inspection 3.1.3.3.A. Background and Findings Regarding the step following permit issuance--the on-site inspection to ensure installations are compliant once the work is completed--, the results of our audit show that, for the four files we reviewed, the first inspection was conducted within a timeframe of between 63 and 98 days (2 months and a little over 3 months). Although it is possible to enter the estimated start and end date of the pool installation work in the computer system, we noticed that this information was not included in any of the files we analyzed. This makes it more difficult to determine a suitable moment for the inspector to visit the premises. We are aware that various circumstances, sometimes out of the borough's control, can lead to delays (e.g., when a resident does not proceed with an installation right away). However, we consider that following up on the work after the permit has been issued is an important step that should be carried out as quickly as possible to ensure that installations are safe and comply with the necessary regulatory provisions, but primarily with a view to minimizing drowning risks. In this regard, we learned of a guideline, which was updated by the DAUSE while we were carrying out our audit on December 12, 2012, which stipulates that inspections involving permits issued for the installation of a residential swimming pool or related equipment must be treated as a top priority by inspectors and completed in a timely manner. Consequently, we feel it may also be advisable that the borough use a tool it deems appropriate to track the progress of required inspections once a permit has been issued so that these activities are not overlooked and non-compliant installations are identified in a timely manner. 3.1.3.3.B. Recommendation We recommend that the Direction de l'am?nagement urbain et des services aux entreprises in Rivi?re-des-Prairies-Pointe-aux-Trembles borough determine the tool to be used to keep track of the inspections that need to be carried out once a permit has been issued so that these activities are not overlooked and non-compliant installations can be identified in a timely manner. Business unit's response: [TRANSLATION] Shortly after the Provincial Regulation on swimming pools came into force, the [TRANSLATION] "Residential Swimming Pool Installation Certificate of Authorization" was created in the Oracle - Gestion du territoire system. While following up on these types of projects, inspectors must indicate the anticipated date for the next inspection in the corresponding field. This procedure allows inspectors to generate a specific report based on anticipated inspection dates. Inspectors will Auditor General of the Ville de Montr?al 344 2012 Annual Report 5.8. Residential Swimming Pool Safety Regulations be required to generate this report every two weeks for monitoring purposes. (Completed, March 2013) 3.2. Mechanisms for Identifying Non-Compliant Installations 3.2.A. Background and Findings Much of the responsibility for ensuring the safety of residential swimming pools obviously falls to the residents who own them. Residents must therefore, in accordance with the Provincial Regulation, apply to their borough for a permit in this regard. Nevertheless, Montr?al's boroughs, in their role as representatives of the government, and in accordance with section 2 of the Act, are responsible for enforcing the Provincial Regulation. They must therefore put mechanisms into place to ensure a certain level of oversight with regard to residents who: o Did not apply for a permit and whose installations may not meet the safety standards set forth by the government; o Did not keep appurtenances designed to provide or limit access to a swimming pool in good working order or made non-compliant modifications since their permit was issued. Bearing this in mind, we took a closer look during our audit at the mechanisms implemented by the boroughs to identify installations that may be in violation of the safety standards outlined in the current Provincial Regulation. Following this process, we determined that in the three boroughs we audited there was no preventive plan in place to identify non-compliant installations. Based on the information we obtained from the people with whom we met, it would seem that non-compliant installations are generally discovered by chance during on-site inspections carried out for other reasons (e.g., a construction or renovation project) or during an investigation following a resident complaint. There are no statistics kept on these randomly discovered violations. Complaints received in this regard are few in number. We feel that these detection methods alone may not be sufficient to show that the city has acted with diligence in fulfilling its responsibilities under the Provincial Regulation, the primary aim of which is to reduce drowning risks. Considering that under the By-law concerning inspections 21 municipal authorities are fully entitled to enter any private property to ensure the by-laws for which they are responsible are being observed, we believe that 21 City council, by-law 6678, April 2, 1985. Auditor General of the Ville de Montr?al 345 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit the boroughs should develop mechanisms designed to identify properties with a swimming pool, so they can be incorporated into annual preventive inspection plans. Our audit revealed the existence of a computer application that may be useful for this purpose, namely the SGIS (spatial geographic information system), which uses aerial photographs to determine the existence of various objects, including swimming pools. This technique was used by Montr?al in 2004, when nearly 33,000 swimming pools were identified within in the city. Other means could also be used to achieve this objective (e.g., hiring a group of students to go from street to street to spot and list properties with pools). 3.2.B. Recommendation We recommend that the Direction du d?veloppement du territoire in AhuntsicCartierville borough and the Direction de l'am?nagement urbain et des services aux entreprises in both Pierrefonds-Roxboro and Rivi?re-des-Prairies-Pointe-auxTrembles boroughs take the appropriate measures to fulfill their responsibilities and ensure compliance with the Residential Swimming Pool Safety Regulation by: o Incorporating activities aimed at promoting residential swimming pool safety within their respective boroughs into annual preventive inspection plans; o Using mechanisms deemed appropriate to identify the location of residential swimming pools. Business units' responses: AHUNTSIC-CARTIERVILLE BOROUGH [TRANSLATION] One of the borough's construction and planning inspectors will be assigned on a part-time basis to preventive inspections (older files) and detection work as of June 2013. Detection mechanisms still need to be defined, based on the technologies at our disposal (aerial photography). (Planned completion: June 2013) Additional funds will be requested in the 2014 budget to provide the Direction du d?veloppement du territoire with seasonal resources to carry out prevention work related to residential swimming pools. (Planned completion: Summer 2014) PIERREFONDS-ROXBORO BOROUGH [TRANSLATION] A project to hire additional personnel will be submitted to the borough council for the resources necessary to carry out preventive inspections over a threeyear period. Auditor General of the Ville de Montr?al 346 2012 Annual Report 5.8. Residential Swimming Pool Safety Regulations Rire-Spatial software will be one of the detection mechanisms used to visually identify the location of properties with swimming pools. Using a comparison of orthographs prior to 2010 with recent data (2011 onward), we will be able to identify new installations for which a permit has not been issued and follow up with the necessary on-site inspection. (Planned completion: June 2014) RIVI?RE-DES-PRAIRIES-POINTE-AUX-TREMBLES BOROUGH [TRANSLATION] To apply the fines provided for in the Provincial Regulation, we will need to establish the definitive date when installations are completed. We therefore propose carrying out preventive inspections on a random sample of properties every year. To determine whether installations are subject to these regulations: o We will identify new installations by comparing aerial photographs in a given sector taken after the Provincial Regulation came into effect (using Map-Info and the SGIS); o We will check the presence of residential swimming pools installed after the Provincial Regulation came into effect for which no certificate of authorization has been issued. In addition, we propose that the files associated with a certificate of authorization for the installation of a residential swimming pool remain active for three years following issuance. These locations will be systematically followed up at the three-year point, after which the files will be closed. (Planned completion: between May and August 2013) 3.3. Follow-Up on Notices of Non-Compliance Once a permit has been issued, an inspector must visit the premises to ensure the installation-related work is in order. In other circumstances, for example if other identification mechanisms were to be implemented or a complaint is received, an inspector may also be called upon to check installations that have been identified as being in violation of regulatory safety standards. Any notices of non-compliance resulting from these inspections must specify the corrective action necessary on the part of the swimming pool owner to meet the stipulated requirements within an established timeframe. In the three boroughs we audited, the inspector is required to enter all inspection results into the Gestion du territoire - Permis computer system and carry out the necessary follow-up. A swimming pool owner who is in violation of the safety standards set out in the Provincial Regulation could be subject to a fine of $500 to $700, or as much as $1,000 in the case of a repeat violation. Auditor General of the Ville de Montr?al 347 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit At this stage of the process, our audit focused on the following aspects: o The nature of the notices of non-compliance that may be issued to swimming pool owners (verbal or written); o The manner in which an inspector follows up on the corrective action to be taken by the resident within a specified timeframe; o The details to be entered into the Gestion du territoire - Permis computer system for follow-up purposes. To substantiate our observations, we selected a new sample for each borough to include files where notices of non-compliance had been issued. The samples were picked randomly from a list of notices of non-compliance provided by the boroughs for the years 2010, 2011 and 2012. The sample for each borough therefore contained four files where a notice of non-compliance had been issued, except in the case of Ahuntsic-Cartierville borough for which only three files containing a notice of non-compliance could be found from these years. Our findings concerning the three boroughs are presented below. Note that statements of violation were not issued in any of these cases. 3.3.1. Ahuntsic-Cartierville Borough 3.3.1.A. Background and Findings For the three files we audited in this borough, we observed the following: o In one of the three files, we ascertained that a notice of non-compliance had been issued to a resident with a 10-day deadline for taking the necessary corrective action. A further review of the information in the file showed that the inspector followed up on the situation on an ongoing basis to ensure the corrective active was taken. o In a second file, we found a note in the computer system used to manage these files to the effect that a verbal warning had been issued to the resident in the spring of 2010 to make the access to the resident's pool safer: one of the gates was not secure and a section of the fence was missing. However, the file does not contain any information about the amount of time given to the resident to carry out the required corrective action. We then found a note indicating that the inspector carried out a new inspection a few days after the verbal warning but that none of the problems had been addressed. The parties once again came to a verbal agreement, although there is no mention of timeframes. According to the information in the file, it was only more than a year later Auditor General of the Ville de Montr?al 348 2012 Annual Report 5.8. Residential Swimming Pool Safety Regulations (fall 2011) that the inspector followed up with another inspection, during which he ascertained that corrective action had still not been taken. We then determined that there were subsequent verbal warnings made and that the matter was settled only in early 2012. In our opinion, although a verbal notice may be sufficient the first time a resident is warned of a violation, we feel that the files should systematically be updated with information on the terms of any arrangements made, including the timeframes involved. This would make it possible to: - Follow up on the requested corrective action in a timely manner; - Better plan subsequent efforts to be made over time; - Ensure a smooth transition should a different inspector be assigned to the file. In addition, in situations where the offender is slow to cooperate, we feel it may be more appropriate to issue a written notice of non-compliance containing a specific timeframe. This would make it possible to formalize the expected corrective measures and have an official paper trail of the actions undertaken should it become necessary to issue a statement of violation. o In the third file we audited, a written notice of non-compliance was issued indicating that the resident had five days to carry out the necessary corrective action. Based on the information in the computer system being used as of the date of our audit (November 28, 2012), there was no evidence on file that a subsequent inspection had been carried out to ensure the corresponding measures had been taken within the stipulated timeframe. We feel it is important for notices of non-compliance to be followed up systematically once the deadline given to the offender has passed, so that additional pressure can be exerted to ensure the specified corrective action is taken quickly so that situations involving a higher risk of drowning do not persist unnecessarily. Auditor General of the Ville de Montr?al 349 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.3.1.B. Recommendation We recommend that the Direction du d?veloppement du territoire in AhuntsicCartierville borough take the necessary measures to remind its inspectors of the importance of entering in the file all the terms of any arrangements made with residents, including the timeframe specified, in a first verbal notice of noncompliance (warning) following an inspection, in order to follow up on the requested corrective action in a timely manner, better plan subsequent efforts to be made and ensure a smooth transition should a different inspector be assigned to the file. Business unit's response: [TRANSLATION] The Direction du d?veloppement du territoire is currently working on an administrative guideline aimed at inspectors to clarify the importance of issuing written notices for violations involving swimming pools or related regulatory requirements. Some notices can be drafted on the spot; these will now be required in lieu of a verbal warning, although they will not carry the same weight as a statement of violation processed through the computer system. Meanwhile, all inspectors currently on staff and the head of the Section de l'inspection have received a verbal reminder in this regard. (Planned completion: May 2013) 3.3.1.C. Recommendation We recommend that the Direction du d?veloppement du territoire in AhuntsicCartierville borough take the necessary measures in situations where the offender is slow to cooperate so that its inspectors favour the use of a written notice of noncompliance specifying a timeframe, to formalize the expected corrective measures and have an official paper trail of the actions undertaken should it become necessary to issue a statement of violation. Business unit's response: [TRANSLATION] As we indicated in our response to the recommendation 3.3.1.B, the administrative guideline that is currently in the works will require the systematic use of written notices (manual or computer-generated). Verbal warnings will no longer be used. (Planned completion: May 2013) Auditor General of the Ville de Montr?al 350 2012 Annual Report 5.8. Residential Swimming Pool Safety Regulations 3.3.1.D. Recommendation We also recommend that the Direction du d?veloppement du territoire in AhuntsicCartierville borough exert additional pressure to ensure the specified corrective action is taken quickly so that situations that engender a higher risk of drowning do not persist unnecessarily, by taking the necessary measures to follow up systematically on notices of non-compliance once the deadline given to the offender has passed. Business unit's response: [TRANSLATION] The administrative guideline to which we refer in the two previous recommendations will also outline the procedure for entering follow-up dates in the permit system. Each file will therefore contain a reminder date, date on which inspectors can follow up on written notices. (Planned completion: May 2013) 3.3.2. Pierrefonds-Roxboro Borough 3.3.2.A. Background and Findings Regarding the follow-up for notices of non-compliance for the four selected files in Pierrefonds-Roxboro borough, our audit revealed the following: o The four notices of non-compliance were found through a personal follow-up document prepared by a student. Based on the information obtained from the manager with whom we met, the Gestion du territoire - Permis computer system should be the one used by inspectors to compile the results of their efforts with respect to notices of noncompliance issued and the subsequent follow-up. However, we observed that the notices of non-compliance had not in fact been posted in the system. It was therefore impossible to use it to determine the number of notices of non-compliance issued over time. However, the administration occasionally hires summer students to follow up on certain matters, including cases of non-compliance involving residential swimming pools. These students develop personal follow-up tools using spreadsheets (Excel) which contain a list of the issued notices. This source of information, which nobody could confirm was either complete or incomplete, is the only resource we found that identifies the notices of non-compliance that had been issued. Furthermore, our review of the selected files also revealed that, once these notices of non-compliance were issued, information on follow-up efforts was occasionally entered into the Gestion du territoire - Permis system, although not in its entirely, as some of it was in the student's file instead. Auditor General of the Ville de Montr?al 351 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit In order to make it easier to track these operations and provide timely follow-up on notices of non-compliance issued to residents, we feel that all of the information needs to be entered into the computer system recommended by the administration. This would make it possible to generate reports on the number of notices of non-compliance issued and those that are still pending and avoid confusion and lack of follow-up with respect to notices of non-compliance because of the absence of centralized information. o For two of the four files we reviewed, we observed that a verbal notice of noncompliance (warning) was issued to the resident. However, a closer examination of the information in the files shows that only one of them indicates that a timeframe (15 days) was given to the resident to undertake the necessary corrective action. In the other file, we did not find any information concerning timeframes. Regarding the file where the timeframe was specified and for which corrective action was required to address, among other things, the lack of a gate to control access to the pool from an adjacent patio, we observed that the matter was considered settled and closed after 52 days (nearly two months) after the initial 15-day deadline. The file did note, however, that a subsequent verbal warning had been issued, although there was no mention of the stipulated timeframe. In the other file, for which we found no evidence of a timeframe provided to the resident, we observed that the only other effort made to ensure the corrective active was taken occurred 134 days (nearly 4.5 months) after the initial warning was issued. At that time, the file was considered settled and had been closed. We feel this response time is too long, especially since the file involved an unsecure gate that did not adequately control access to the pool and since the warning was a verbal one without any further information on the timeframe granted for carrying out the corrective action. o For two of the four files, the resident was informed of the corrective action to be taken by way of a written notice of non-compliance. Once again, we observed that only one of the files specified a timeframe (15 days), whereas the other file contained no such information. The file for which a 15-day deadline was imposed involved a fenceless inground pool. We observed that the file was considered to be settled and closed 31 days after the initial notice was issued to the resident. We also found evidence that a verbal reminder had been given in the meantime, although no timeframes were specified in the file. Auditor General of the Ville de Montr?al 352 2012 Annual Report 5.8. Residential Swimming Pool Safety Regulations We ascertained that the file for which we did not find any evidence concerning the amount of time indicated to the resident was considered settled 55 days (nearly two months) after the notice was issued. Once again, the file indicates that a second verbal notice had been provided, but again, without any mention of how long the resident was given to comply. In our opinion, although a verbal notice may be sufficient the first time a resident is warned of a violation, we feel that the files should systematically contain documentation on the terms of the arrangements made, including the timeframes involved. This would make it possible to: - Follow up on the requested corrective action in a timely manner; - Better plan subsequent efforts to be made over time; - Ensure a smooth transition should a different inspector be assigned to the file. In addition, in situations where the offender is slow to cooperate, we feel it may be more appropriate to issue a written notice of non-compliance specifying a timeframe. This would make it possible to formalize the expected corrective measures and have an official paper trail of the actions undertaken should it become necessary to issue a statement of violation. 3.3.2.B. Recommendation We recommend that the Direction de l'am?nagement urbain et des services aux entreprises in Pierrefonds-Roxboro borough take the necessary measures to ensure that all of the information related to the issuance of and follow-up notices of noncompliance be entered into the Gestion du territoire - Permis computer system in order to be able to: o Generate reports on the number of notices of non-compliance issued and those that are still pending; o Easily identify what has already been done in the file and ensure timely follow-up; o Avoid confusion and lack of follow-up with respect to notices of non-compliance because of the absence of centralized information. Business unit's response: [TRANSLATION] Use the Gestion du territoire - Permis system at all times and compile data for every inspection. Follow up on all inspections. Auditor General of the Ville de Montr?al 353 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Produce a monthly report to check the status of various files. (Planned completion: May 2013) 3.3.2.C. Recommendation We recommend that the Direction de l'am?nagement urbain et des services aux entreprises in Pierrefonds-Roxboro borough remind its inspectors of the importance of including the timeframe indicated to residents to take corrective action in the file in order to follow up on the requested corrective action in a timely manner, better plan subsequent efforts to be made and ensure a smooth transition should a different inspector be assigned to the file. Business unit's response: [TRANSLATION] Monthly reports will also be sent to those inspectors who must follow up on files for the purpose of issuing statements of violation or briefing replacement staff. Plan meetings with inspectors to discuss file monitoring and follow-up procedures. (Planned completion: May 2013) 3.3.2.D. Recommendation We recommend that the Direction de l'am?nagement urbain et des services aux entreprises in Pierrefonds-Roxboro borough undertake the necessary actions to encourage inspectors to issue written notices of non-compliance specifying a timeframe, in situations where the offender is slow to cooperate, in order to formalize the expected corrective measures and have an official paper trail of the actions undertaken should it become necessary to issue a statement of violation. Business unit's response: [TRANSLATION] Notices of non-compliance will always be in written form and specify a timeframe for corrective action. Violations must also always be clearly indicated. All data required for follow-up purposes must be compiled in the Gestion du territoire - Permis system. (Planned completion: May 2013) 3.3.3. Rivi?re-des-Prairies-Pointe-aux-Trembles Borough 3.3.3.A. Background and Findings In the four selected files for which notices of non-compliance were issued, our audit revealed that, for all four, a written notice was issued to the offender, indicating that Auditor General of the Ville de Montr?al 354 2012 Annual Report 5.8. Residential Swimming Pool Safety Regulations corrective action was required within a period of 10 or 30 days. Based on the information found in the file through the Gestion du territoire - Permis computer system, we determined that, in the two cases where a 10-day timeframe was given, the inspector followed up on the file on an ongoing basis once the initial deadline had passed, to ensure the required corrective action was carried out within a satisfactory timeframe. In the two other files, where a 30-day grace period was granted, the information we reviewed in the file indicated that an inspector followed up on the file in a timely manner and the file was settled before the deadline. We did wonder, however, why a 30-day timeframe was given in situations where the noncompliant elements indicated in the file involved a missing or inadequate enclosure around the pool, considering that such situations represent a high drowning risk. Furthermore, although section 9 of the Provincial Regulation mentions that temporary measures must be taken to control access to the pool in these scenarios, we found no mention in the file of any such course of action. 3.3.3.B. Recommendation We recommend that the Direction de l'am?nagement urbain et des services aux entreprises in Rivi?re-des-Prairies-Pointe-aux-Trembles borough help reduce drowning risks in situations of non-compliance where swift corrective action is required by: o Assessing the possibility of imposing a deadline shorter than 30 days in situations where a notice of non-compliance is to be issued; o Reminding inspectors about the importance of ensuring that temporary measures designed to control pool access are in place, in accordance with regulatory requirements, and of updating the files with this information. Business unit's response: [TRANSLATION] A section containing guidelines on temporary measures that may be required while work is under way has been added to the [TRANSLATION] "Regulatory Analysis Sheet - Review/Inspection." These measures are henceforth an integral part of the items to be checked by inspectors. The deadline stipulated in these notices of non-compliance will be shortened to 15 days. (Completed, March 2013) Auditor General of the Ville de Montr?al 355 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.4. Mechanisms for Raising Public Awareness An additional purpose of our audit was to determine the various mechanisms used by the boroughs to educate residents about the regulations concerning residential swimming pool safety and the need to obtain a permit for all construction, installation and replacement work as well as for the erection of any structures enabling or preventing pool access. For each of the boroughs we visited, we listed the mechanisms used and examined the nature and public accessibility of the distributed information. 3.4.1. Ahuntsic-Cartierville Borough 3.4.1.A. Background and Findings In this borough, the mechanisms used to raise residents' awareness about regulations concerning residential swimming pools were observed in three main sources of information: o Leaflets available at the borough office, including a fact sheet entitled [TRANSLATION] "Swimming Pool Permits"; o Information communicated via the borough website, including: - The ACC?S MONTR?AL network's 311 databank: fact sheet entitled [TRANSLATION] "Ahuntsic-Cartierville - Private Pools," - The [TRANSLATION] "Permits and authorizations" section ("Services for residents / Permits and regulations"): "Info-Permit" sheets, - Borough newsletters found under the [TRANSLATION] "All publications" heading on the home page; o Community newspapers. Although it is interesting to note that the borough has developed a range of information sources to apprise residents of the various requirements concerning swimming pool installations, an examination of the information these sources contain revealed the following: o In some instances, the information is not up to date. The [TRANSLATION] "Swimming Pool Permits" fact sheet, for one, which is available through the borough office, does not contain any details on the new regulatory provisions concerning residential swimming pool safety. Moreover, the [TRANSLATION] "Ahuntsic-Cartierville - Private Pools" fact sheet in the 311 databank indicates that the fence around a pool must be assembled in a manner that Auditor General of the Ville de Montr?al 356 2012 Annual Report 5.8. Residential Swimming Pool Safety Regulations prevents a spherical object 12 centimetres (4 inches) in diameter from getting through, whereas the figure used in the Provincial Regulation and the borough's fence by-law is 10 centimetres. Finally, the [TRANSLATION] Info-Permit sheet entitled [TRANSLATION] "Looking for an Appurtenance?" (No. 10), which can be found in the [TRANSLATION] "Permits and authorizations" section of the website, is dated September 2010, whereas the new borough by-law on fences was adopted in October 2011 and incorporates new provisions on fences and enclosures around residential swimming pools. o In other cases, the information provided could be more detailed. During our examination of borough newsletters, we noted that the fall 2012/winter 2013 22 issue reminds residents about the need to obtain a renovation permit (which extends to swimming pools), but it does not include any details as to the safety requirements stipulated in the Provincial Regulation. The same applies to the community newspapers through which a range of information is relayed to the public. In 2012, for instance, we observed that this means of communication was used by the borough to remind residents that a permit was required to install a pool, but no mention was made about the mandatory safety standards to be observed. In the interest of ensuring that current regulations aimed at reducing drowning risks are adhered to, and to avoid potential confusion with respect to the safety measures to be taken when installing a swimming pool, we feel that additional efforts should be made by the borough so that the various sources of information used to apprise residents of the applicable rules in terms of residential swimming pool safety are complete and up to date. In addition, we believe that it would be a good idea for the borough to continue to remind residents on an annual basis that a permit is required to install a pool, only it should also include a list of the corresponding safety requirements as dictated by the borough and the provincial government. 22 [TRANSLATION] Ahuntsic-Cartierville Newsletter, Fall 2012/Winter 2013, Vol. 9, No. 2. Auditor General of the Ville de Montr?al 357 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.4.1.B. Recommendation We recommend that the Direction du d?veloppement du territoire in AhuntsicCartierville borough ensure applicable regulations are adhered to and avoid potential confusion in the implementation of safety measures related to residential swimming pools by: o Investing the necessary efforts so that the information in the various mechanisms used to educate and inform the public is complete and up to date; o Continuing to issue annual reminders to residents about the need to obtain a permit to install a swimming pool, including the various safety standards set forth in both municipal by-laws and the Provincial Regulation. Business unit's response: [TRANSLATION] The [TRANSLATION] Info-Permit fact sheet for residential swimming pools is being reviewed and will be uploaded to the website in May 2013. This new version will include the same checklist as the one used by permit clerks and inspectors. Once the fact sheet is online, a notice to this effect will be published in the monthly resident newsletter along with a hyperlink to the Info-Permit sheet. (Planned completion: May 2013) The spring issues of the [TRANSLATION] Ahuntsic-Cartierville Newsletter and resident newsletters will also contain this information. (Planned completion: June 2014) 3.4.2. Pierrefonds-Roxboro Borough 3.4.2.A. Background and Findings The main public awareness tool used to inform Pierrefonds-Roxboro borough residents about residential swimming pool safety regulations is the borough website. Pool-related information can be found in: o The ACC?S MONTR?AL network's 311 databank; o The "Permits and authorizations" section (under "Services for residents / Permits and regulations"); o The municipal magazine available under the "Borough newsletter" heading on the home page. Auditor General of the Ville de Montr?al 358 2012 Annual Report 5.8. Residential Swimming Pool Safety Regulations In reviewing the contents of these documents, we observed that: o The 311 databank indicates that residents are required to obtain a permit to build or install a residential swimming pool and provides an overview of the latest safety standards contained in the corresponding regulations. o The May 2011 issue of the municipal magazine Life in Pierrefonds-Roxboro 23 reminds residents that a permit is required to install a swimming pool more than 600 mm high and lists the applicable safety standards. The May 2012 issue 24 features a similar set of reminders. o The "Permits and authorizations" section (under "Services for residents / Permits and regulations") contains a brief mention of the need to obtain a permit to install an inground or aboveground pool or an inflatable pool more than 600 mm high. It does not, however, provide any information about the safety standards to be adhered to. The borough initiative aimed at reminding residents every year about the regulatory requirements concerning the installation of swimming pools is, in our opinion, a sound management practice in terms of helping to raise public awareness about the safety standards to be observed to reduce drowning risks. We believe, however, that it would be appropriate for the borough to enhance the information in the "Permits and authorizations" section of its website to incorporate the regulatory safety standards associated with swimming pool installation. We feel that the very title of this section of the website makes it more likely to be the source of information residents turn to most frequently to find out what the rules are for obtaining a project permit. 3.4.2.B. Recommendation We recommend that the Direction de l'am?nagement urbain et des services aux entreprises in Pierrefonds-Roxboro borough improve public awareness about existing regulations by: o Taking the necessary measures to enhance the "Permits and authorizations" section of the site (under "Services for residents / Permits and regulations") with additional information on the regulatory safety standards to adhere to when installing or constructing a residential swimming pool so that the public has access to complete information no matter which source of information they use; o Continuing to remind residents every year about regulatory requirements governing residential swimming pools. 23 24 Life in Pierrefonds-Roxboro, municipal magazine, May 2011, Vol. 6, No. 2. Life in Pierrefonds-Roxboro, municipal magazine, May 2012, Vol. 7, No. 2. Auditor General of the Ville de Montr?al 359 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Business unit's response: [TRANSLATION] The information contained in the Life in Pierrefonds-Roxboro magazine, along with other important details, will also be published on the borough website under "Services for residents / Permits and regulations." (Planned completion: May 2013) 3.4.3. Rivi?re-des-Prairies-Pointe-aux-Trembles Borough 3.4.3.A. Background and Findings The mechanisms used by the borough to educate and inform residents about existing regulations as they apply to residential swimming pool safety are channelled through two main sources of information: o The borough website: - The ACC?S MONTR?AL network's 311 databank, - The [TRANSLATION] "Permits and authorizations" section ("Services for residents / Permits and regulations"): [TRANSLATION] Info-Permit: Swimming Pools, 25 - The e-newsletter available under the [TRANSLATION] "Newsletter" heading, - The [TRANSLATION] "News" and "Public notices" sections of the website, accessible from the home page; o Community and local newspapers distributed in the borough, in particular an article entitled [TRANSLATION] "Borough Report." In examining the content of these documents, we made the following observations: o The information contained in the 311 databank specifies the need for residents to obtain a permit to install a pool. However, it does not provide any information on the regulatory safety standards applicable to residential swimming pools. o The [TRANSLATION] "Permits and authorizations" section ("Services for residents / Permits and regulations") features an [TRANSLATION] "Info-Permit fact sheet" specifically about pools entitled [TRANSLATION] "Pools, Spas and Other Outdoor Basins" with a range of information that was last updated on May 7, 2012. It stipulates that residents are required to obtain a permit to install a pool and notes the regulatory safety standards applicable to residential swimming pools that must be observed. o The e-newsletter is sent to residents who subscribe to the service. The April 15, 2011, issue reports on new regulations applicable to residential swimming pools and emphasizes the importance of applying for a permit. In 2012, this medium was not used to inform residents about the rules related to residential swimming pool safety. 25 [TRANSLATION] Info-Permit: "Pools, Spas and Other Outdoor Basins." Auditor General of the Ville de Montr?al 360 2012 Annual Report 5.8. Residential Swimming Pool Safety Regulations o o Pool-related information occasionally appears in the [TRANSLATION] "News" and "Public notices" sections of the website. On May 5, 2011, for example, information was posted in the [TRANSLATION] "News" section to announce that new residential swimming pool safety regulations were in force. In addition, on April 14, 2011, we ascertained that a notice entitled [TRANSLATION] "What You Need to Know Before You Install Your Residential Swimming Pool" was published. In 2012, these communication tools were not used to convey information on the rules related to residential swimming pool safety. As for local and community newspapers distributed in the borough, we observed that the borough published an article on July 17, 2012, entitled [TRANSLATION] "Pool Safety: Think About It!" which contained a section devoted to residential swimming pool safety. All in all, we determined that the borough uses various means of communication to inform and educate the public about regulatory standards applicable to residential swimming pool safety. Apart from the 311 databank, which does not provide any information on regulatory safety standards applicable to residential swimming pools, the information distributed overall by the borough is complete and up to date. We also encourage the borough to continue to send out annual reminders to residents about the provisions of the Provincial Regulation. 3.4.3.B. Recommendation We recommend that the Direction de l'am?nagement urbain et des services aux entreprises in Rivi?re-des-Prairies-Pointe-aux-Trembles borough improve resident awareness about compliance with existing regulations by: o Taking the necessary measures to ensure the information in the 311 databank also incorporates information about the regulatory safety standards for residential swimming pools so that the public has access to complete information, regardless of the source of information they use; o Continuing to remind residents every year about regulatory requirements governing residential swimming pools. Business unit's response: [TRANSLATION] The files contained in the 311 databank are updated annually. For the [TRANSLATION] "Pools, Spas and Other Outdoor Basins" fact sheet, this update is planned for May 2013. We will take advantage of this update to include information on the regulatory safety standards applicable to residential swimming pools. We will work with our borough's Division des relations avec les citoyens et communications to publish articles every year that remind residents of the regulatory Auditor General of the Ville de Montr?al 361 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit requirements involved in completion: May 2013) installing residential swimming pools. (Planned 3.5. Accountability 3.5.A. Background and Findings As part of our audit, we also examined the accountability mechanisms used to ensure the boroughs periodically assess the performance of their activities as they relate to the application of the Provincial Regulation. This section outlines our observations as they apply to the three boroughs we visited. In light of the information we obtained from the people with whom we met at each of the boroughs we visited, it would seem that there are no particular accountability mechanisms in place in this regard. In Ahuntsic-Cartierville borough, we observed that the statistical reports produced by the computer system used by the borough to manage permit-related information (Gestion du territoire - Permis) are produced annually by the division head and submitted to the manager of the Direction du d?veloppement du territoire. Although these reports provide indications concerning such aspects as the number of permits issued, the cost of work and the cost of permits as they relate to construction, demolition, conversion and occupation of the public domain, none of the information they contain is related to swimming pool permits. Regarding the Pierrefonds-Roxboro borough, the information obtained from the manager with whom we met revealed that a monthly report is produced for management purposes and submitted to the manager of the DAUSE, as well as to the borough council for information purposes. This report, entitled [TRANSLATION] Permits and Certificates Issued for the Month of xx, is generated using information derived from the computer system used to manage and issue permits (i.e., Gestion du territoire - Permis). Among the data it contains are the number of permits issued, the corresponding value with respect to new constructions, expansions, building renovations and demolitions, and activities related to occupation of the public domain. We did note, however, that the report does not contain any information on residential swimming pool permits. Finally, in Rivi?re-des-Prairies-Pointe-aux-Trembles borough, we observed that yearly reports on permits issued were also being produced for the DAUSE. These reports include statistics on the number of permit applications processed on an annual basis as well as the length of time taken to issue them. The data are compiled by type of application: o Construction, renovation and demolition permits; Auditor General of the Ville de Montr?al 362 2012 Annual Report 5.8. Residential Swimming Pool Safety Regulations o o Certificate of occupation, certificate of approval to post notices or signs; Cadastral operations. None of these reports contains information on permits issued for the installation or construction of residential swimming pools. Considering that, under the Act, the city is responsible for enforcing the Provincial Regulation, we feel it is appropriate that management reports be produced for activities related to the issuance of permits for the installation of residential swimming pools. This information would make it easier to determine the workload associated with this activity and inform decision-making based on available resources. 3.5.B. Recommendation We recommend that the Direction du d?veloppement du territoire in AhuntsicCartierville borough and the Direction de l'am?nagement urbain et des services aux entreprises in both Pierrefonds-Roxboro and Rivi?re-des-Prairies-Pointe-auxTrembles boroughs undertake the necessary measures to produce regular, brief management reports on activities related to issuing residential swimming pool permits, with the goal of using this information to help determine the workload associated with this activity and, as required, inform decision-making based on available resources. Business units' responses: AHUNTSIC-CARTIERVILLE BOROUGH [TRANSLATION] We are currently looking into the feasibility of reprogramming the permit system to include the option of retrieving swimming pool and renovation permits. If this is done, annual reports could be issued on every aspect of the residential swimming pool permit and inspection process, starting in 2014. (Planned completion: January 2014) PIERREFONDS-ROXBORO BOROUGH [TRANSLATION] Produce an annual report with statistics on the number of certificates of authorization produced, the on-site inspections carried out, the number of statements of violation issued, response times and other details deemed useful in determining the required level of resources. (Planned completion: December 2013) Auditor General of the Ville de Montr?al 363 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit RIVI?RE-DES-PRAIRIES-POINTE-AUX-TREMBLES BOROUGH [TRANSLATION] The certificate of authorization for residential swimming pool installations, which is created using the Oracle - Gestion du territoire system, makes it possible to produce the reports required to assess the workload associated with this activity and, as needed, guide decision-making based on the available resources. (Completed, March 2013) Auditor General of the Ville de Montr?al 364 2012 Annual Report Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al) 5.9 5.9. First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al) Table of Contents 1. Introduction ................................................................................................................... 369 2. Audit Scope................................................................................................................... 373 3. Findings and Recommendations .................................................................................. 374 3.1. First Responder Service........................................................................................ 374 3.2. 2009-2013 Fire Safety Cover Plan ....................................................................... 387 4. Appendix ....................................................................................................................... 413 4.1. Implementation of the First Responder Service - Detailed Cost Estimates, 2007-2009 .......................................................................................... 413 Auditor General of the Ville de Montr?al 367 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit List of Acronyms ASSSM Agence de la sant? et des services sociaux de Montr?al SGIAP integrated fire prevention activities management system DSTI Direction des strat?gies et transactions immobili?res SIM Service de s?curit? incendie de Montr?al FR first responder SPVM Service de police de la Ville de Montr?al FSCP fire safety cover plan STI GDD decision-making record management system Service des technologies de l'information TCEP MSP Minist?re de la S?curit? publique three-year capital expenditures program SGAP fire prevention management system Auditor General of the Ville de Montr?al 368 2012 Annual Report 5.9. First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al) 5.9. First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al) 1. Introduction In recent years, management at the Service de s?curit? incendie de Montr?al (SIM) has been entrusted with two major projects: the implementation of a first responder (FR) service and the development and implementation of a fire safety cover plan (FSCP). The goal of the first project was to enable the SIM to expand the services that it offered citizens across all of the Montr?al agglomeration and to change an entire organizational culture. The second project was intended to meet the requirements of the Fire Safety Act 1 in order to manage fire risks on the entire territory of the agglomeration in compliance with the directives and objectives of the Minist?re de la S?curit? publique (MSP). In both cases, a project team reporting to SIM management was put in place. First Responder Service At the time of the creation of the new city in 2002, eight fire stations belonging to the former suburban municipalities 2 already offered a FR Service. However the service operated differently from one fire station to the next and was not standardized. In 2003, Montr?al city council decided to extend the FR service to the entire Island of Montr?al and mandated SIM management to develop an island-wide standard FR service. In 2005 and 2006, SIM management reached an agreement with firefighters on implementation of the FR service, harmonized the service in the eight fire stations that already offered it, and developed a plan to extend the service to the 58 other fire stations on the Island of Montr?al (19 boroughs and 15 related municipalities). These actions, which SIM management supported, resulted, in subsequent years, in the city (under an urban agglomeration power) finalizing two memorandums of understanding and one agreement to implement an FR service and have it up and running: o A 10-year memorandum of understanding (2007-2017) between the city and the Agence de la sant? et des services sociaux de Montr?al (ASSSM) and the Corporation d'urgences-sant?, setting out the terms and conditions for involvement of the parties in implementing the FR service and funding for the city (under an urban agglomeration 1 2 RSQ, chapter S-3.4 Beaconsfield, Dollard-des-Ormeaux, Hampstead, Kirkland, Montr?al-Ouest, Outremont, Pointe-Claire and Westmount. Auditor General of the Ville de Montr?al 369 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit o o power). This memorandum was approved by the urban agglomeration council in March 2007. A 10-year memorandum of understanding between the city and the Corporation d'urgences-sant? (2007-2017) for the acquisition of goods and services. This memorandum was approved by the urban agglomeration council in March 2007, and amended in 2011. An agreement between the city and the Association des pompiers de Montr?al for implementation of an FR service was signed in January 2007, and integrated into the existing collective agreement. The purpose of these agreements was to provide a framework for the implementation of the FR service, based on the following objectives: o To set up an FR service that would be integrated into the chain of pre-hospital emergency services across the entire territory of Montr?al; o To reduce population morbidity and mortality rates, in particular by reducing the response time of the pre-hospital emergency services system. To ensure that the FR service would be up and running, SIM management prepared a three-year implementation plan (2007-2009) whose main activities were: o To harmonize the SIM's communication procedures with those of the Corporation d'urgences-sant?; o To provide firefighters with necessary basic training (62 hours) and ongoing training (24 hours every two years) to enable them to act as FRs and maintain their certification; o To acquire the necessary equipment for FR interventions (including kits, specialized devices, etc.); and o To refurbish the vehicles in the existing fleet and acquire additional vehicles. SIM management prepared cost estimates for the three-year program. These appear in Table 1. Auditor General of the Ville de Montr?al 370 2012 Annual Report 5.9. First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al) Table 1 - Initial Cost Estimates for the Implementation and Start-Up Period of the FR Service (in thousands of dollars) 2007 2008 2009 Total 18 21 58 Expenditures Implementation costs No. of fire stations to be integrated 19 Amount $4,290.7 $3,626.3 $4,436.8 $12,353.8 Operating costs No. of active fire stations - start of year 8 No. of active fire stations - end of year 27 Amount 45 66 $2,501.1 $8,389.1 $16,371.6 $6,791.8 $9,107.7 $12,825.9 $28,725.4 $7,207.5 $5,500.0 $5,500.0 $18,207.5 $2,210.0 Total expenditures $5,481.4 $2,000.0 $2,000.0 $6,210.0 Revenues Based on the memorandum of understanding a b TCEP Acquisition of vehicles a Amounts to be indexed yearly at the rate set by the Government of Qu?bec. Three-year capital expenditures program. Source: SIM, 2007 b The anticipated total cost of the FR service during the three-year implementation and startup period was $28,725,400. For the same period, the city expected to receive $18,207,500 (on behalf of the agglomeration) as compensation from the ASSSM. In terms of the acquisition of vehicles, SIM management anticipated allocating $6,210,000 for the purchase of seven fire engines. Appendix 4.1 presents detailed estimates of the implementation and operating costs for the years 2007 to 2009. Fire Safety Cover Plan The main purpose of an FSCP is to determine how fire safety services are to be organized on a given territory. It is essentially a risk management exercise, that is, an analysis of the fire risks present on a territory with the objective of planning prevention measures to reduce potential fire hazards and planning the necessary responses to limit the harmful effects when fire does occur. Auditor General of the Ville de Montr?al 371 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Adopted in December 2008 by the Montr?al urban agglomeration council, the FSCP established the actions that needed to be taken, along with the additional resources that would be required, to comply with the directives issued by the Ministre de la S?curit? publique regarding fire safety: o reduce loss of life; o reduce material losses resulting from a fire; o increase the effectiveness of municipal bodies in this field; o obtain immunity from prosecution. These directives 3 were aimed at achieving eight objectives decreed by the MSP: o Objective 1: Given the proven effectiveness of preventive measures in reducing fires, prioritize fire prevention approaches and measures to protect citizens and our heritage from fire. o Objective 2: Taking into account existing resources at the regional level, structure fire safety services, plan the organization and delivery of rescue operations, and plan conditions for response in low-risk situations within the urbanisation parameters defined in the development plan, to deploy a strike force that can intervene effectively. o Objective 3: Taking into account existing resources, structure fire safety services, plan the organization and delivery of emergency services, and plan conditions for response in the case of other classes of risks, that can deploy an optimum strike force. o Objective 4: Compensate for future shortcomings in fire response through adapted selfprotection measures. o Objective 5: In the case of other disaster risks that are likely to require the use of fire safety resources, plan the organization of rescue operations and plan conditions of response that will enable the deployment of an optimum strike force using available resources at the regional level. o Objective 6: Maximize the use of fire safety resources. o Objective 7: Act at the supramunicipal level of the regional county municipalities (MRC) to organize and manage certain fire safety operations. o Objective 8: Plan fire safety with a view to linking together resources and organizations with other public safety structures, such as civil security, emergency services, prehospital emergency services or police services. SIM management made provisions in its FSCP for a 5-year implementation plan (2009- 2013) to achieve these objectives. The plan revolved around three components: prevention (consisting of six programs aimed at improving fire safety on the territory of the Montr?al 3 Directives of the Ministre de la S?curit? publique in the area of fire safety, Gazette officielle du Qu?bec, Part 2, Vol. 133, No. 22, May 30, 2001, pp. 3315-61. Auditor General of the Ville de Montr?al 372 2012 Annual Report 5.9. First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al) agglomeration), response through the organization of an optimum strike force, and the provision of water in the event of a fire. In 2008, SIM management estimated that the operating budget for the five years needed to fully implement the FSCP would be $63,399,400 and that investments would total $30,369,100. Table 2 - Cost Estimates for Implementation of the FSCP (in thousands of dollars) a Operating budget TCEP 2009 4,598.6 3,148.5 2010 7,376.0 7,091.3 2011 12,666.8 2,855.0 2012 17,701.0 11,772.5 2013 21,057.0 5,501.8 Total 63,399.4 30,369.1 a Estimates include debt servicing (interest to pay down the debt). Source: SIM, April 2008 Following the favourable notice issued by the urban agglomeration council regarding the FSCP, the MSP deemed that the plan complied with its fire safety directives and issued a certificate of compliance on October 17, 2008. The urban agglomeration council subsequently adopted the FSCP at its December 18, 2008, session, and the plan came into force on January 1, 2009. As a result, the two major projects that we mentioned are now in a major phase of development. In the case of the FR service, while implementation is completed, operating costs continue to rise while funding reached (in 2011) the ceiling of $7,000,000 set out in the memorandum of understanding. In the case of the FSCP, whose objectives are of major importance, almost a year remains before implementation of the proposed activities will be completed. We believe, therefore, that conducting an audit at this stage in the development of these two projects is timely since it will enable us to see how the projects have unfolded to date and, if necessary, suggest adjustments. 2. Audit Scope The objectives of our audit were to ensure that: o Implementing the FR service achieved the expected results; and o The FSCP was moving forward in accordance with the implementation plan adopted by the urban agglomeration council. Auditor General of the Ville de Montr?al 373 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit To do this, we looked at how the proposed completion schedules and costs were monitored, as well as at the accountability mechanisms that were put in place to report on the progress of the work. In order to understand the legal obligations that the SIM faced in regard to the FR service and the FSCP, we also familiarized ourselves with the laws, regulations and agreements that form the framework of these obligations. Our audit was conducted with the SIM managers who were directly involved in implementing the FR service and in carrying out the implementation plan of the FSCP. In addition to key SIM staff, we also met with individuals at the head of the Service des technologies de l'information (STI), the Direction des strat?gies et transactions immobili?res (DSTI), which reports to the Service de la concertation des arrondissements et des ressources mat?rielles, and the Service de l'eau, who were directly involved, in a supporting role, in running the operational data system used by the FR service or in completing the implementation of the FSCP. Our audit covered the implementation and start-up period, as well as the consolidation period, of the FR service (2007-2011), and the implementation period of the FSCP (2009- 2012). Our work extended mainly from May to July 2012. 3. Findings and Recommendations 3.1. First Responder Service The implementation of the FR service was completed in 2009 throughout the Montr?al agglomeration, with the exception of Ville de C?te-Saint-Luc, which operates its own FR service. 4 The FR service is operational in the SIM's 65 fire stations and is currently in the consolidation phase. In 2011, more than 1,800 firefighters out of a total of 2,392 active firefighters were certified as FRs. Since its launch in 2007, the FR service has carried out 244,844 responses and has maintained an average response time of 4 minutes, 50 seconds, which is well below the reference target response time of 7 minutes, 59 seconds, set out in the memorandum of understanding between the ASSSM, the Corporation d'urgences-sant? and the city (under an urban agglomeration power). 4 Section 28.1 of the Act respecting the exercise of certain municipal powers in certain urban agglomerations (RSQ, chapter E-20.001) provides an exception for Ville de C?te-Saint-Luc. However, this exception was made to this bill in 2008 (An Act to amend various legislative provisions concerning Montr?al, SQ, 2008, chapter 19). Auditor General of the Ville de Montr?al 374 2012 Annual Report 5.9. First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al) Although the FR service is now deployed in all the island's fire stations and the expected response time is being largely met, we evaluated the extent to which implementing the FR service was instrumental in achieving the anticipated results. We will begin here by briefly describing how the implementation of the service unfolded over the first three years, in keeping with the goal of training firefighters and deploying FR services. Secondly, we will examine the degree of compliance with the obligations set out in the memorandum of understanding with the ASSSM and the Corporation d'urgences-sant?. Finally, we will examine the cost to the agglomeration of receiving these FR services since the memorandum of understanding came into force, and we will address the issue of accountability reporting. 3.1.1. Management of the Implementation 3.1.1.A. Background and Findings An umbrella strategy was deployed at the time of implementation of the FR service, which spelled out in a coherent manner the main components (organizational, human, material and financial resources) of the plan. This implementation strategy was established jointly with the Corporation d'urgences-sant?, the ASSSM and the SIM based on the following parameters: o territorial map of the agglomeration; o availability of Urgences-sant? trainers; o availability of firefighter recruits. At the start of the project in 2007, a governance structure was put in place, which consisted of a governing board made up of managers of the SIM, the ASSSM and the Corporation d'urgences-sant?, a steering committee made up of the project heads at the SIM and the Corporation d'urgences-sant?, and six working committees. Subsequently, in 2008, SIM management completed the coordination and completion structure by creating: o A governing board overseeing the memorandum of understanding, made up of representatives from the SIM, the MSP, the ASSSM and the Corporation d'urgencessant?; o Joint coordination (the SIM and the Corporation d'urgences-sant?) of working tables composed of two coordinators, one from the SIM and one from the Corporation d'urgences-sant?; o Three working tables (operationalization, training and quality assurance, and technology), each reporting to two individuals, one from the SIM and the other from the Corporation d'urgences-sant?; Auditor General of the Ville de Montr?al 375 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit o Eight working groups whose members came from the SIM, the STI and the Corporation d'urgences-sant?. These groups worked on specific sectors (purchasing and supply, communications and media relations, training, quality assurance, computer-assisted scheduling, telecommunications, operations of the Centre de communication sant?, and reporting). Implementation of the FR service was not to interfere with operations related to the SIM's mission. To ensure this, SIM management planned the training activities for firefighters over three years. It also made provisions for the gradual implementation of the FR service to keep pace with the training of the nine cohorts of firefighter FRs. Table 3 summarizes this plan. Table 3 - Plan of Training Activities and Deployment of the FR Service on the Territory Served by the SIM Phase 1 Phase 2 Phase 3 Phase 1 Phase 2 Phase 3 Phase 1 Phase 2 Phase 3 Boroughs and related municipalities Training 2007 implementation phases - 19 new FR fire stations L'?le-Bizard/Sainte-Genevi?ve/ Spring 2007 Sainte-Anne-de-Bellevue/Pierrefonds/Senneville/ Roxboro/Dorval/L'?le-Dorval Ahuntsic/Cartierville/Montr?al-Nord Fall 2007 Anjou/Rivi?re-des-Prairies/ Pointe-aux-Trembles/Montr?al-Est/ Fall 2007 Mercier/Hochelaga-Maisonneuve 2008 implementation phases - 18 new FR fire stations Saint-Laurent/Mont-Royal/Lachine/LaSalle Start of 2008 Villeray/Saint-Michel/Parc-Extension/ Spring 2008 Saint-L?onard Le Plateau-Mont-Royal/Rosemont/ Fall 2008 La Petite-Patrie 2009 implementation phases - 21 new FR fire stations Ville-Marie/?le Sainte-H?l?ne/ Start of 2009 Mercier/Hochelaga-Maisonneuve Le Sud-Ouest/Verdun/l'?le-des-Soeurs Spring 2009 C?te-des-Neiges Notre-Dame-de-Gr?ce/ Fall 2009 Ville-Marie/C?te-Saint-Luc Implementation Summer 2007 Fall 2007 End of 2007 Spring 2008 Summer 2008 End of 2008 Spring 2009 Summer 2009 End of 2009 Source: SIM, 2007 Overall SIM management put in place structures and tools that enabled it to implement the FR service within the planned completion dates. Through the training program that it instituted, management was able to certify firefighter FRs in accordance with the Auditor General of the Ville de Montr?al 376 2012 Annual Report 5.9. First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al) requirements of the bill 5 and to set up the FR service in the fire stations based on the proposed schedule. 3.1.2. Obligations Set Out in the Memorandum of Understanding 3.1.2.A. Background and Findings The FR service essentially consists of administering the care needed by a victim of a vital medical or trauma emergency, in accordance with established clinical response protocols. In summary, for the SIM, this FR service implies the following actions: o Respond immediately to a request dispatched by the Centre de communication sant? and go to the scene as soon as the call is received; o Perform a summary stabilization of the victim to prevent deterioration of the person's condition while awaiting the arrival of the ambulance technicians; o Communicate the necessary information for case management of the victim by the ambulance technicians and assist them in performing their duties; o Complete a "First Responders" pre-hospital response report. The FR service and responses by firefighter FRs are governed by the bill, as well as by the memorandums of understanding reached by the city (under an urban agglomeration power). Our audit showed that SIM management had fulfilled its obligations, as outlined in the memorandums of understanding reached between the ASSSM and the Corporation d'urgences-sant?, in all respects except the one dealing with radio communication. The radio communication systems of the Corporation d'urgences-sant? and of the city (under an urban agglomeration power) need to be linked to ensure transparent communications between the SIM's communications centre, the Centre de communication sant?, the FRs and the ambulance technicians. To avoid any confusion, article 6 of the memorandum of understanding provides for an agreement to be reached between the Corporation d'urgences-sant? and the SIM aimed at harmonizing the radio codes used by firefighter FRs and ambulance technicians. At the time of our audit, this agreement had still not been reached. Such a situation could lead to confusion in communications between firefighter FRs and the ambulance technicians of the Corporation d'urgences-sant? during a response, with consequences for 5 The Act respecting pre-hospital emergency services (RSQ, chapter S-6.2), among other things, defines the roles and responsibilities of FRs. Moreover, the Regulation respecting the professional activities that may be engaged in within the framework of pre-hospital emergency services and care (Gazette officielle du Qu?bec, Part 2, Vol. 144, No. 5, February 1, 2012, pp. 685-8) describes the activities that FRs may perform. Auditor General of the Ville de Montr?al 377 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit the victims. It should be noted that an administrative directive issued by the SIM to all staff in April 2008 states that clear language and not radio codes are to be used during communications with the Corporation d'urgences-sant?. Given that the mandate of the governing board overseeing the memorandum of understanding is specifically to facilitate interfacing between the parties and also to conduct periodic reviews of the memorandum of understanding upon completion of the implementation and start-up of the FR service and every two years thereafter, it is our opinion that the SIM should request that this board clearly issue a ruling, in writing, about the need for a radio communication agreement. If an agreement is still needed, a completion date should be set and a determination made as to whether or not it requires approval by the urban agglomeration council. If it is no longer needed, the memorandum of understanding approved by the urban agglomeration council in March 2007 should be amended to reflect the parties' intention. 3.1.2.B. Recommendation We recommend that the Service de s?curit? incendie de Montr?al obtain a written ruling from the governing board overseeing the memorandum of understanding as to the need to reach a radio communication agreement (as set out in article 6 of the memorandum of understanding) so that appropriate actions can be taken, either to comply with this memorandum of understanding, or to amend said memorandum based on the parties' intention and, if necessary, establish a completion date and obtain the necessary authorizations. Business unit's response: [TRANSLATION] A meeting with the Corporation d'urgences-sant? is planned for February 2013 to implement a temporary "in the field" interoperability solution while awaiting implementation of the new SERAM radio communication network (expected to be in service by mid-March 2014). A needs assessment for a formal radio communication agreement will be done in the context of the review of the current agreement. (Planned completion: June 2013) 3.1.3. Costs 3.1.3.A. Background and Findings One of the ASSSM and the Corporation d'urgences-sant?'s objectives in entrusting the SIM to deliver the FR service was to reduce population morbidity and mortality rates, in particular by reducing the response times of the pre-hospital emergency services system. Auditor General of the Ville de Montr?al 378 2012 Annual Report 5.9. First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al) Article 8 of the memorandum of understanding, which deals with mutual agreements, translated this objective into action. The city (under an urban agglomeration power) agreed to achieve set target goals in response times and the quality of pre-hospital emergency services. In the case of response times, achievement of the desired outcome was directly reflected in funding, which we will address later. Regarding quality of services, this involved complying with the clinical protocols to be applied during responses (based on the guidelines prepared by the Minist?re de la Sant? et des Services sociaux). Two employees (paramedics) of the Corporation d'urgences-sant? were assigned full-time to the SIM's Division des premiers r?pondants to review these responses. Based on the information obtained from the manager in charge, responses involving cardiac arrest and injections of epinephrine are systematically reviewed, while other responses are reviewed on an as needs basis. In both situations, a follow-up is done with the FRs, if necessary. In return for the city's commitments, the ASSSM agreed to provide the city (under an urban agglomeration power) with funding during the three years of the implementation and startup phase. Following this, proposed funding would be determined based on a performance scale. Table 4 summarizes the parameters of this funding. The amounts are expressed in 2006 dollars and a provision was made for indexation in the memorandum of understanding. Auditor General of the Ville de Montr?al 379 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Table 4 - Funding Parameters Based on the Memorandum of Understanding Implementation and start-up period Consolidation period starting in 2009 Fundingb allocated Government of without taking into account Qu?bec's fiscal eeara performance targets Fundingb based on achieving the performance target of 7 minutes, 59 seconds 2006-2007 2007-2008 2008-2009 $3,000,000 $5,500,000 $5,500,000 Base amount to fund the management infrastructure If at least 70% of the targeted calls meet the performance target If at least 80% of targeted calls meet the performance target If at least 90% of the targeted calls meet the performance target If more than 90% of the targeted calls meet the performance target $1,100,000 base amount $3,300,000 $5,500,000 $7,000,000 $7,000,000 maximum a The government's fiscal year is for the period from April 1 to March 31. Indexed yearly based on the rate set by the Government of Qu?bec. Source: Memorandum of understanding for the implementation of the FR service, March 2007 b At the time of signing of the memorandum of understanding, the city (under an urban agglomeration power) recognized the value of an FR service by designating the SIM to implement this service on condition that the Government of Qu?bec provide sufficient funding for this activity. During our audit, we analyzed the costs of the FR service since its implementation. For the purposes of this analysis, we examined implementation and startup costs (2007-2009), post-implementation costs (2010-2011) and, finally, funding received. Table 5 presents annual cost estimates for the FR service implementation and start-up period, along with real costs incurred, as well as operating costs for the period of consolidation of the service. Lastly, it shows the funding received during the implementation and start-up period, as well as during the period of consolidation of the FR service. Appendix 4.1 details the cost estimates put forward by SIM management. Auditor General of the Ville de Montr?al 380 2012 Annual Report 5.9. First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al) Table 5 - Costs and Revenues of the FR Service During the Implementation and Start-Up Period and the Consolidation Period (in thousands of dollars) Implementation and start-up period 2007 2008 2009 Total 2007-2009 Consolidation period a 2010 2011 Total 2012 2007-2011 estimates Costs Implementation No. of fire stations to be integrated 19 18 21 58 Anticipated costs $4,290.7 $3,626.3 $4,436.8 $12,353.8 Real costs $4,290.7 $4,554.0 $5,845.7 $14,690.4 $0.0 ($927.7) ($1,408.9) ($2,336.6) $234.3 Variance (anticipated - real) $0.0 $234.3 $0.0 ($2,102.3) Operating expenses No. of active fire stations 27 45 66 66 65 b 65 65 Anticipated costs $2,501.1 $5,481.4 $8,389.1 $16,371.6 $9,798.1 $13,619.1 $39,788.8 $13,899.1 Real costs $2,501.1 $5,116.0 $8,275.0 $15,892.1 $12,187.8 $13,603.9 $41,683.8 $14,095.8 $0.0 $365.4 $114.1 Discrepancy (anticipated - real) $479.5 ($2,389.7) $15.2 ($1,895.0) ($196.7) Total anticipated costs $6,791.8 $9,107.7 $12,825.9 $28,725.4 $10,032.4 $13,619.1 $52,376.9 $13,899.1 Total real costs $6,791.8 $9,670.0 $14,120.7 $30,582.5 $12,187.8 $13,603.9 $56,374.2 $14,095.8 Variance (anticipated - real) $0.0 ($562.3) ($1,294.8) ($1,857.1) ($2,155.4) Revenues Indexed revenues Variance (revenues - real costs) a b c $7,207.5 $5,500.0 $5,788.6 $15.2 ($3,997.3) ($196.7) c $18,496.1 $6,003.0 $10,571.4 $35,070.5 $7,953.0 $415.7 ($4,170.0) ($8,332.1) ($12,086.4) ($6,184.8) ($3,032.5) ($21,303.7) ($6,142.8) Real costs represent the projected costs to December 31, 2012. Fire station 79 in Hampstead was closed at the time of implementation of the FSCP. Amounts indexed yearly based on the rate set by the Government of Qu?bec. In the case of the implementation and start-up period, our analysis of the results led us to conclude that the real costs of the FR service were 19% higher than the cost estimated originally in the decision-making summary presented to the urban agglomeration council in March 2007, or $14,690,400 compared with $12,353,800. This increase of $2,336,600 can be attributed to: o Certifying and maintaining certification for a higher number of firefighters targeted ($185,300); o The cost of overtime hours to replace firefighters being trained, since the SIM's basic mission could not be compromised ($242,700); o Upward adjustment in the acquisition costs of equipment and various contingency costs ($484,400); and o Calling on the services of firefighters as attendants during FR responses instead of instructors from the Corporation d'urgences-sant? as originally planned ($1,424,200). Auditor General of the Ville de Montr?al 381 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit The operating costs of the FR service increased steadily but did not exceed estimates. The FR service was gradually activated in the fire stations as firefighter first responders received their certification. In the case of the consolidation period, we concluded that operating costs rose significantly during 2010. This can be explained in large part by the retroactive application of an arbitration settlement awarded during that year, in response to claims made by the Association des pompiers de Montr?al. The arbitration decision ordered that the annual bonus paid to firefighter FRs be increased from $1,500 to $1,950 and that the hourly bonus be increased from $1.10 to $1.50. Following this, costs continued to rise in 2011, with the certification of 190 additional firefighters who had completed the necessary training. Consequently, they were paid an amount equivalent to the training costs and annual bonus. It should be noted that, pursuant to the most recent collective agreement signed in March 2012, the annual bonus and hourly bonus will now be included in the salary of firefighter FRs and will come into force as of January 1, 2012. The impact of these increases on operating costs is fairly substantial. Finally, the ASSSM's contribution is the only source of external revenue that supports the FR service. As mentioned earlier, the amount allocated to March 31, 2009, was fixed in the memorandum of understanding without taking into account the performance target. As of April 1, 2009, this funding was subject to performance targets. We concluded that the SIM was entitled to the maximum amount permitted under the memorandum of understanding. In 2011, an amount of $10,571,400 was paid to the city (under an urban agglomeration power) and included adjustments based on the performance of the FR service for the periods of 2009-2010 and 2010-2011. In fact, the SIM achieved a response time of 4 minutes and 29 seconds in 2009, 4 minutes and 19 seconds in 2010, and 4 minutes and 22 seconds in 2011, for an average response time of 4 minutes and 23 seconds compared with the target response time for the system of 7 minutes and 59 seconds. The FR service provided by the SIM, therefore, met the highest expectations set out in the memorandum of understanding. In spite of indexation and the service's progression based on achieving its performance targets, we concluded that the contribution did not cover all the operating expenses of the FR service provided by the SIM. For the first five years covered by the memorandum of understanding (2007 to 2011), we concluded that the amount received was $35,070,500, while the total cost incurred by the SIM for the implementation and operation of the FR service was $56,374,200. The result, at December 31, 2011, was a net cumulative cost to the agglomeration of $21,303,700. While Auditor General of the Ville de Montr?al 382 2012 Annual Report 5.9. First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al) 2012 is not yet over, a shortfall can be expected in the range of $6,142,800, for a net cumulative cost to the agglomeration of $27,446,500. The shortfall can certainly be explained in part by the arbitration decision, which was unknown at the time of the signing of the memorandum of understanding. Based on the information received, another explanation is the fact that the ASSSM's estimated contribution was based on a reference number of 50,000 responses. In fact, the number of responses has been greater than this since 2009. In 2011, the SIM responded to more than 72,426 calls or 44% more than originally anticipated. This situation had repercussions on costs related to the increased use and operation of fire vehicles (maintenance, fuel, wear and tear). Again according to information received, the SIM advanced the premise of using existing resources (firefighters, trucks and fire stations) and drawing inspiration from response models already in place in Canada and the United States. In conclusion, implementation of the FR service on the territory of the Montr?al agglomeration represents an increase in the level of service provided not only to citizens but also to non-residents who are in the city temporarily. This level of service has contributed to improving the response time of pre-hospital services. According to reports prepared by Urgences-sant?, the FR service has also contributed to improving the survival rate of victims of cardiac arrest during calls. According to statistics are for the regions of Laval and of Montr?al, the rate of survival has risen from 12.2% in 2007 to 24.3% in 2010. However the cost overage noted as of Year 2 is being assumed by the agglomeration. In our opinion, the SIM should perform an analysis to determine, on the one hand, the causes of this overage and, on the other hand, to explore possible solutions to reduce the operating costs it is assuming, such as finding other sources of funding, for example charging fees. This kind of analysis would also enable SIM management to approach the ASSSM to obtain financial compensation to cover more adequately the real cost of the services rendered. Auditor General of the Ville de Montr?al 383 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.1.3.B. Recommendation We recommend that the Service de s?curit? incendie de Montr?al, in an effort to achieve fiscal balance for the first responder service: o Approach the Agence de la sant? et des services sociaux de Montr?al to obtain adequate funding to cover the real cost of the services rendered, given the higher percentage of responses (more than 40%) compared with initial estimates, as well as the arbitration ruling rendered in 2010; o Look into the possibility of other sources of funding; o Analyze various solutions to reduce operating costs. Business unit's response: [TRANSLATION] A comprehensive review of the memorandum of understanding is under way (begun in 2012). One of the main objectives of this review is to explore avenues for cost savings, including: o Reducing the number of FR interventions by regulating calls to determine more accurately when to dispatch resources based on the level of the emergency and the location of the resources; o Improving supply processes; o Improving the teaching structure. The project committee's recommendations are to be submitted, in March 2013, to the governing board overseeing the memorandum of understanding. (Completed, March 2013) Following this, the governing board overseeing the memorandum of understanding is to submit them to the Minist?re de la Sant? et des Services sociaux accordingly. (Planned completion: December 2013) 3.1.4. Accountability Reporting 3.1.4.A. Background and Findings In order for managers to be informed about the progress of the projects with which they have been entrusted, accountability mechanisms need to be put in place. Management reports should be produced periodically and contain information that is sufficiently relevant to assess the extent to which target objectives have been achieved and at what cost. These reports should be useful for decision-making on a timely basis. In this regard, our audit consisted in inquiring about the mechanisms that were put in place to be kept abreast of how activities surrounding the FR service were managed. Auditor General of the Ville de Montr?al 384 2012 Annual Report 5.9. First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al) During the implementation period (2007-2009), the project team produced follow-up reports for SIM management. These reports provided information on how implementation of the FR service was progressing in terms of cohorts that were trained and the phasing in of the service in fire stations based on the program. They made it possible, therefore, to evaluate to what extent the target objectives for implementation of the FR service had been achieved. At the end of the implementation phase, responsibility for the FR service was entrusted to the Division des premiers r?pondants. Beginning in 2010, this division continued to produce and send progress reports to SIM management on the work completed during the year. We concluded, nevertheless, that there had been no formal accountability report provided to the Direction g?n?rale regarding the FR service. Furthermore, article 4 of the memorandum of understanding stated that the governing board overseeing the memorandum of understanding should produce a periodic evaluation at the end of the implementation and start-up period and every two years thereafter. While exchanges did take place on an ongoing basis between the SIM, the ASSSM and the Corporation d'urgences-sant?, no such evaluation reports were produced. In conclusion, no formal accountability report was provided to the Direction g?n?rale regarding the FR service that would have enabled management to ensure appropriate actions were taken in a timely manner. In our opinion, there should have been accountability reporting not only for work done in the area of responses but also for costs incurred by the agglomeration for services rendered, as well as information on the city's contribution (under an urban agglomeration power) to achieving the objectives of the FR service, reducing population morbidity and mortality rates, including by reducing the response time of the pre-hospital emergency services system. Finally, the Direction g?n?rale should also have been informed of the results of the periodic evaluation of the memorandum of understanding. 3.1.4.B. Recommendation We recommend that the Service de s?curit? incendie de Montr?al request that the governing board overseeing the memorandum of understanding evaluate said protocol, as stated in article 4, in order to ensure compliance with this article. Business unit's response: [TRANSLATION] The meetings and work of the governing board overseeing the memorandum of understanding will be planned as stated in article 4 of the memorandum of understanding. (Completed, March 2013) Auditor General of the Ville de Montr?al 385 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.1.4.C. Recommendation We recommend that the Service de s?curit? incendie de Montr?al periodically produce an accountability report for the Direction g?n?rale on the status of: o Responses carried out by the first responders service; o Costs incurred on behalf of the agglomeration; o The contribution of the city (under an urban agglomeration power) in achieving the target objectives of the first responder service, i.e. reducing population morbidity and mortality rates, including by reducing response time of the prehospital emergency services system; o Results of the periodic evaluation of the memorandum of understanding reached with the Agence de la sant? et des services sociaux de Montr?al and the Corporation d'urgences-sant?; so that the necessary corrective measures can be applied to the management of the first responder service or so that the appropriate submissions are made to the Government of Qu?bec. Business unit's response: [TRANSLATION] During their regularly scheduled statutory meetings, the SIM manager reports to the city manager on ongoing achievements, risks and solutions. Costs incurred on behalf of the agglomeration are presented yearly to the Commission de la s?curit? publique as part of the presentation of the SIM activity report, as well as at the time of preparing and presenting the SIM's budget. The city's contribution to achieving the target objectives of the FR service is presented annually to the Direction g?n?rale and the Commission de la s?curit? publique during the presentation of the SIM's activity report including the number of responses, the profile of the types of calls, and the response time of the pre-hospital emergency services system. Moreover, the report on the rate of resuscitation is included as soon as it is published by the Corporation d'urgences-sant?, which is the only body authorized to provide the clinical data. The periodic examination of the memorandum of understanding will be included on the agenda of the regularly scheduled statutory meetings between the SIM manager and the city manager. (Planned completion: December 2013) Auditor General of the Ville de Montr?al 386 2012 Annual Report 5.9. First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al) 3.2. 2009-2013 Fire Safety Cover Plan The FSCP implementation plan, as adopted in December 2008, is based essentially on the effective completion of the following three components: o Prevention component: this component includes six programs that are aimed at improving fire safety on the territory of the Montr?al agglomeration by reducing fires and their repercussions. Below is an overview of the six programs: - Program 1 - Incident assessment and analysis: Use computer management tools and the Fire Commissioner's Office to assess and analyze incidents in order to determine prevention, inspection and public education activities, - Program 2 - Municipal by-law: Adopt and apply an agglomeration fire prevention bylaw aimed at updating and standardizing the by-law in effect in the boroughs and related municipalities, taking into account their own particular needs, - Program 3 - Smoke detectors: Increase the number of operational smoke detectors, currently estimated at 70%, to reach a penetration rate of 75% to 80% in rental units and residences, especially in at-risk neighbourhoods, - Program 4 - Periodic inspection of the highest fire hazards: Continue to carry out periodic inspection of the highest fire hazards, using five specific action subprograms aimed at reducing the occurrence and consequences of fires in buildings classed as high-risk and very high-risk, - Program 5 - Public awareness activities: Boost public awareness activities that encourage the adoption of safe fire prevention behaviours and take appropriate measures in the event of fire, - Program 6 - Fire Commissioner's Office: Set up a Fire Commissioner's Office that brings together the competencies of experts from the SIM and the Service de police de la Ville de Montr?al (SPVM) to research the causes of fire and conduct fire investigations. The latter program was inserted into the FSCP for information purposes; o Response component - strike force: the goal of this component is to organize an optimum and effective strike force, by: - Closing fire station 79, - Building a training centre in the West Island, - Building three new fire stations (32, 59 and 63), - Refurbishing four fire stations (51, 56, 65 and 67), - Refurbishing the fire prevention offices, - Acquiring new vehicles, including two fire engines, two protection unit vehicles, two water trucks, and two aerial devices for responses, - Acquiring light vehicles for fire prevention; Auditor General of the Ville de Montr?al 387 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit o Water supply component: this component is aimed at improving the supply of water in the event of fire on the territory of the agglomeration, especially on the West Island, by: - Building underground reservoirs equipped with dry hydrants: at Parc de l'Anse-?-l'Orme, at the Morgan Arboretum, near mont?e Wilson and chemin Bord-du-Lac in ?le-Bizard, near avenue Angus in Senneville, - Installing two water supply points at the ends of the Senneville water distribution system: avenue Pacific, the westernmost point of Anse-?-l'Orme. To ensure that all activities for which it is responsible and for which it acts as coordinating manager are completed successfully, SIM management prepared a five-year implementation plan that included its human resources and funding needs. In the area of human resources, the plan provides for the creation of 68 new prevention positions and 126 new response positions (including four positions for the West Island training centre). In the area of funding, cost estimates for each of the five years are presented in summary form in Table 6. Table 6 - Cost Estimates for Implementation of the FSCP (2009-2013) (in thousands of dollars) 2009 2010 2011 2012 2013 Recurring at term Operating budgeta Response Prevention 1,452.1 3,146.5 3,102.2 4,273.8 6,519.2 6,147.6 9,222.6 8,478.4 12,309.7 8,747.3 15,330.4 8,760.7 Total 4,598.6 7,376.0 12,666.8 17,701.0 21,057.0 24,091.1 2009 2010 2011 2012 2013 Total TCEP Response Prevention Total a 2,226.9 921.6 3,148.5 6,444.6 646.7 7,091.3 1,917.4 937.6 2,855.0 11,438.5 334.0 11,772.5 5,161.0 340.8 5,501.8 27,188.4 3,180.7 30,369.1 Estimates include debt servicing (interest to pay down the debt). Source: SIM, April 2008. Since the implementation plan for the FSCP (2009-2013) was nearing its fifth and final year, in the course of our audit we evaluated the extent to which progress had been made based on the plan and what had been adopted by the authorities. We began by examining the Auditor General of the Ville de Montr?al 388 2012 Annual Report 5.9. First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al) work that had been completed on each of the components of the FSCP. We then looked at project management as set out in the implementation plan for the FSCP, and at the costs of implementation. Finally, we examined accountability reporting for the FSCP. 3.2.1. Completion Status The implementation plan is spread over five years, 2009-2013. The SIM defined and programmed in the delivery of all the projects planned for this period for each of the three components (prevention, response - strike force, and water supply). For the purposes of our audit, we evaluated the extent to which the completion dates were met or had been in the process of being met for each of the components. Table 7 presents a summary of the final expected results for each of the components, as well as the planned delivery year, as shown in the FSCP that was adopted in 2008 by the urban agglomeration council. Auditor General of the Ville de Montr?al 389 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Table 7 - Main Anticipated Results and Delivery Year Based on FSCP Components Components Prevention o Program 1: Evaluation and analysis of incidents supported by an operational SGAPa in 2011. o Program 2: Adoption of an agglomeration fire prevention by-law in 2012. Response - strike force Water supply o Construction of 3 new fire o Establishment of 4 supply stations and a training centre: points (water reservoirs and dry hydrants), as well - Fire station 59 in 2010 as 2 new water mains - Fire station 63 in 2012 between 2010 and 2011. - Fire station 32 in 2013 - West Island training centre in o Establishment of 2 supply 2012 points (underground reservoir and dry hydrant) o Refurbishment of 4 fire stations: between 2012 and 2013. - Fire stations 56, 65 and 67 in 2010 - Fire station 51 in 2012 o Program 3: - Increase in the number of operational smoke detectors (19 boroughs and 15 related municipalities) o Refurbishment of the offices of - Annual verification of the presence the prevention teams in 2012. of operational smoke detectors in at least 50,000 rental units and o Acquisition of additional residences vehicles: End of 2013 - Prevention: 15 light vehicles: o Program 4: 3 in 2010 Completion of 22,500 class 3 and 4 4 in 2011 inspections over five years 4 in 2012 End of 2013. 4 in 2013 - Response: o Program 5: 9 heavy vehicles: Increase in public awareness 2 in 2009 activities, ongoing 2 in 2010 2 in 2011 o Program 6: 1 in 2012 Setting up of a fire commissioner's 2 in 2013 office (included in the FSCP for information purposes) a Fire prevention management system. Examination of the implementation plan led us to the following conclusions regarding each of the three components. 3.2.1.1. Prevention Component 3.2.1.1.1. Program 1 - Evaluation and Analysis of Incidents 3.2.1.1.1.A. Background and Findings The aim of this program is to use computer tools for the purposes of analysis and to determine prevention actions to be taken. The FSCP was particularly concerned with Auditor General of the Ville de Montr?al 390 2012 Annual Report 5.9. First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al) continued development and the renewal of the fire prevention management system (SGAP) in 2011. Our audit showed a considerable delay in the development and implementation of the SGAP. In fact, as of 2009, the SIM had started reviewing the former SGAP to adapt it to its needs. After concluding that the SGAP did not have the necessary functionalities to ensure the management and execution of preventive activities and that it had data integrity problems, the SIM turned to the STI to develop and implement a complete technical solution. In 2011, the STI began working on a computer tool, which has since become known as the integrated fire prevention activities management system (SGIAP). For this project, the STI also made provisions to acquire portable computers that would enable prevention officers to use the SGIAP in the field in real time. According to the planned completion date set by the STI, this system will not be completely delivered until 2014, one year after the deadline for implementation of the FSCP (2009-2013). Our work also led us to conclude that the costs of the initial SGAP project had not been estimated by the SIM at the time of developing the FSCP. It was only when the STI took charge of the SGIAP project, in 2011, that cost estimates appeared in the STI's TCEP. As of September 20, 2012, the estimated total cost of the SGIAP, when completed, was $4,176,000. We are of the opinion that the SIM should have properly defined its needs when the FSCP was being produced, which would have made it possible to establish a more realistic completion date and also to estimate the costs of the project more accurately. Consequently, the new completion date planned for 2014 does not correspond to the FSCP sent to the MSP and which permitted the city (under an urban agglomeration power) to obtain a certificate of compliance. Regarding the FSCP, the Fire Safety Act states that any change or postponement in the planned completion deadlines must be authorized by the MSP. This authorization is in addition to the attestation of compliance. The Fire Safety Act states that the FSCP can be amended in certain cases and that this must be reviewed during the sixth year while complying with the procedure set out in the bill. In our opinion, in the current situation, the SIM should obtain the MSP's approval to postpone this project. Moreover, since prevention represents a cornerstone in meeting the FSCP's objectives, in particular the decrease in fires, loss of life and damage costs, the development of this application is essential to the management of prevention activities. We are of the opinion, therefore, that every effort should be made to comply with the new planned completion date. It should be specified that the start up of this system and the operation of its database will Auditor General of the Ville de Montr?al 391 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit enable the SIM to manage inspection plans targeting high-risk areas and to determine priority inspections. It will facilitate awareness-raising activities by contributing to determining at-risk sectors. This targeted planning should improve the prevention teams' performance. 3.2.1.1.1.B. Recommendation We recommend that the Service de s?curit? incendie de Montr?al take all necessary means, when including a new project as part of amending or revising the fire safety cover plan, so that the latter is well defined and rests on a reliable estimation of costs in order to be able to evaluate with greater certainty the planned completion dates and costs. Business unit's response: [TRANSLATION] Mechanisms are in place to define and evaluate costs, using the following tools: o Participation of the Planification strat?gique and Ressources financi?res divisions; o Use of the city's decision-making record management system (GDD) to document files and notify the following authorities: - Direction g?n?rale, - Commission de la s?curit? publique, - MSP, - Internal and external partners. (Planned completion: December 2013) 3.2.1.1.1.C. Recommendation We recommend that the Service de s?curit? incendie de Montr?al obtain the approval of the Minist?re de la S?curit? publique to delay the timeline for implementing the integrated fire prevention activities management system to 2014 in order to comply with the Fire Safety Act regarding fire safety cover plans. Business unit's response: [TRANSLATION] Letter issued explaining the reasons and rationale for the postponement. The annual statement will be submitted in March 2013, through the official GDD. (Completed, March 2013) Auditor General of the Ville de Montr?al 392 2012 Annual Report 5.9. First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al) 3.2.1.1.1.D. Recommendation We recommend that the Service de s?curit? incendie de Montr?al take the necessary measures to ensure compliance with the new timeline for delivery of the integrated fire prevention activities management system so that it can facilitate inspection and awareness-raising actions. Business unit's response: [TRANSLATION] Good project management practices, combined with regularly scheduled statutory meetings periodically with partners (the STI, the DSTI), will ensure compliance with delivery plans. o Governance; o Planning and timeframe; o Change requests; o Follow-up; o Documentation; o Minutes of regularly scheduled statutory meetings. (Planned completion: December 2013) 3.2.1.1.2. Program 2 - Municipal By-Law 3.2.1.1.2.A. Background and Findings In developing the FSCP, the SIM's intention was to have a single fire prevention by-law adopted by all the cities in the agglomeration of Montr?al. To do this, an amendment to the bill was necessary to give the urban agglomeration council clear jurisdiction in this area. The Direction du contentieux (currently the Service des affaires juridiques et de l'?valuation fonci?re) submitted a request to this effect in 2007. In the absence of any decision by the government, the SIM sought approval of the Fire Prevention By-law by the city council on January 23, 2012, which would apply to the city's 19 boroughs. As regarded the related municipalities, the SIM made representations so that each municipal council approve the same by-law. Based on the information and documents provided as of September 17, 2012, two-thirds of the 15 related municipalities had adopted said by-law. In our opinion, it is important that all the related municipalities adopt the fire prevention by-law proposed by the SIM in order to comply with the 2012 completion date set out in the implementation plan of the FSCP. This would enable prevention officers to apply a uniform by-law throughout the territory of the Montr?al agglomeration regarding, in particular, minimum prevention standards and administrative regulations. Conversely, a by-law that is not uniform throughout the territory of the Montr?al agglomeration could lead to problems when the time comes to apply the by-laws. Auditor General of the Ville de Montr?al 393 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.2.1.1.2.B. Recommendation We recommend that the Service de s?curit? incendie de Montr?al take the necessary measures to encourage all the related municipalities to adopt the Fire Prevention Bylaw proposed by the Service de s?curit? incendie de Montr?al, not only to comply with the planned completion date stated in the implementation plan of the fire safety cover plan but also to encourage uniform application of the by-law across the territory of the agglomeration. Business unit's response: [TRANSLATION] Pursue the activities outlined in the current action plan in the related municipalities to have a fire prevention by-law adopted. (Planned completion: December 2013) 3.2.1.1.3. Program 3 - Smoke Detectors 3.2.1.1.3.A. Background and Findings The aim of this program is to increase the number of operational smoke detectors in rental units and residences, in particular in at-risk neighbourhoods, so that the sounding of an alarm will help alert residents to the presence of a disaster in their home and lead them to exit the premises. When the FSCP was conceived, the SIM had estimated that 70% of smoke detectors were operational on the territory of the Montr?al agglomeration. According to the implementation program, the distribution of new smoke detectors, as well as visits to check that smoke detectors were operational, should have been completed in order to increase the number of operational smoke detectors from 75% to 80%. Moreover, yearly verifications for at least 50,000 rental units and residences from 2010 to 2013 were also planned. In fact, from 2009 to 2011, the SIM purchased 27,000 smoke detectors that it distributed free of charge. According to the 2009-2011 report, more than 100,000 visits, including 30,000 rental units, were completed by the end of 2011. At the time of our audit, the statistical reports produced by the Centre de services de l'expertise et du d?veloppement de la pr?vention indicated more specifically that 55,527 smoke detector checks were carried out in 2010, and 99,610 in 2011. At the time of our audit, the data for 2012 had not been compiled to allow us to calculate the rate of functional smoke detectors. Even though more visits were completed than planned in 2010 and 2011, we concluded that the SIM had measured the rate of operational smoke detectors solely for 2011. We are of the opinion that the SIM should have measured this rate as of 2010 when it began the Auditor General of the Ville de Montr?al 394 2012 Annual Report 5.9. First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al) visits as part of its program. In regard to 2011, based on the initial information received, the rate of operational smoke detectors was 78.2%. Once corrective measures were taken, in particular the distribution of batteries and smoke detectors, as well as the issuing of a notice of non-compliance, the rate of functional smoke detectors rose to 92.1%. While 2012 is still not over, the SIM should nevertheless calculate this rate during the year in order to evaluate the extent to which the objective is being reached and to adapt awareness-raising efforts, if necessary. Moreover, although the rate of functional smoke detectors was calculated for 2011, we concluded that this information was not included in the yearly reports tabled at the urban agglomeration council and sent to the Ministre de la S?curit? publique (2009-2010 report and 2009-2011 report), although this objective was clearly mentioned in Program 3 related to smoke detectors. We are of the opinion that the information regarding this rate should be mentioned in the reports that are tabled at the urban agglomeration council and sent to the Ministre de la S?curit? publique. 3.2.1.1.3.B. Recommendation We recommend that the Service de s?curit? incendie de Montr?al gauge the rate of operational smoke detectors during the year, for both 2012 and 2013, in order to evaluate to what extent the objective of the "Smoke Detector" Program is being achieved and to adapt awareness-raising activities, if necessary. Business unit's response: [TRANSLATION] For 2012, information about the rate of operational smoke detectors will be included in the program reports. For 2013, information about the rate of operational smoke detectors will be included in the program reports. In collaboration with the regional managers, implement measures adapted to educational activities, where necessary. (Planned completion: December 2013) 3.2.1.1.3.C. Recommendation We recommend that the Service de s?curit? incendie de Montr?al provide information about the rate of operational smoke detectors in the reports sent to the urban agglomeration council and to the Ministre de la S?curit? publique in order to be aware of whether the target objective has been reached. Auditor General of the Ville de Montr?al 395 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Business unit's response: [TRANSLATION] Official link to the MSP. Yearly report issued. 2012 report submitted in March 2013 through the official GDD, in the form of documents showing the level of progress. (Completed, March 2013) 3.2.1.1.4. Program 4 - Periodic Inspections of the Highest Risks 3.2.1.1.4.A. Background and Findings This program institutes periodic inspections of buildings that present high risks (class 3) and very high risks (class 4). The objective of the SIM according to the FSCP was 22,500 buildings, without specifying the distribution between categories 3 and 4. According to the information that appeared in the 2009-2011 report, only 4,300 inspections of categories 3 and 4 were carried out up to 2011 (20%). It should be specified that it was not possible to obtain distinct data for each of the categories. At the time of our audit, the follow-up report of the 2012 engagements, produced by the Centre de services de l'expertise et du d?veloppement de la pr?vention as of September 28, 2012, indicated that 9,383 additional inspections had been carried out, for a total of 13,683 inspections (61%). Therefore, 8,817 inspections (39%) still need to be carried out by December 31, 2013, to reach the 22,500 inspections planned in the implementation plan of the FSCP. Moreover, in order to evaluate the results obtained following these inspections, as well as the scope of the follow-up that was to be done by the prevention teams, we attempted to obtain the number of notices of non-compliance that were issued. The SIM was unable to provide us with the information requested. Reaching this objective was to be facilitated by the implementation of SGIAP, which would contribute to improving the performance of the prevention teams by orienting the inspection activities to the highest risk sites. The delay in implementing this application could, therefore, ultimately have consequences on achieving the goal of 22,500 inspections. Although the final result of this activity is only expected in 2013, we are of the opinion that the situation requires closer planning and monitoring in order to comply with the planned completion date. Auditor General of the Ville de Montr?al 396 2012 Annual Report 5.9. First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al) 3.2.1.1.4.B. Recommendation We recommend that the Service de s?curit? incendie de Montr?al implement an inspection program for class 3 and class 4 buildings, complete with measurable objectives on a monthly basis, along with monitoring and evaluation measures to enable it to carry out the 22,500 inspections planned for December 31, 2013, in accordance with the fire safety cover plan. Business unit's response: [TRANSLATION] Develop an action plan setting out the objectives for inspections on a monthly basis, as well as a process to monitor deliverables to ensure that objectives are met. (Completed, March 2013) 3.2.1.1.5. Program 5 - Public Education Activities 3.2.1.1.5.A. Background and Findings This program consists in developing and applying tools and programs that encourage the adoption of safe behaviours to prevent fire. The awareness-building activities defined in the FSCP involve, in particular, the establishment of a collaboration of partners in contact with seniors, the determination of the highest risk sectors and vulnerable clients, and the aim of equipping fire stations with educational and promotional material to assist firefighters in their awareness-raising activities during visits to the public. Our audit led us to conclude that statistics were compiled for the years 2010 to 2012 regarding, among other things, the number of educational talks in day care centres and the number of talks in seniors residences. In 2011, educational and promotional material was delivered to eight fire stations for validation, with the intention of making it available in all fire stations. Although the SIM did compile statistics for awareness-raising activities related to the highest risk sectors and vulnerable clients, our audit led us to conclude that measurable objectives had not been established. In our opinion, determining such measurable objectives facilitates the planning of activities and makes it possible to evaluate whether these have been achieved in relation to a target. 3.2.1.1.5.B. Recommendation We recommend that the Service de s?curit? incendie de Montr?al determine, for the year 2013 and at the time of reviewing the fire safety cover plan, measurable objectives for public awareness activities in order to facilitate the planning of activities and to enable their completion in relation to a target. Auditor General of the Ville de Montr?al 397 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Business unit's response: [TRANSLATION] Develop an action plan setting out the objectives for public education activities, as well as a process to monitor deliverables to ensure that objectives are met. (Completed, March 2013) 3.2.1.1.6. Program 6 - Fire Commissioner's Office 3.2.1.1.6.A. Background and Findings Although the fire commissioner's office was included in the FSCP, it is not part of the actual implementation plan since it is included for information purposes only. According to the people we met, the fire commissioner's office was included in the FSCP for information purposes only, since it required human and financial resources, as well as legislative amendments in order to be implemented, the latter being the main reason that explained the commissioner's office not being realized. Moreover, the persons we met mentioned to us that good practices already exist around the holding of inquiries following a fire. In fact, as soon as a SIM inspector detects suspicious elements, the file is transferred to the SPVM and an inquiry is opened. However, opinions are divided since a feasibility study carried out jointly by the SIM and the SPVM in 2006 recommended setting up a fire commissioner's office. One such recommendation advanced the further rationale that external funding from the provincial government is provided for in the Fire Safety Act and the Charter of Ville de Montr?al 6 anticipates recovering certain amounts from the insurers who do business on its territory. According to the benefits outlined in the FSCP, setting up a Fire Commissioner's Office with the collaboration of the police, firefighters and the technical and scientific staff involved, would make it possible to implement fire prevention solutions. We are of the opinion that the implementation of such an office, on the eve of reviewing the FSCP, should be re-evaluated. We also wonder if including the Fire Commissioner's Office for information purposes only was planned in the context of the current FSCP. In our opinion, if an action or measure is not really an integral part of the FSCP or that the latter does not provide for it, it should not be included in the FSCP to avoid any confusion among authorities. 6 RSQ, chapter C-11.4. Auditor General of the Ville de Montr?al 398 2012 Annual Report 5.9. First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al) 3.2.1.1.6.B. Recommendation We recommend that the Service de s?curit? incendie de Montr?al re-evaluate, at the time of reviewing the fire safety cover plan, the relevance of including the implementation of a Fire Commissioner's Office in order to strengthen fire prevention. Business unit's response: [TRANSLATION] The SIM agrees to begin re-evaluating the feasibility of putting in place a Fire Commissioner's Office, taking into account common management principles, as well as the involvement of various partners (the SIM, the city, the SPVM, insurers, MSP). (Planned completion: December 2013) 3.2.1.1.6.C. Recommendation We recommend that the Service de s?curit? incendie de Montr?al ensure that all actions and measures that will figure in the revised fire safety cover plan be realisable and provided for in the implementation plan in order to avoid entering them as being for information purposes only and thus creating confusion among authorities. Business unit's response: [TRANSLATION] The SIM has noted the recommendation. (Planned completion: December 2013) 3.2.1.2. Response Component - Strike Force 3.2.1.2.A. Background and Findings Completion of the implementation plan for the response component - strike force is, on the one hand, dependent upon the available budget of the TCEP and, on the other hand, on the planned construction or refurbishment of the fire stations. As regards the plan that was tabled, we concluded the following. o The relocation of fire station 63 is no longer necessary because of the changes brought about in the Highway 20 interchange in Dorval. The current fire station will remain at the same location and is being refurbished. Completion of the refurbishment work is planned for March 2014. Auditor General of the Ville de Montr?al 399 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit o Construction of fire station 59, planned for 2010, will instead be completed in 2013. Fire station 32 will be delivered as planned in 2013. o Construction of the West Island training centre, initially planned for 2012, is in jeopardy since it does not appear in the recently tabled 2013-2015 TCEP. Based on the information received, the absence of the West Island training centre would not jeopardize the training activities of the SIM's firefighters, since training will continue to be held in the East End training centre. However, this decision carries with it additional costs related to transportation, maintenance and travel to the East End training centre by West Island response units. Although the SIM indicated that, when the FSCP was being developed, initial cost estimates for the training centre were in the order of $6,061,000 and that these were revised upward to $9,850,000 in 2012, SIM management did not provide analyses or studies to support the project as initially planned. o The refurbishment of fire stations 56 and 65 was completed as planned in 2010. o The refurbishment of fire station 51, planned for 2012, is under way with likely delivery in 2013. In the case of the refurbishment of fire station 67, planned for 2010, a provisional (partial) receipt was accepted by the DSTI in December 2011. o Refurbishment of the offices was completed and proved to be less costly than predicted. The SIM decided to occupy and refurbish a building (at 200, rue de Bellechasse) that belonged to the city. Moreover, slight refurbishing was done in the region. o To support strike force and prevention activities, SIM management planned to acquire 15 light vehicles for prevention and nine heavy vehicles for response. Based on the information collected at October 22, 2012, all the light vehicles and eight of the heavy vehicles had been received. A heavy vehicle assigned to fire station 59 and planned for 2013 should be delivered in 2014. Comparison between the initial cost estimates and the cost estimates at October 16, 2012, show a cost overrun of $8,360,000 compared with initially planned costs of $30,370,000, as shown in Table 8. These overruns are reflected as much in fire station construction as in refurbishment projects. After eliminating the work on fire station 63 and the refurbishment of the prevention offices, which was no longer required as initially planned, we concluded that all the other projects, once completed, would incur cost overruns totalling $13,342,000. Auditor General of the Ville de Montr?al 400 2012 Annual Report 5.9. First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al) Table 8 - Response Component - Strike Force Initial Budget Estimates Compared with 2012 Revised Estimates (in thousands of dollars) Estimates 2009 2008 real 2012 2013 projection projection Variance (total planned) 2010 real 2011 real 823 16 47 664 1,303 240 5,972 6,216 2,435 7,475 7,566 2,675 9,850 2,429 2,709 (2,272) 3,789 89 9 1,000 1,068 2,157 675 505 3,123 1,361 359 301 2,373 220 (2,710) Total Construction Fire station 32 Fire station 59 Fire station 63 West Island training centre 5,046 4,857 4,947 6,061 Refurbishment Fire station 51 Fire station 56 Fire station 65 Fire station 67 Refurbishment of offices - prevention Acquisition of vehicles Total 796 316 204 750 24 67 57 2,930 180 2,371 27 307 40 918 4,463 30,370 642 411 388 328 1,632 895 1,039 4,484 21 3,182 4,204 4,436 16,730 38,730 8,360 Source: SIM. In the case of fire station construction projects, these cost overruns are mainly due to the use of cost estimates using a non-actualized reference dating from the construction of the most recent fire station (between 2002 and 2005). The cost overruns linked to the fire station refurbishment projects, on the other hand, are explained by the fact that constraints and requirements were not properly taken into consideration, for example constraints related to the fire stations themselves (old fire stations), urban planning by-law requirements of the boroughs and related municipalities, and the implementation of the new sustainable development policy. This conclusion regarding cost overruns raises questions about the reliability of the cost estimates originally taken into account when the FSCP was developed. In our opinion, the SIM should have ensured that the cost estimates for each project were made using an actualized cost referential and should have taken into account constraints and requirements that were likely to change the costs. It would also have been necessary for the cost estimation to have been reliable and based on the costs of materials and labour in the current market in order for the projects to be completed within the planned completion dates and costs. Now that the FSCP has already been approved by the MSP and that the projects have started (with the exception of the West Island training centre), we are of the opinion that Auditor General of the Ville de Montr?al 401 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit strict monitoring of costs should be done, not only in order to comply with the new estimates that were revised in 2012, but also to avoid jeopardizing the implementation of certain projects. Thus only $7,714,000 was spent by the end of 2011, in the context of the implementation of the projects related to the response component - strike force. There remains, therefore, for 2012 and 2013, a total amount of $21,166,000 to be put towards completion of these projects. Upon approving the FSCP, the MSP issued a certificate of compliance with its directives. This was followed by the adoption of the FSCP by the urban agglomeration council and its implementation in accordance with the proposed implementation plan, which bestowed on the city (under an urban agglomeration power) immunity from future prosecution. By this very fact, the response component - strike force constitutes a key element of the FSCP and the measures initially planned for the latter must be completed. In the event that some projects cannot be completed as planned before the end of 2013, we are of the opinion that the SIM should notify the MSP to obtain authorization of the extension or abandonment. 3.2.1.2.B. Recommendation We recommend that the Service de s?curit? incendie de Montr?al: o Take the necessary steps to comply with the completion dates for construction of fire stations 32 and 59, as well as for the refurbishing of fire station 51 now planned for 2013; o Examine with the Direction g?n?rale and the Service des finances all possible solutions for funding the construction of the West Island training centre; so that the response component - strike force of the implementation plan of the Fire Safety Cover Plan is completed as scheduled. Business unit's response: [TRANSLATION] Contracts were awarded for fire stations 32, 51 and 59. Implement strict administrative management of these contracts to ensure that all contractual parties act with diligence. (Completed, January 2013) Implement specific measures to ensure the delivery of fire station 32 in December 2013 and of fire station 59 in September 2013. (Planned completion: fire station 32, December 2013; fire station 59, September 2013) Implement of strict monitoring mechanisms at the worksites of fire stations 51 and 59 by professional consultants. (Completed, March 2013) Hold regular monthly statutory meetings with the DSTI and the SIM to ensure project follow-up. (Completed, January 2013) Auditor General of the Ville de Montr?al 402 2012 Annual Report 5.9. First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al) Funding needs for the construction of the West Island training centre to be presented to city manager. (Completed, February 2013) 3.2.1.2.C. Recommendation We recommend that the Service de s?curit? incendie de Montr?al obtain the approval of the Minist?re de la S?curit? publique if construction of the West Island training centre cannot be completed as planned in the implementation plan of the Fire Safety Cover Plan, so as to comply with the Fire Safety Act. Business unit's response: [TRANSLATION] Official link to the MSP. Letter explaining the reasons and rationale for the change, if necessary. 2012 report submitted in March 2013 through the official GDD, in the form of documents showing the level of progress. (Completed, March 2013) 3.2.1.2.D. Recommendation We recommend that the Service de s?curit? incendie de Montr?al obtain the approval of the Minist?re de la S?curit? publique for delaying the timeframe for acquiring a heavy vehicle for fire station 59, initially planned for 2013, and delaying the timeframe for completing the work on refurbishing fire station 63, initially planned for 2012, in order to comply with the Fire Safety Act regarding the fire safety cover plan. Business unit's response: [TRANSLATION] Letter to the MSP. (Completed, March 2013) 3.2.1.2.E. Recommendation We recommend that the Service de s?curit? incendie de Montr?al implement stringent cost monitoring in order to complete construction of the infrastructure planned in the fire safety cover plan within the limits of the new cost estimates. Business unit's response: [TRANSLATION] Put in place strict worksite monitoring mechanisms by professional consultants. (Completed, March 2013) Auditor General of the Ville de Montr?al 403 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.2.1.2.F. Recommendation We recommend that the Service de s?curit? incendie de Montr?al take all necessary steps, when including other fire station construction or refurbishment projects undertaken as part of an amendment or revision to the fire safety cover plan, to ensure that these projects are well defined and are based on cost estimates that represent true market value to be able to complete them based on planned completion dates and costs. Business unit's response: [TRANSLATION] Provide for a planning period in order to conduct feasibility studies. A model technical functional program for a fire station and access to building audits will make it possible to complete feasibility studies. Use professional services in construction savings beginning with the project plan up to the end of the construction worksite. (Completed, January 2013) 3.2.1.3. Water Supply Component 3.2.1.3.A. Background and Findings At the time the FSCP was being developed, projects had been defined to solve a problem of water supply for the western part of the island when firefighters were fighting a fire. This involved the construction of underground reservoirs equipped with dry hydrants and the inclusion of water supply points within the construction of new water mains. According to the SIM, these projects should have been completed between 2010 and 2013. However, no feasibility study or cost estimate was done by the SIM for these projects before being included in the FSCP. Following adoption of the FSCP by the MSP and by the urban agglomeration council, the planned projects were submitted to Service de l'eau. Once summary analyses were completed (technical drawings of the reservoirs, analyses of the location and soil condition, and analyses regarding adding pumps so that fire trucks could be supplied with water from the reservoirs), the Service de l'eau mentioned to the SIM that these projects presented technical constraints that required major investments, for example, the city's acquisition of land (under an urban agglomeration power) and the obtaining of public easements for the locations where the underground water reservoirs were to be installed. In 2009, the SIM had to postpone completion of these projects. At the time of our audit, the Service de l'eau was unable to provide cost estimates in support of the postponed projects. The SIM redirected its efforts, therefore, towards a less costly solution, the implementation of 11 reserved fire hydrants to serve the western part of the island. As of October 30, 2012, Auditor General of the Ville de Montr?al 404 2012 Annual Report 5.9. First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al) ten reserved fire hydrants had been installed at a total cost of $38,000. The installation of an eleventh hydrant is currently the subject of litigation with the contractor that was awarded the contract. In our opinion, this constitutes a major amendment of the FSCP regarding the supply of water in the event of a fire, and the SIM must ensure that the procedure set out under the Fire Safety Act is followed. This procedure requires that amendments to the FSCP be approved by the urban agglomeration council and by the MSP. Furthermore, we are of the opinion that the SIM must also ensure that it demonstrate to the Ministre de la S?curit? publique that the installation of reserved hydrants, a solution that replaced the projects initially planned in the FSCP, meets the objectives of supplying water in the event of a fire. 3.2.1.3.B. Recommendation We recommend that the Service de s?curit? incendie de Montr?al: A) Ensure that all projects included in the fire safety cover plan at the time of an amendment or revision of said plan are based on feasibility studies and proper costing in order to justify the choices retained and to predict the sums that will be needed in the three-year capital expenditures program based on the planned completion date; B) Formally submit to the Ministre de la S?curit? publique amendments to the fire safety cover plan regarding the supply of water in the event of a fire, demonstrating that the replacement of projects initially proposed meets the same objectives that enabled the city (under an urban agglomeration power) to obtain a certificate of compliance for the fire safety cover plan so as to comply with the provisions of the Fire Safety Act; C) Ensure that all necessary measures are taken to complete the project involving the supply of water in the event of a fire so that the objectives in this matter, which enabled the city (under an urban agglomeration power) to obtain a certificate of compliance for the fire safety cover plan, are achieved as planned. Business unit's response: A) [TRANSLATION] The implementation of a selection and evaluation process, combined with a cost analysis, will ensure a solution that is attainable. The tools to be used are: o Participation of the Division de la planification strat?gique and of other concerned divisions; o Implementation of a methodology and evaluation grid; and o Completion of feasibility studies, and a simulation exercise, if necessary. (Completed, March 2013) Auditor General of the Ville de Montr?al 405 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit B) Official link to the MSP, letter explaining the reasons and rationale for the change in approach in the area of water supply. (Completed, March 2013) C) Good project management practices, combined with regular scheduled meetings periodically with partners (boroughs, related municipalities, public works, Service de l'eau), will ensure compliance with completion plans. o Completion dates; o Change requests; o Follow-up; o Documentation. Minutes of regularly scheduled meetings. (Completed, March 2013) Implement an action plan to obtain the certificate of compliance for the fire safety cover plan. (Planned completion: December 2013) The fire hydrant on the corner of chemin North Ridge and mont?e de l'?glise in ?le-Bizard was installed on December 6, 2012. Testing was done on December 11 and 12. (Completed, December 2012) 3.2.2. Project Management 3.2.2.A. Background and Findings In addition to the SIM's human resources, completion of the implementation required drawing on the services of internal partners of the city: o the STI for the development of the SGIAP; o the DSTI for the construction and refurbishing of fire stations and the West Island training centre; and o The Service de l'eau for the construction of the water supply infrastructure. In order to meet the planned completion date of the FSCP and to monitor the completion of various projects, SIM management prepared a detailed annual plan for all work based on a schedule that required coordination and follow-up using project management methods and tools. To ensure implementation of the FSCP by the planned completion date, SIM management put in place a coordination structure in 2009, comprised of: o a steering committee; o a project office; o an administrative respondent; o a project head; o a project manager; Auditor General of the Ville de Montr?al 406 2012 Annual Report 5.9. First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al) o o administrative support; a person responsible for each of the programs. The staff assigned to this structure comes exclusively from the SIM. However, other business units of the city are involved in completing the implementation of the FSCP: the STI (development of the SGIAP), the Service de l'eau (water supply) and the DSTI (refurbishing and construction of fire stations). In the case of these business units, we noted the absence of designated representatives from the start in the implementation structure of the FSCP. While SIM management had designated a person from their department for each planned outcome or product to be delivered within each of the components and had put in place working tables, in our opinion these actions were not adequate given the delays in the construction and refurbishment of the fire stations, as well as in the delivery of the SGIAP, added to the fact that the scope of the water supply projects was not the same as at the beginning. In our opinion, operating in this manner is problematic for the coordination and monitoring of the completion of projects that fall under other business units that are assisting the SIM in its role as coordinating manager, such as the DSTI, the STI and the Service de l'eau. This indicates that the project management and monitoring mechanisms had flaws. The absence of representatives from the business units at the time of follow-up meetings with SIM management or with the steering committee is a major one. In our opinion, with regard to the FSCP, the heads of each of the components in the business units should have been integrated into the implementation structure in the same way as SIM managers. Provision should have been made for project dockets setting out the responsibilities, expected outcomes and completion dates of each of the stakeholders (the SIM, the STI, the DSTI, and others). The SIM, as the department in charge of the FSCP, should have reached agreements with its partners at the city. 3.2.2.B. Recommendation We recommend that the Service de s?curit? incendie de Montr?al, where completion of the projects anticipated in the fire safety cover plan requires the involvement of other business units of the city, integrate, as soon as possible, the heads of each of these units into the project coordination mechanisms so as to have complete information, in a timely manner, and to be able to meet planned completion dates. Auditor General of the Ville de Montr?al 407 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Business unit's response: [TRANSLATION] Good project management practices, combined with planning, needs assessment and partner meetings, will ensure that communications with partners are compliant. (Completed, March 2013) 3.2.3. Implementation Costs 3.2.3.A. Background and Findings In terms of the full implementation of the FSCP, as adopted by the urban agglomeration council, according to the budget data presented in Table 9, the total operating cost was $49,127,900, or $10,120,100 less than anticipated in the 2008 estimates ($59,248,000). Table 9 - Discrepancies Between FSCP Implementation Costs and Total Investment as of October 16, 2012 (in thousands of dollars) 2009 2010 2011 2012 a 2013 a Total b Operating budget Response - anticipated 1,452.1 2,912.2 5,849.2 8,352.6 10,519.7 484.0 2,841.0 5,709.0 6,604.6 8,703.5 24,342.1 Variance (real - anticipated) (968.1) (71.2) (140.2) (1,748.0) (1,816.2) (4,743.7) Prevention - anticipated 3,146.5 4,201.5 6,022.2 8,276.3 8,515.7 30,162.2 Prevention - real 2,216.0 2,861.0 4,245.9 7,164.5 8,298.4 24,785.8 Variance (real - anticipated) (930.5) (1,340.5) (1,776.3) (1,111.8) (217.3) (5,376.4) Total - anticipated 4,598.6 7,113.7 11,871.4 16,628.9 19,035.4 59,248.0 Response - real Total - real Variance (real - anticipated) 29,085.8 2,700.0 5,702.0 9,954.9 13,769.1 17,001.9 49,127.9 (1,898.6) (1,411.7) (1,916.5) (2,859.8) (2,033.5) (10,120.1) 4,436 16,730 TCEP (of the SIM) Anticipated in 2008 Revised in 2012 30,370 328 3,182 4,204 West Island training centre (CFO) 28,880 9,850 Total including the CFO 38,730 Variance (total - anticipated) 8,360 Other related costs SGIAP (TCEP of the STI) 4,176 Water supply - reserved fire hydrants (TCEP of the Service de l'eau) 38 Total 4,214 Total investment including related costs a b 42,944 Real costs represent projected costs to December 31 of the year in question. Debt servicing not included (interest to pay down the debt). Auditor General of the Ville de Montr?al 408 2012 Annual Report 5.9. First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al) This reduction in operating expenses is due mainly to delays in the construction and refurbishing of the fire stations, completion of the West Island training centre and delivery of the SGIAP. This is reflected in: o Postponing the firefighter recruitment initially planned for the start of 2009 and of 2012. In fact, of the 20 firefighters to be recruited in 2009, seven firefighters were recruited that year and the other 13 were only recruited in 2011. The SIM planned to recruit 24 firefighters in 2012; however, none were recruited that year ($2,457,100). o The postponement to 2014 of the recruitment of 36 firefighters initially planned for 2013 ($1,286,900). o The downward revision of the total salaries of prevention resources in relation to estimates ($2,492,800). o Revision of contingency costs ($2,005,300), which proved to be lower than expected, mainly because of the postponement of the delivery of fire stations. o Downward revision of the smoke detector and summer brigade program ($730,000). o Carry over of operating costs related to the West Island training centre ($648,000). o Downward revision of the subsidy program put in place by the SIM to facilitate the implementation of standards for seniors' residences ($500,000). Regarding the TCEP, the total cost should reach $38,730,000 by the end of 2013, or $8,360,000 more than the initial estimate of $30,370,000. On this matter, the results of our inquiries regarding the rigour with which the estimates were made at the time of the design of the FSCP and our recommendations regarding the reliability of the cost estimates appear in section 3.2.1.2 of this audit report. Finally, the total cost of implementation presented by the SIM at the time of adoption of the FSCP took into account the operating budget and the TCEP for the years 2009-2013. During our audit, we concluded that the costs of the SGIAP project managed by the STI, as well as the water supply projects managed by the Service de l'eau, had not been included in the total cost of implementation. In our opinion, these costs are directly related to implementation of FSCP. Therefore, $4,176,000 should be added for the development of the SGIAP and $38,000 for the supply of water, for a total discrepancy of $12,574,000. 3.2.3.B. Recommendation We recommend that the Service de s?curit? incendie de Montr?al include in the total cost of implementing the fire safety cover plan the cost of projects stemming from it, which are managed by other business units, so as to reflect the real cost to the agglomeration. Auditor General of the Ville de Montr?al 409 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Business unit's response: [TRANSLATION] Present the total cost of implementing the fire safety cover plan, including the cost of projects related to the plan that are managed by other business units, in order to reflect the real cost of the plan. (Completed, April 2013) 3.2.4. Accountability Reporting 3.2.4.A. Background and Findings In order for the Direction g?n?rale and authorities to be informed about the progress of the FSCP project that was entrusted to SIM management, accountability mechanisms must be put in place. Management reports should be produced periodically and contain relevant information to enable the above to fully understand the degree of completion of the implementation plan. These reports should enable decision to be made in a timely manner. Our audit looked into the mechanisms that were put in place in order to account for the progress of the implementation plan for the FSCP. In order to meet requirements of the Fire Safety Act 7 regarding the FSCP, the SIM has produced two activity reports to date. The 2009-2010 activity report was tabled with the urban agglomeration council in March 2011, while the 2009-2011 activity report was tabled with the urban agglomeration council in March 2012. According to the documents we examined, these reports were sent to the Ministre de la S?curit? publique. They describe all the actions set out in the implementation plan of the FSCP based on their likely year of completion (2009-2013). Moreover, the reports indicate the degree of completion for each action. A brief description is provided for the actions that were completed. Summary explanations are given for those actions that were completed ahead of schedule or whose completion was postponed to a later year. In conclusion, SIM management produces activity reports for the Ministre de la S?curit? publique to comply with the Fire Safety Act, which provide information on the degree of completion of the work planned in the FSCP. However, upon reading the latest report produced, i.e. the 2009-2011 activity report, we were not able to retrace information 7 Section 35: "Every local or regional authority and every intermunicipal board in charge of the implementation of measures provided for in a fire safety cover plan must, within three months after the end of their fiscal year, adopt, by resolution, and transmit to the Minister a report of their fire safety activities for the preceding fiscal year and their fire safety projects for the coming year to their council." Auditor General of the Ville de Montr?al 410 2012 Annual Report 5.9. First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al) indicating that the complete delivery of the SGIAP will be delayed until 2014. We are of the opinion that the SIM should have informed the Ministre de la S?curit? publique of the postponement of the planned completion date of the SCIAP project since it involved information already known at the time of producing this report. While SIM management sent the activity reports to the MSP, we are of the opinion that a formal accountability report on the progress of the implementation of the FSCP, which included more management information, should also have been completed for the Direction g?n?rale and authorities. In fact, for a project of the scope of that of the FSCP, whose implementation is spread over a period of five years, we are of the opinion that it is important that the Direction g?n?rale and authorities know the planned and real costs of the implementation plan, as well as its degree of progress. Furthermore, they should be informed periodically of problems encountered that could compromise the full implementation for the end of 2013, as approved by the urban agglomeration council and the MSP. 3.2.4.B. Recommendation We recommend that the Service de s?curit? incendie de Montr?al produce periodically for the Direction g?n?rale and authorities accountability reports detailing the degree of progress, problems encountered and total costs of the plan to implement the fire safety cover plan, so that appropriate decisions may be made to comply with the planned completion date in 2013 or advise them of the consequences in the event that these completion dates are not met. Business unit's response: [TRANSLATION] The SIM agrees to produce an accountability report consisting of the three elements requested on a quarterly basis for the Direction g?n?rale and the authorities involved. (Planned completion: December 2013) Conclusion Once the FSCP was approved by the Ministre de la S?curit? publique, it enabled the city (under an urban agglomeration power) to benefit from immunity from prosecution under section 47 of the Fire Safety Act, which states the following in the second paragraph: The exemption applies to the authority having established the service or having requested the person's response or assistance, except if the authority has failed to adopt a plan for the implementation of the fire safety cover plan as required or if the Auditor General of the Ville de Montr?al 411 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit measures or procedures provided for in the applicable implementation plan and relating to the acts in question were not implemented as established. As we presented earlier, our audit led us to conclude that several measures planned in the FSCP that was approved by the Ministre de la S?curit? publique, as well as for the prevention and response components, remain to be completed before the end of 2013, the final year of the implementation plan. It is true that the completion or accomplishment of some of them, in particular the refurbishment of fire station 32 and the acquisition of light and heavy vehicles, was already planned for 2013. However, other measures, such as the construction of fire station 59 and the refurbishment of fire station 51, were not completed in accordance with the initial completion schedule, in 2010, 2011 and 2012, and, therefore, needed to be postponed to 2013. Moreover, the complete delivery of SGIAP is now planned for 2014, beyond the five-year period allotted by the Ministre de la S?curit? publique to implement the FSCP. Although planned for the FSCP, the delivery of the West Island training centre appears to be in jeopardy since it has not been officially postponed to a later year. Consequently, in order to avoid future legal action that could raise questions about the exemption from responsibility (immunity) from which the city benefits (under an urban agglomeration power), we are of the opinion that all the necessary efforts must be deployed to complete the implementation of all the measures set out in the FSCP, between now and the end of 2013. In the event that the city (under an urban agglomeration power) is unable to meet the completion date, we are of the opinion that section 30.1, paragraph 1, of the Fire Safety Act would apply: "As an exceptional measure, following a request with reasons from a regional authority, the Minister may authorize the amendment of a fire safety cover plan in force in order to extend one or more deadlines contained in the plan." Moreover, a request to postpone the completion date should be sent formally to the Ministre de la S?curit? publique in the case of the SGIAP project to complete the refurbishing fire station 63 and the delivery of a heavy vehicle that was planned for fire station 59. The SIM should also notify the Ministre de la S?curit? publique of the fact that the West Island training centre will not be completed as planned in the FSCP. Auditor General of the Ville de Montr?al 412 2012 Annual Report 5.9. First Responder Service and Fire Safety Cover Plan (Service de s?curit? incendie de Montr?al) 4. Appendix 4.1. Implementation of the First Responder Service - Detailed Cost Estimates, 2007-2009 Table A - Detailed Initial Cost Estimates for the Implementation and Start-Up Period of the FR Service 2007 2008 2009 Total Expenditures Implementation costs No. of fire stations to be integrated Communication Certification of firefighters Equipment Administration 19 18 $300,000 $50,000 $2,522,300 $2,047,300 $710,000 $664,000 $758,400 $865,000 21 $50,000 $2,923,100 $765,500 $698,200 $4,290,700 $3,626,300 $4,436,800 $12,353,800 No. of active fire stations - start of year No. of active fire stations - end of year Salary bonuses Ongoing training Equipment Administration 8 27 45 $1,118,900 $2,991,000 $109,200 $470,200 $254,700 $954,400 $1,018,300 $1,065,800 66 $4,562,300 $721,900 $1,662,700 $1,442,200 Total $2,501,100 $5,481,400 $8,389,100 $16,371,600 Total Operating expenses Total expenditures Based on the memorandum of understanding Acquisition of vehicles a 58 $400,000 $7,492,700 $2,139,500 $2,321,600 $8,672,200 $1,301,300 $2,871,800 $3,526,300 $6,791,800 $9,107,700 $12,825,900 $28,725,400 Revenuesa $7,207,500 Investments $5,500,000 $5,500,000 $18,207,500 $2,210,000 $2,000,000 $2,000,000 $6,210,000 Amounts that will be indexed annually based on the rate set by the Government of Qu?bec. Source: SIM. Auditor General of the Ville de Montr?al 413 2012 Annual Report Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 Subcontracted Construction Work 5.10 5.10. Subcontracted Construction Work Table of Contents 1. Introduction ................................................................................................................... 419 2. Audit Scope................................................................................................................... 421 3. Findings and Recommendations .................................................................................. 421 3.1. Clauses Contained in the Books of Specifications and in the Call for Tenders Documents.............................................................................................. 425 3.2. Monitoring the Application of the Clauses Related to the Information on Subcontractors and Verifications Carried Out on the Compliance of Subcontractors ...................................................................................................... 433 4. Appendix ....................................................................................................................... 448 4.1. Summary of the Work Contracts Examined .......................................................... 448 Auditor General of the Ville de Montr?al 417 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit List of Acronyms AMF Autorit? des march?s financiers CTA Cities and Towns Act DEP Direction de l'eau potable DI Direction des infrastructures DSTI Direction des strat?gies et transactions immobili?res IPCA MAMROT Minist?re des Affaires municipales, des R?gions et de l'Occupation du territoire Integrity in Public Contracts Act RBQ RENA Auditor General of the Ville de Montr?al register of enterprises ineligible for public contracts SCARM 418 R?gie du b?timent du Qu?bec Service de la concentration des arrondissements et des ressources mat?rielles 2012 Annual Report 5.10. Subcontracted Construction Work 5.10. Subcontracted Construction Work 1. Introduction Ville de Montr?al (the city) enters into contracts with contractors to carry out the construction work set out in the three-year capital expenditures program. Under the Cities and Towns Act 1 (CTA), business units must issue a public call for tenders when the contract involves an anticipated expenditure of $100,000 or more. For the years 2011 and 2012, the monetary value of the work contracts awarded by authorities amounted to approximately $500 million. The major steps in the public call for tenders process include the preparation of call for tenders documents, market solicitation, analysis of the bids tendered, awarding of the contract and management of the contract. In the case of work to be executed, it is the central departments and boroughs that see to each of these major steps, depending on the jurisdiction within which the work is to be done or the mandate entrusted to them (e.g., the Direction des infrastructures [DI]). Work contracts are awarded to the lowest qualified tenderer, and the bid price is the determining factor. It may happen that a contractor who is the lowest qualified tenderer entrusts part of the work to subcontractors. The city, as the client, is legally bound to the contractor to whom it has awarded the contract. In the case of the subcontractor, the legal bond is between the contractor and the subcontractor. However, for the purposes of sound management and to ensure that subcontractors on work sites are qualified, the city has a duty to know their identity and the nature of the work that they will be doing. Since 2010, the provincial government has also brought several amendments to the laws in an effort to prevent, combat and penalize certain fraudulent practices in the construction industry and to fight against collusion. A licence issued to a contractor under the Building Act 2 may contain a restriction from obtaining a public contract if the holder has been found guilty of certain criminal or tax offenses. Under these laws, the city must obtain assurances that the contractors that will be doing the work within the scope of the contracts awarded to them are compliant. While this legal 1 2 RSQ, chapter C-19. RSQ, chapter B-1.1. Auditor General of the Ville de Montr?al 419 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit obligation does not apply to subcontractors, the city is responsible, nonetheless, for ensuring that they also are compliant. In order to have such assurances, the city must receive relevant information or check with the R?gie du b?timent du Qu?bec (RBQ), the Secr?tariat du Conseil du tr?sor and Revenu Qu?bec. In the case of contractors that are awarded city contracts, this compliance is assessed at the time bids are analyzed. In the case of subcontractors, while the city is not legally bound to them, the tender documents should contain clauses that require the contractors to provide the necessary information about the subcontractors throughout the period of the contract. Since December 7, 2012, other amendments to the laws have come into effect. The Integrity in Public Contracts Act 3 (IPCA), which amended the Act respecting contracting by public bodies 4 , states that any enterprise that wishes to enter into a contract with a public body must obtain pre-authorization from the Autorit? des march?s financiers (AMF). It should be pointed out that the AMF works in close collaboration with the Unit? permanente anticorruption (UPAC). The authorization also applies to any subcontract that is directly or indirectly related to the contract concerned and that involves expenditures equal to or greater than the amount determined by the government. The latter estimates that it will take three years to draw up the register of enterprises authorized to enter into a public contract. As a first step, beginning on January 15, 2013, authorization to contract is required for any new construction contract and subcontract valued at or above $40 million, as well as for contracts defined in the decrees adopted by the government. 5 It should be noted that the government can, before March 31, 2016, amend the amount of $40 million set by the IPCA and adopt new decrees governing other contracts that will require authorization from the AMF. In the interim, contracts and subcontracts that are not currently regulated by the IPCA or by decree should continue to be checked with the RBQ and the register of enterprises ineligible for public contracts (RENA) prior to the awarding of the contract in order to ensure that the tenderer can enter into a contract with a public body. Moreover, a certificate of compliance from Revenu Qu?bec is also required for all contracts and subcontracts unless the contractor has been authorized by the AMF. 3 4 5 SQ, 2012, chapter 25. RSQ, chapter C-65.1. Decree 1226-2012 of December 19, 2012, and Decree 96-2013 of February 13, 2013, for certain contracts of Ville de Montr?al. Each of these decrees focuses on 25 contracts resulting from calls for tenders for which the two lowest qualified tenderers must obtain authorization from the AMF. In the case of these two first decrees, subcontractors are not required to obtain an authorization from the AMF. Auditor General of the Ville de Montr?al 420 2012 Annual Report 5.10. Subcontracted Construction Work 2. Audit Scope The purpose of the audit was to evaluate the extent to which the city: o Assures itself that the work entrusted by the contractor to subcontractors complies with the call for tenders documents and books of specifications; o Implements the necessary audit procedures to obtain relevant information for decision making. More specifically, our audit focused on the work contracts that followed a public call for tenders. We selected a sample of 13 contracts that were awarded during 2012, ranging across three directorates from central departments and four boroughs that were planning construction work. In the case of the central departments, our audit concentrated on the Direction de l'eau potable (DEP) of the Service de l'eau, the DI of the Service des infrastructures, du transport et de l'environnement, and the Direction des strat?gies et transactions immobili?res (DSTI) of the Service de la concertation des arrondissements et des ressources mat?rielles (SCARM). In the case of the four other business units selected, our audit focused on Ahuntsic-Cartierville, Mercier-Hochelaga-Maisonneuve, PierrefondsRoxboro and Verdun boroughs. In order to corroborate the legal aspects of the contracts, we enlisted the assistance of representatives of the Division du droit contractuel, which reports to the Direction des affaires civiles of the Service des affaires juridiques et de l'?valuation fonci?re. 3. Findings and Recommendations As mentioned at the start, no contractual bond exists between the city and subcontractors. Since 2010, however, the adoption by the provincial government of bills aimed at preventing, combating and penalizing certain fraudulent practices in the construction industry, in an effort to fight collusion, reinforces the fact that the city must know the identity of subcontractors that perform work on behalf of its contractors. We will briefly look at how provincial acts and regulations have an impact on the city's practices regarding subcontractors. Contract Management Policy To begin, in 2010, the CTA required municipalities to have a contract management policy that applied to all contracts as of January 1, 2011. In the case of the city, this policy was adopted by city council in December 2010. Its scope covers, inter alia, the stakeholders Auditor General of the Ville de Montr?al 421 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit involved in all the steps leading to entering into a contract, including work contracts, from the time the contract is awarded through its management. The term "stakeholder" includes tenderers, contracting authorities, suppliers, co-contractors of the city, consultants but, also, subcontractors. In this policy, the city specifically reiterates its commitment to preventing any situation, such as influence peddling, intimidation, corruption, collusion or conflicts of interest, which is likely to besmirch the integrity or hinder the efficacy of the procurement process and the awarding of contracts. To do this, the city adopted requirements for the conduct expected in all of its contract dealings, regardless of the value of the contract, in order to ensure compliance and to prevent and avoid collusion, potential sources of fraud or malfeasance. If we concentrate on the scope of the policy as it concerns subcontractors, this specifically means that, in tendering a bid, the signatory solemnly affirms that, to the best of his personal knowledge and upon careful verification, there has been no collusion in preparing the bids, and that the tenderer has not been found guilty of collusion, corruption, malfeasance or bribery in the five years preceding the call for tenders. Any misleading solemn declaration can result in the rejection of a bid. The city may also apply sanctions if, during the course of the contract, it is discovered that the solemn declaration was false. For this reason, we believe that the city must, at the very minimum, know the subcontractors in order to properly monitor the application of the policy. Register of Enterprises Ineligible for Public Contracts The Anti-Corruption Act 6 (given assent in June 2011) amended the Act respecting contracting by public bodies to allow for the creation of the RENA. It also amended municipal legislation, including the CTA, so that provisions regarding the RENA would be applicable to municipal contracts. More specifically, the contracts affected are those that are required to carry out work or provide insurance, equipment, materials or services. The provisions regarding the RENA and those amending municipal legislation accordingly came into effect on June 1, 2012. From the moment a conviction is recorded in the RENA, a contractor or an associate of the contractor as defined in the Act who is convicted, by a final judgment, of any offence determined by government regulation, is ineligible for public contracts. 6 RSQ, chapter L-6.1. Auditor General of the Ville de Montr?al 422 2012 Annual Report 5.10. Subcontracted Construction Work In its Muni-Express bulletin of May 2012, the Minist?re des Affaires municipales, des R?gions et de l'Occupation du territoire (MAMROT) specifies that it is the responsibility of the municipal body to ensure, prior to entering into a contract, that each tenderer or successful bidder is not recorded in the RENA or, if so, that the period of ineligibility for public contracts has ended. The following information, which appears in the RENA, has been accessible since June 1, 2012: o Name of the enterprise; o Address of the principal establishment; o Qu?bec business number, if applicable; o Offences for which the enterprise has been convicted, or those for which a guilty verdict involving an associate has led to it being entered in the register; o Date on which its ineligibility for public contracts will end; o All other information prescribed under government regulation. Again, according to the same MAMROT bulletin, the contractor who enters into a contract with a municipal body must forward to the body, prior to the start of work, a list showing the following information for each subcontract, where applicable: o Name and address of the principal establishment of the subcontractor; o Amount and date of the subcontract. Moreover, this list must be updated and sent to the municipal body, where applicable. The contractor who enters into a subcontract with an ineligible contractor commits an offence and is liable to a fine. As well, if this contractor has already been convicted of a similar offence in the two years preceding a second conviction, the contractor becomes ineligible for public contracts for a period of two years from the recording of the conviction in the RENA. These legal provisions carry with them the obligation on the part of the city to obtain information about subcontractors and the responsibility to perform the necessary checks. Certificate of Compliance from Revenu Qu?bec The Regulation respecting construction contracts of municipal bodies 7 (adopted in August 2011) sets out new rules regarding the obligation to request a Revenu Qu?bec certificate of 7 Gazette officielle du Qu?bec. Part 2, Vol. 143, No. 35, August 31, 2011, pp. 3899-3901. Auditor General of the Ville de Montr?al 423 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit compliance from construction contractors and their subcontractors. This regulation applies to calls for tenders issued and to contracts awarded since January 1, 2012. The Regulation requires that all contractors seeking to enter into a construction contract involving an expenditure of $25,000 or more with a municipal body must obtain a certificate of compliance from Revenu Qu?bec. This certification is granted to all contractors who have filed the returns and reports required under Qu?bec tax laws and who do not have any overdue accounts with Revenu Qu?bec. To be valid, the contractor's certification must not have been issued more than 90 days prior to the date and time set for receiving the bids or after such date and time or, in the case of mutual agreement contracts, more than 90 days before the contract award date. The same certification requirement also applies to subcontractors who wish to enter directly into a contract with a contractor who has secured a construction contract. To be valid, the subcontractor's certification must not have been issued more than 90 days before the date set for receiving bids for the contractor's contract nor after the subcontract award date or, in the case of mutual agreement contracts, more than 90 days prior to the subcontract award date. The contractor who has been awarded a construction contract by a municipal body must, prior to the start of work, forward to the body a list showing the following information for each subcontract: o Name and address of the subcontractor; o Amount and date of the subcontract; o Number and date of issue of the subcontractor's certificate of compliance from Revenu Qu?bec. These legal provisions carry with them the obligation on the part of the city to obtain information about subcontractors and proof that subcontractors hold a valid certificate of compliance from Revenu Qu?bec. Authorization of the Autorit? des march?s financiers Since December 7, 2012, the IPCA stipulates that any contractor who wishes to be a party to a public contract must have the prior authorization of the AMF. In a first phase, as of January 15, 2013, an authorization is required for any new construction or services contract or subcontract equal to or greater than $40 million, as well as for contracts defined in the decrees adopted by the provincial government on December 19, 2012 (Decree 1226-2012), and on February 13, 2013 (Decree 96-2013). It should be noted that the government has Auditor General of the Ville de Montr?al 424 2012 Annual Report 5.10. Subcontracted Construction Work until March 31, 2016, to amend the amount of $40 million set by the IPCA and to adopt new decrees aimed at other contracts for which AMF authorization will be required. Accordingly, in the case of contracts that are not regulated by the IPCA or by decree, the RENA and the list of licences that are restricted by the RBQ must still be consulted to obtain assurance, prior to the awarding of a contract, that the tenderer is eligible to enter into a contract with a public body. Moreover, a certificate of compliance from Revenu Qu?bec is also still required for all contracts or subcontracts, unless the contractor has been authorized by the AMF. Against the backdrop of legal provisions that have been strengthened to prevent, combat and penalize certain fraudulent practices in the construction industry and fight against collusion, our audit examined the extent to which the city has information about subcontractors that were performing work within various contracts. We also looked into the mechanisms in place to ensure that these subcontractors were compliant at the time that the contract was awarded and during the contract. In order to substantiate our findings, we selected a sample of 13 contracts (see Appendix 4.1) from among those awarded in 2012, including seven valued at over $7.5 million each and five contracts that involved a wide diversity of work for which subcontractors were more likely to be involved. Our sample consisted, therefore, of work contracts awarded by authorities following a public call for tenders. The amount of the contracts that we examined ranged from $0.6 million to $16.4 million. In the first phase, we examined the clauses contained in the books of specifications and in the tender documents, which enabled the city to receive information about subcontractors. In a second phase, we evaluated how the application of these clauses was followed up, as well as the mechanisms that were put in place to ensure that subcontractors were compliant. 3.1. Clauses Contained in the Books of Specifications and in the Call for Tenders Documents 3.1.A. Background and Findings A book of specifications is a contractual document that describes what the client expects from the contractor. The document outlines in the most precise way possible and in simple terms the needs, requirements and obligations that the contractor must meet. Books of specifications include general and specific documents. Auditor General of the Ville de Montr?al 425 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit The general documents consist of the general administrative clauses document and the general technical clauses document. The first establishes the administrative provisions that apply to a category of goods and services (for example, the work to be done). The second establishes the technical provisions that apply to all deliverables of the same kind (for example, road and sidewalk paving). The specific documents are comprised of specific administrative and technical specification clauses. In the first case, they establish the administrative provisions of each contract. In the second case, they establish the technical provisions that are required to complete the work of each contract. The call for tenders documents are made up of bid forms, plans and specifications, addenda and instructions to tenderers that establish all the conditions required to tender a bid. When the contract is awarded, the call for tenders documents, books of specifications and retained bid together form the contract. During our audit, we identified, for each of the contracts selected, the clauses in the books of specifications and in the call for tenders documents that enabled the city to request information about the subcontractors. To begin, in the case of the seven business units that we audited, we found that the general (administrative and technical) clauses had been prepared on different dates. For example, Pierrefonds-Roxboro borough awarded a contract that refers to general administrative clauses dating from 1997, while the DI awarded a contract that refers to general clauses dating back to March 2009. With this in mind, we identified the general subcontracting clauses in each of the contracts we examined to evaluate the extent to which they enabled the city to request information about subcontractors. These clauses were found either in the general administrative clauses or in the instructions to tenderers. A comparison of the clauses revealed items that were common to all the business units. The books of specifications and the call for tenders documents of all the clients (the DEP, the DI, the DSTI, and Ahuntsic-Cartierville, Mercier-Hochelaga-Maisonneuve, PierrefondsRoxboro and Verdun boroughs) stipulated that, when a contractor wishes to subcontract work: o The contractor must provide the manager with a complete and detailed list of subcontractors; o The list must include a description of the work to be done; Auditor General of the Ville de Montr?al 426 2012 Annual Report 5.10. Subcontracted Construction Work o o o A copy of the licences issued by the RBQ must accompany the list of subcontractors, as well as any amendment to this list; The contractor must provide, at the manager's request, all additional information about these subcontractors; The contractor must, prior to the start of work, provide a list indicating the following for each construction subcontract valued at more than $25,000: - the name and address of the subcontractor, - the amount and award date of the subcontract, - the number and date of issue of the subcontractor's certificate of compliance from Revenu Qu?bec, - during the contract, an amended list, where applicable, when a new subcontract is awarded and before the work entrusted to this subcontractor begins. A comparison of the clauses also revealed differences between the business units as to: o The time of receipt of the list of subcontractors: - a clause in the administrative documents of Pierrefonds-Roxboro borough provides for the list of subcontractors to be sent within 15 days of the awarding of the contract, - the general administrative clauses of the other business units (the DEP, the DI, the DSTI, and Ahuntsic-Cartierville and Mercier-Hochelaga-Maisonneuve boroughs) provide for receipt of the list as soon as the contract is awarded, - the clause of Verdun borough is silent in this regard; o The approval of the list of subcontractors: - a clause in the administrative documents of Pierrefonds-Roxboro borough provides for the client's approval, - the general administrative clauses of the other business units are silent in this regard; o The approval of amendments made to the list of subcontractors: - a clause in the administrative documents of Pierrefonds-Roxboro borough specifies that the contractor must have all amendments to this list approved by the borough, - an administrative clause for Verdun borough specifies that the written permission of the engineer must be obtained for any addition or amendment to the list after the contract is awarded, - the clauses of the other business units (the DEP, the DI, the DSTI, and AhuntsicCartierville and Mercier-Hochelaga-Maisonneuve boroughs) specify that the contractor must submit to the manager all amendments made to the list after the contract is awarded; Auditor General of the Ville de Montr?al 427 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit o The transmission, in a timely manner, of a copy of the renewal of all licenses that will expire before the end of the contract: - a clause makes provisions to this effect for the DEP, the DI, and the DSTI, as well as Ahuntsic-Cartierville and Mercier-Hochelaga-Maisonneuve boroughs, - the other business units (Pierrefonds-Roxboro and Verdun boroughs) do not address this issue. This comparison led us to the conclusion that there is no consistency regarding all the items in the administrative clauses on subcontracting. When tendering a bid, contractors need to familiarize themselves with clauses that vary from one business unit to the other. In the case of the list of subcontractors, in particular, clauses differ as to deadlines for transmission, approval and amendments made. In addition, when an item is not specified in a clause, it becomes subject to interpretation by the contractors and those responsible for its application. This kind of situation can make it more difficult to obtain the required information. Finally, as mentioned earlier, the general administrative clauses in the books of specifications provide for the receipt of a list of subcontractors to be accompanied by copies of licences from the RBQ. We also found that, in the wake of new legal provisions that made certificates of compliance mandatory as of January 1, 2012, the Service des affaires juridiques et de l'?valuation fonci?re informed the business units, in November 2011, that all call for tenders documents must now include the new requirements. During our audit, we effectively noted that all the business units had integrated the new requirements either into the instructions to tenderers or into the general administrative clauses. Thus, a clause in the call for tenders documents provides for the transmission of a list of subcontractors with the information required by law (name and address of the subcontractor, date and amount of the subcontract, and certification number from Revenu Qu?bec). In our opinion, this is an example of where the books of specifications and the call for tenders documents include two clauses that ask contractors to supply a list of subcontractors, but with information that achieves different objectives. We are of the opinion that only one list of subcontractors should be required, showing all the necessary information. During our audit, we were informed that the SCARM had begun thinking about standardizing the books of specifications for the execution of work and that the file has been entrusted to the Direction de l'approvisionnement. Based on the information we obtained, the mandate needs to be clarified. Although the scope of the project to standardize the clauses of the various books of specifications is much broader than subcontracting in work contracts, we believe that our findings can be taken into account within the framework of this project. Auditor General of the Ville de Montr?al 428 2012 Annual Report 5.10. Subcontracted Construction Work While it would be preferable if the clauses of books of specifications and of call for tenders documents were standardized, the fact of the matter is that contracts must continue to be awarded and managed on the basis of the clauses that are currently in effect. In the case of the subcontracting clauses, business units must obtain the required information to ensure that subcontractors are compliant. During our audit, we noted that, in addition to these clauses, call for tenders documents generally include a reminder to tenderers about the documents that they must provide. The form that this reminder takes varies from one business unit to another: o In the case of Pierrefonds-Roxboro borough, the tenderer must declare that he has familiarized himself with the list of documents, including the list of subcontractors and materials, that must be provided within 15 days of the contract award date. o In the case of Verdun borough, the tenderer must ensure that his bid submission form meets all the requirements of the books of specifications with respect to 10 items, including those involving the list of subcontractors and copies of the appropriate licenses issued by the RBQ. Mention is made that this verification avoids the bid being rejected for reasons of irregularities. o For some business units, the instructions to tenderers require them to check off and sign a list entitled [TRANSLATION] "Documents and information that must be submitted with the bid": - In the case of the DEP, it clearly states the list of subcontractors. The same is true for Mercier-Hochelaga-Maisonneuve borough, with the exception of the signature on this list; - In the case of the DI and of Ahuntsic-Cartierville borough, this list includes an item worded as follows: [TRANSLATION] "Other documents that may be required in the bid," without referring directly to the list of subcontractors. o In the case of the DSTI, a reminder of certain requirements of the books of specifications is included in the "instructions to tenderers," but does not specifically refer to the information to be provided about subcontractors. It should be pointed out that, for this business unit, it is the only place where the clause about the obligation to provide the certificate of compliance is mentioned. In all the contracts we examined, with the exception of those of the DSTI, we found that the reminder of the documents to be provided did not mention the list of subcontracts above $25,000 and the relevant information required, such as the date and amount of the subcontracts awarded, as well as the date and number of the certificate of compliance. In our opinion, this list should have been included in the reminder of the documents to be provided. Auditor General of the Ville de Montr?al 429 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Furthermore, the reminder about the information and documents that are expected helps emphasize the importance of the requirements of the books of specifications and encourages receipt of the information requested. This is particularly true when there is a deadline to produce the information requested or a cautionary note is added to this reminder to the tenderer, should the information be omitted. 3.1.B. Recommendation We recommend that the Direction g?n?rale clarify the mandate to standardize the books of specifications related to the execution of work and ensure that it makes provisions for the standardization of clauses dealing with the receipt of a list of subcontractors and of required information so that the same requirements apply to all business units. Business unit's response: [TRANSLATION] In late 2012, the city manager and senior department managers met to put in place the means to complete a review and harmonization of the specifications. Following this meeting, the actions below were taken: o Each senior manager named a construction site superintendent for his department. These individuals are responsible for overseeing the review process in their respective departments, supporting working groups and taking all other measures to help advance the work, while ensuring good interdepartmental coordination. These include: - for the Service de la qualit? de vie: Superintendent 1, - for the Service des infrastructures, du transport et de l'environnement: Superintendent 2, - for the Service de l'eau: Superintendent 3; o The required specifications for work on the street right-of-way have been prioritized; o The objective is to finalize a block of updated specifications that will be used for calls for tenders beginning in January; o The general coordinating officer is preparing a work plan for the end of April 2013; o We are evaluating the addition of interns to support the work of the teams; o The required lists will be added to the work plan and integrated starting in January 2014. (Planned completion: January 2014) Auditor General of the Ville de Montr?al 430 2012 Annual Report 5.10. Subcontracted Construction Work 3.1.C. Recommendation We recommend that the Direction de l'eau potable, the Direction des infrastructures, and the Direction des strat?gies et transactions immobili?res, as well as AhuntsicCartierville, Mercier-Hochelaga-Maisonneuve, Pierrefonds-Roxboro and Verdun boroughs, include the following in the reminder of documents to be received with the bid, following a call for tenders for the execution of work: o The obligation of the tenderer to provide a list of subcontracts and all the information required to comply with the requirements of the books of specifications and call for tenders documents; o A deadline or cautionary note to the tenderer should the documents and information requested not be sent, as a way of emphasizing the importance of the city obtaining them. Business units' responses: SERVICE DE L'EAU - DIRECTION DE L'EAU POTABLE [TRANSLATION] The DEP will add a section in the instructions to remind tenderers of their obligation to provide known information about subcontractors when submitting their bid. (Planned completion: June 2013) The DEP will amend the format of the document entitled "List of subcontractors" that is in the bid form, so that it includes more detailed information about subcontractors. (Planned completion: September 2013) In the DEP's call for tenders contract documents, there is no applicable deadline for the receipt of documents. SERVICE DES INFRASTRUCTURES, DU TRANSPORT ET DE L'ENVIRONNEMENT - DIRECTION DES INFRASTRUCTURES [TRANSLATION] The clause in the specifications that provides for this obligation will be revised as part of the harmonization of the specifications documents currently under way. (Planned completion: October 2013) SERVICE DE LA CONCERTATION DES ARRONDISSEMENTS ET DES RESSOURCES MAT?RIELLES - DIRECTION DES STRAT?GIES ET TRANSACTIONS IMMOBILI?RES [TRANSLATION] A reminder of the documents to be provided by the successful contractor is made at the work site start-up meeting. To reinforce this, a document in the form of a checklist will be added to the bank of standard documents used by contract managers in order to improve follow-up of the various information and documents to be produced by the contractor. (Planned completion: June 2013) Auditor General of the Ville de Montr?al 431 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit AHUNTSIC-CARTIERVILLE BOROUGH [TRANSLATION] At the start-up meeting, the borough will require the contractor to provide the list of all subcontractors along with the required supporting documents. A paragraph will be added to the contract award letter as a reminder to the contractor of the importance of his obligations regarding subcontractors. The start work order could be conditional on receipt of all the documents pertaining to the contractor's subcontractors. (Planned completion: May 2013) MERCIER-HOCHELAGA-MAISONNEUVE BOROUGH [TRANSLATION] We have already amended our specifications with the following clauses 3.3.3.1 and 3.3.3.6: [TRANSLATION] 3.3.3 Subcontracting Any contractor who has work done by subcontractors must: 3.3.3.1. Submit to the manager, upon the awarding of the contract, a complete and detailed list specifying the name of the known subcontractors and the work that they are required to do, and advise the manager of any amendments and/or additions made to this list after the contract is awarded and during the contract. He must attach to this list and to any notice of amendment and/or addition to the list, where applicable, a photocopy of the licence issued by the R?gie du b?timent du Qu?bec for each subcontractor according to the nature of the contract, the certificate of compliance from Revenu Qu?bec and, where applicable, the authorization of the Autorit? des march?s financiers (AMF). He must also provide, in a timely manner, a copy of the renewal of any licence that will expire before the end of the contract. 3.3.3.6. Changes to this list do not amend the contract, create a contractual bond between the city and the subcontractors, nor release the contractor from the obligations arising out of the contract. We added clause 3.3.3.8 to these clauses, which reads as follows: [TRANSLATION] 3.3.3.8. In failing to provide all the information requested by the city (RBQ, Revenu Qu?bec and AMF) on all subcontractors, the contractor exposes himself to the withholding of payment for each of the subcontracted specialties. Payment will be made once the city receives all the information requested. (Completed, April 2013) Auditor General of the Ville de Montr?al 432 2012 Annual Report 5.10. Subcontracted Construction Work PIERREFONDS-ROXBORO BOROUGH [TRANSLATION] An article will be drafted and added to the instructions to tenderers, specifying the list of documents that must be provided with the bid, including the list of subcontractors. This same article will emphasize the importance of obtaining the documents requested. (Completed, April 2013) VERDUN BOROUGH [TRANSLATION] Review the clauses in the books of specifications (see book A, articles 4.7 and 4.8) in order to integrate a reminder about the documents to be received with the bid. (Planned completion: May 2013) 3.2. Monitoring the Application of the Clauses Related to the Information on Subcontractors and Verifications Carried Out on the Compliance of Subcontractors The clauses in the books of specifications and the call for tenders documents correspond to the needs, requirements and obligations with which contractors must comply when preparing their bids and executing the work. The business units responsible must follow up to ensure that these clauses are being respected and that the information obtained is compliant. Measures must be taken in the event of non-compliance with the clauses. For each of the contracts selected, we evaluated more specifically how the application of the subcontracting clauses in the books of specifications and in the call for tenders documents was being monitored. We began by examining the moment at which the list of subcontractors was received by the business units. Secondly, we verified the contents of this list, along with the documents supplied, and evaluated the extent to which there were mechanisms in place to ensure that subcontractors were compliant. Thirdly, we examined the extent to which contractors communicated additions and amendments to subcontractors. Finally, we examined measures that were taken to apply the clauses provided. 3.2.1. Receipt of the List of Subcontractors 3.2.1.A. Background and Findings Once the bids have been analyzed, the business units recommend to authorities that a contract be awarded to the lowest qualified tenderer. In the weeks following the awarding of the contract, a work site start-up meeting is held. Representatives from the business unit, the contractor, the construction site superintendent and other stakeholders, where applicable, are present. The purpose of this meeting, inter alia, is to ensure implementation of the project, to review the contractual obligations, to validate the roles and responsibilities Auditor General of the Ville de Montr?al 433 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit of the stakeholders, and to remind everyone of the importance of complying with timelines and costs. Following the meeting, the business unit must issue, by registered letter to the contractor, the order to begin work. During our audit, we assured ourselves that a list of subcontractors had been received by the business units and, if so, when it was received. To begin, a list of subcontractors was indeed received for all the contracts that we examined, with the exception of one for the DI. However, while the administrative clauses in the books of specifications and the call for tenders documents required receipt of the list of subcontractors when the contract was awarded or within 15 days of its being awarded (Pierrefonds-Roxboro borough), we concluded that this was not interpreted the same way by everyone. Overall, our audit showed that the list of subcontractors was not always obtained (1/13) 8 , that there was not always proof of the date of receipt (2/13) 9 , that the list was received at the start-up meeting in one case (1/13) 10 and after the start-up meeting in nearly half of the cases (6/13) 11 . In other words, in only three 12 out of the 13 cases we examined was the list of subcontractors received at the time that the bid was submitted. In the opinion of some of the managers that we met, receiving a list as soon as the contract is awarded means that it must be accessible when authorities award the contract. However, in the opinion of others we met, the list cannot be received until after the start-up meeting; in fact it is an item on the agenda of that meeting. In the case of two of the contracts of the DI, we even noticed that the list of subcontractors was requested in the start work order. This kind of situation clearly shows that the interpretation of when the list of subcontractors should be sent varies, reinforcing the importance of clearly specifying the city's requirements in the clause provided for this purpose. However, with the exception of the contracts for which a list of subcontractors was received when the bid was submitted, the results of our audit failed to reassure us that, in the majority of other cases, every possible effort was made to obtain a list of subcontractors as soon as possible following the awarding of the contract. 8 9 10 11 12 The DI. The DEP and Ahuntsic-Cartierville borough. The DSTI. The DEP, the DI (2 cases), the DSTI, and Ahuntsic-Cartierville and Verdun boroughs. Mercier-Hochelaga-Maisonneuve, Pierrefonds-Roxboro and Verdun boroughs. Auditor General of the Ville de Montr?al 434 2012 Annual Report 5.10. Subcontracted Construction Work In summary, it is our opinion that the list of subcontractors should be received as soon as possible after the contract is awarded so that the designated person in charge is able to carry out the required checks to ensure that the subcontractors are compliant. 3.2.1.B. Recommendation We recommend that the Direction de l'eau potable, the Direction des infrastructures, and the Direction des strat?gies et transactions immobili?res, as well as AhuntsicCartierville, Mercier-Hochelaga-Maisonneuve, Pierrefonds-Roxboro and Verdun boroughs, undertake a thorough follow-up to obtain the list of subcontractors as soon as possible, once the contract is awarded, in order to be able to ensure that they are compliant. Business units' responses: SERVICE DE L'EAU - DIRECTION DE L'EAU POTABLE [TRANSLATION] An addition will be made to the letter awarding the contract, indicating the contractor's obligation to comply with article 2.3.3 "subcontracting" of the general administrative clauses. (Planned completion: June 2013) SERVICE DES INFRASTRUCTURES, DU TRANSPORT ET DE L'ENVIRONNEMENT - DIRECTION DES INFRASTRUCTURES [TRANSLATION] The list of subcontractors is requested in the letter awarding the contract. A follow-up will be undertaken to ensure that the list is received within the required timeframe. (Planned completion: May 2013) SERVICE DE LA CONCERTATION DES ARRONDISSEMENTS ET DES RESSOURCES MAT?RIELLES - DIRECTION DES STRAT?GIES ET TRANSACTIONS IMMOBILI?RES [TRANSLATION] Contracts for work on buildings involve a high number of subcontractors. It is impossible to obtain the complete list of subcontractors with the bid since not all subcontracts have been entered into before the bidding process. The checklist will enable us to follow up contracts as they are communicated to us. (Planned completion: June 2013) AHUNTSIC-CARTIERVILLE BOROUGH [TRANSLATION] The successful tenderer must provide a list of subcontractors before the borough will award the contract. A table containing all the items to be verified will be prepared to ensure compliance (RBQ, certificate of compliance from Revenu Qu?bec, RENA). These documents will be placed in the contract file folder and digitized. (Planned completion: May 2013) Auditor General of the Ville de Montr?al 435 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit MERCIER-HOCHELAGA-MAISONNEUVE BOROUGH [TRANSLATION] We are adding the following clause to our specifications: [TRANSLATION] 3.3.3.7. The successful tenderer must provide at each work-site meeting an updated copy of the list of subcontractors (amendments and/or additions). He must also provide, for each subcontractor, a photocopy of the licence issued by the R?gie du b?timent du Qu?bec based on the nature of the contract, the certificate of compliance from Revenu Qu?bec and, where applicable, the authorization of the Autorit? des march?s financiers (AMF). (Completed, April 2013) PIERREFONDS-ROXBORO BOROUGH [TRANSLATION] A contract award letter is sent to the contractor as soon as the council awards the contract. This letter specifies a list of documents to be provided, including a list of subcontractors, within 15 calendar days. An internal schedule will be used to follow up receipt of the documents to ensure that they are received within the established timelines. (Planned completion: May 2013) VERDUN BOROUGH [TRANSLATION] Review the process used to analyze the compliance of bids submitted in order to include the list of subcontractors as a document to be received and as a condition for awarding the contract. Review the clauses in the books of specifications accordingly. (Planned completion: May 2013) 3.2.2. Contents of the List of Subcontractors and of the Documents Transmitted 3.2.2.A. Background and Findings The clauses in the books of specifications of all the business units stipulate that the contractor must provide a complete and detailed list of subcontractors. The list must also include a description of the work to be done by each subcontractor, accompanied by a copy of the license issued by the RBQ to each one. The clauses dealing with the obligation to provide subcontractors' certificates of compliance state that the following information must be provided for each subcontract over $25,000: o Name and address of the subcontractor; o Amount and date of the subcontract; o Number and date of issue of the subcontractor's certificate of compliance from Revenu Qu?bec. Auditor General of the Ville de Montr?al 436 2012 Annual Report 5.10. Subcontracted Construction Work We sought assurances that the business units had obtained this information and that it had been entered into the files. Our audit showed that the lists of subcontractors obtained did not have all the required information. In the 12 cases where we found that a list existed, it contained the name and specialty of the subcontractor and, in some, the subcontractor's address and telephone number. Generally speaking, we did not find the following items in the files: o A description of the work to be done by each subcontractor; o A copy of the licence from the RBQ for each subcontractor; o The amount and date of each subcontract above $25,000; o The number and date of issue of the certificate of compliance from Revenu Qu?bec held by the subcontractors regulated under the Act. In all the contracts that we examined, the contractors failed to comply with the subcontracting clauses in the books of specifications and in the call for tenders documents. As well, we did not find any evidence that the project managers or other designated persons had followed up with the contractors to obtain the missing information. For three of the contracts, the contractors did send some information at the time of submitting their bid: o The first involved the contract that we examined for Pierrefonds-Roxboro borough. In this case, the contractor submitted a copy of the certificates of compliance from Revenu Qu?bec for two subcontractors. However, the RBQ licences were not provided. o The second involved one of the contracts that we examined for Verdun borough. In this case, the contractor provided copies of the subcontractors' RBQ licences. However, the certificates of compliance from Revenu Qu?bec were not provided. o The third involved the contract of Mercier-Hochelaga-Maisonneuve borough. In this case, a copy of the RBQ licence of one of the subcontractors was specifically required at the time that the bid was submitted, to ensure that he was compliant. However, copies of RBQ licences for the other subcontractors were not obtained. The information required is necessary to enable the designated persons in charge within the business units to, on the one hand, verify that the subcontractors are qualified to do the work and, on the other hand, ensure that the subcontractors present on the work sites are compliant. Verifications must be made with the RBQ to ensure that the contractor does not have a restricted licence. Other verifications need to be made with the Secr?tariat du Conseil du tr?sor to ensure that the subcontractors are not among the enterprises that are ineligible for public contracts. Auditor General of the Ville de Montr?al 437 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Finally, the amount and date of each subcontract above $25,000, as well as the number and date of issue of the certificate of compliance from Revenu Qu?bec held by the subcontractors are required to enable the designated persons in charge within the business units to have assurances that the subcontractors present on the work sites are in good standing with Revenu Qu?bec. During out audit, we also looked for evidence that business units had performed checks. According to the people that we met during our interviews, these checks of subcontractors are generally not undertaken. This kind of situation means that the business units failed to show that they had sought assurances that the subcontractors had the required licences to perform the work. It is worth mentioning at this point an exception for the contract of the DSTI. We found evidence that the licenses were validated on the RBQ website for each of the subcontractors present on the list provided by the contractor. However, we noted that the checks were completed after work had begun, not before. For all the contracts that we examined, the business units failed to show that they had consulted the RBQ's and the RENA's lists of restricted licences for assurances that subcontractors' licences were compliant. Finally, for all the contracts examined (13/13), the business units failed to obtain assurances that subcontractors had a valid certificate of compliance when entering into the subcontracts. In summary, it is our opinion that the business units should put mechanisms in place to monitor the receipt of all the required information concerning the subcontractors. This would enable them to apply the clauses of the books of specifications and the call for tenders documents and to ensure that the subcontractors are qualified and compliant. It goes without saying that the business units should enter evidence of the follow-up and checks that were undertaken in the contract files. 3.2.2.B. Recommendation We recommend that the Direction de l'eau potable, the Direction des infrastructures, and the Direction des strat?gies et transactions immobili?res, as well as AhuntsicCartierville, Mercier-Hochelaga-Maisonneuve, Pierrefonds-Roxboro and Verdun boroughs, set up mechanisms to strictly monitor compliance with the subcontracting clauses in the books of specifications and the call for tenders documents in order to obtain all the required information. Auditor General of the Ville de Montr?al 438 2012 Annual Report 5.10. Subcontracted Construction Work Business units' responses: SERVICE DE L'EAU - DIRECTION DE L'EAU POTABLE [TRANSLATION] There will be a systematic follow-up of the list of subcontractors at each work site meeting. (Planned completion: June 2013) A daily list of subcontractors present on the work site will be entered in the work site log kept by the construction superintendent. (Planned completion: June 2013) SERVICE DES INFRASTRUCTURES, DU TRANSPORT ET DE L'ENVIRONNEMENT - DIRECTION DES INFRASTRUCTURES [TRANSLATION] Upon receipt of the list, a contract management officer will verify that it is compliant. Any non-compliance will be communicated to the project manager who will notify the contractor so that the latter may submit a compliant list. (Planned completion: June 2013) SERVICE DE LA CONCERTATION DES ARRONDISSEMENTS ET DES RESSOURCES MAT?RIELLES - DIRECTION DES STRAT?GIES ET TRANSACTIONS IMMOBILI?RES [TRANSLATION] The information on the checklist includes: o The name of the subcontractor; o The number of the RBQ license and the Qu?bec business number; o The number of the certificate of compliance from Revenu Qu?bec. (Planned completion: June 2013) AHUNTSIC-CARTIERVILLE BOROUGH [TRANSLATION] The successful tenderer must provide a list of subcontractors before the borough will award the contract. The list should specify the subcontractor's contact information, the amount and date of the subcontract, and the certificate of compliance from Revenu Qu?bec. The construction site superintendent will have this list to ensure monitoring during work on the site. (Planned completion: May 2013) MERCIER-HOCHELAGA-MAISONNEUVE BOROUGH [TRANSLATION] We will make the following amendments to our list of documents to be provided with the call for tenders: we already indicated in the checklist that the list of subcontractors must be provided. On this point, we will complete this document by specifying the information to be provided. (Completed, April 2013) PIERREFONDS-ROXBORO BOROUGH [TRANSLATION] The list of documents requested in the contract award letter will refer the contractor to the subcontracting articles in the books of specifications and call Auditor General of the Ville de Montr?al 439 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit for tenders documents as a reminder of his obligations. An internal schedule will be used to follow up receipt of the documents to ensure that they comply with the requirements of the contract documents. (Planned completion: May 2013) VERDUN BOROUGH [TRANSLATION] Establish follow-up mechanisms to obtain all the required information. (Planned completion: June 2013) 3.2.2.C. Recommendation We recommend that the Direction de l'eau potable, the Direction des infrastructures, and the Direction des strat?gies et transactions immobili?res, as well as AhuntsicCartierville, Mercier-Hochelaga-Maisonneuve, Pierrefonds-Roxboro and Verdun boroughs, put in place mechanisms to record evidence in the files of information obtained on each subcontractor, prior to the start of work, including: o Valid licences from the R?gie du b?timent du Qu?bec; o The certificate of compliance from Revenu Qu?bec; o The verification that the subcontractor is not entered on the register of enterprises ineligible for public contracts. Business units' responses: SERVICE DE L'EAU - DIRECTION DE L'EAU POTABLE [TRANSLATION] Creation of an internal mechanism to verify the documents received from subcontractors. (Planned completion: September 2013) SERVICE DES INFRASTRUCTURES, DU TRANSPORT ET DE L'ENVIRONNEMENT - DIRECTION DES INFRASTRUCTURES [TRANSLATION] The compliant list of subcontractors will be entered in the file, along with copies of valid RBQ licences, certificates of compliance from Revenu Qu?bec and proof of RENA verification. (Planned completion: June 2013) SERVICE DE LA CONCERTATION DES ARRONDISSEMENTS ET DES RESSOURCES MAT?RIELLES - DIRECTION DES STRAT?GIES ET TRANSACTIONS IMMOBILI?RES [TRANSLATION] The information on the checklist includes: o The name of the subcontractor; o The number of the RBQ licence and the Qu?bec business number; o The number of the certificate of compliance from Revenu Qu?bec. (Planned completion: June 2013) Auditor General of the Ville de Montr?al 440 2012 Annual Report 5.10. Subcontracted Construction Work AHUNTSIC-CARTIERVILLE BOROUGH [TRANSLATION] A table comprising all the items to be verified will be prepared in order to ensure that subcontractors are compliant (RBQ, certificate of compliance from Revenu Qu?bec, RENA). Include in the current filing system a section on subcontractors by contract and gather all the subcontractors' certificates of compliance. These documents will be filed in the contract folder. (Planned completion: May 2013) MERCIER-HOCHELAGA-MAISONNEUVE BOROUGH [TRANSLATION] We will create a file for each project that systematically groups together evidence of all the information obtained before the start of work or as soon as it is made known to us, as the case may be. (Completed, April 2013) PIERREFONDS-ROXBORO BOROUGH [TRANSLATION] Upon receipt of the documents requested in the contract award letter, we will carry out an internal verification with the RBQ and the RENA. The certificate of compliance from Revenu Qu?bec will be validated against article 1.4 of the instructions to tenderers. An internal follow-up form placed in the contract file will ensure that this procedure is followed. (Planned completion: May 2013) VERDUN BOROUGH [TRANSLATION] Put in place mechanisms to enter the information received on each of the subcontractors in the file before work begins. (Planned completion: June 2013) 3.2.3. Follow-Up on Additions and Amendments to Subcontractors During the Contract 3.2.3.A. Background and Findings The subcontracting clauses in the books of specifications and call for tenders documents stipulate that, once a contract is awarded, the manager must receive, where applicable, an amended list of subcontractors. The clauses related to the obligation to have a certificate of compliance specify that [TRANSLATION] "for the duration of the contract, the successful tenderer must provide the manager an amended list when entering into any new subcontract before the work entrusted to this subcontractor can begin." Copies of RBQ licences must accompany any amendments to the list. In the case of the certificate of compliance from Revenu Qu?bec, the same information is required before work can begin, i.e. the amount and date of each new subcontract above $25,000, along with the number and date of issue of the subcontractor's certificate of compliance from Revenu Qu?bec. Auditor General of the Ville de Montr?al 441 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit During our audit, we looked for the existence of amended lists of subcontractors and examined the follow-up undertaken by the business units to receive the required information, as well as the nature of the checks carried out. To begin, of the 13 contracts that we examined, amended lists were obtained for three. Two of these were managed by the DSTI and involved construction work on municipal buildings. We failed to find evidence that the project managers had followed up with the contractors, after receiving the amended lists, to obtain the required information. When adding to or amending the list of subcontractors, therefore, the contractors were non-compliant with the subcontracting clauses in the books of specifications and call for tenders documents. As well, we found no evidence that checks were carried out to ensure that the subcontractors were qualified and compliant. During our audit, we also sought assurances that additions or substitutions of subcontractors had been communicated to the business units involved. To do this, we looked at the superintendent's daily work site log and the minutes of work site meetings. Keeping in mind that business units receive contract withdrawals by subcontractors, we compared the list of subcontractors initially sent by the contractor with the withdrawals received. In the case of withdrawals, it should be specified that, under the Qu?bec civil code, a subcontractor can withdraw from a contract it has entered into with a contractor in order to maintain its right to a legal mortgage in the event that the contractor fails to pay him. Based on the information obtained, subcontractors can withdraw voluntarily from their contract with contractors. The business units have no assurance, therefore, that they are being informed of all the subcontractors based on the withdrawals received. Upon examining the contract monitoring reports, we found that the form and content of these reports varied from one business unit to the other, depending on whether the reports had been prepared by the city's superintendents or by outside firms. The form used for the monitoring reports of Pierrefonds-Roxboro borough already has a section reserved for subcontractors. In the case of two contracts of the DI, we noted that the work site logs showed the name of the subcontractors present and the nature of the work done. In the case of the other contracts that we examined, we could not be sure that the monitoring reports systematically showed the names of all the subcontractors that were performing the work. Based on the information obtained for all the contracts selected, our examination of the daily monitoring reports revealed that the names of some subcontractors did not appear on the original list provided by the contractor. Our audit also failed to find in these monitoring reports the names of all the subcontractors who appeared on the initial list provided by the Auditor General of the Ville de Montr?al 442 2012 Annual Report 5.10. Subcontracted Construction Work contractor. This situation leads us to believe that either the monitoring reports do not contain all the information about the subcontractors or that the initial list is not representative of all the subcontractors who actually worked on the work site. The minutes of the work site meetings failed to reveal any significant information that would enable us to evaluate the completeness of the initial list of subcontractors. Furthermore, our analysis of contract withdrawals received from subcontractors showed that, in seven of the contracts, some subcontractors did not appear on the list provided by the contractor. Since the value of the subcontract is shown on the withdrawals, we found that that almost half of the subcontracts had a value that ranged from $25,000 to several hundred thousand dollars. This analysis also revealed the presence of subcontracts issued by subcontractors for three of the contracts that we examined. In summary, the analysis of the monitoring reports and of the contract withdrawals revealed the existence of subcontractors that were not on the lists of subcontractors sent by the contractor. This situation means that, in the case of these contracts, an amended list of subcontractors was not received, as required by the books of specifications and the call for tenders documents. Also, because the business units failed to receive the amended lists and required information, they were obviously unable to validate whether the new subcontractors were qualified and compliant. According to the information obtained, no comparison was done between monitoring reports, contract withdrawals received and lists of subcontractors. While we wondered about the completeness of the information on the subcontractors contained in the monitoring reports, it is our opinion that construction site superintendents should be made aware of the importance of being thorough in logging information about subcontractors. Business units should also ensure that monitoring reports are compared with lists of subcontractors submitted by contractors. Along the same lines, we also feel that contract withdrawals received from subcontractors should be compared with the list of subcontractors. Auditor General of the Ville de Montr?al 443 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.2.3.B. Recommendation We recommend that the Direction de l'eau potable, the Direction des infrastructures, and the Direction des strat?gies et transactions immobili?res, as well AhuntsicCartierville, Mercier-Hochelaga-Maisonneuve, Pierrefonds-Roxboro and Verdun boroughs, put control mechanisms in place to ensure receipt, from contractors, of complete and updated information about subcontractors in order to have all the required information and carry out the necessary verifications to ensure compliance with the requirements set out by the city in the books of specifications. Business units' responses: SERVICE DE L'EAU - DIRECTION DE L'EAU POTABLE [TRANSLATION] There will be a systematic follow-up of the list of subcontractors at each work site meeting. (Planned completion: June 2013) A daily list of subcontractors present on the work site will be entered in the work site log kept by the construction superintendent. (Planned completion: June 2013) SERVICE DES INFRASTRUCTURES, DU TRANSPORT ET DE L'ENVIRONNEMENT - DIRECTION DES INFRASTRUCTURES [TRANSLATION] At the start-up meeting, the contractor will be reminded of his obligation to submit all amendments to the list of subcontractors to the city for approval; and the procedure that applies in the event of non-compliance with this requirement will be explained. (Planned completion: June 2013) SERVICE DE LA CONCERTATION DES ARRONDISSEMENTS ET DES RESSOURCES MAT?RIELLES - DIRECTION DES STRAT?GIES ET TRANSACTIONS IMMOBILI?RES [TRANSLATION] In addition to the measures already mentioned, a correlation will be made between the list of subcontractors who have withdrawn their contract and the contractor's list of subcontractors in order to ensure that the latter is complete. A sample document enabling this correlation will be added to the electronic bank of contract management documents. The checklist will refer back to this document. (Planned completion: June 2013) AHUNTSIC-CARTIERVILLE BOROUGH [TRANSLATION] The construction site superintendent and the project manager (architect, engineer, property manager, landscape architect, and others) must keep an updated list of subcontractors. At each work site meeting, an item on the agenda will deal with subcontractors so that they can be updated. If there are any changes, checks can be carried out and Auditor General of the Ville de Montr?al 444 2012 Annual Report 5.10. Subcontracted Construction Work the contractor must provide all the supporting documents. (Planned completion: May 2013) MERCIER-HOCHELAGA-MAISONNEUVE BOROUGH [TRANSLATION] See the specifications clauses mentioned above in response to recommendations 3.1.C and 3.2.1.B (clauses 3.3.3.1, 3.3.3.7 and 3.3.3.8.). In addition, a dual verification is systematically undertaken when payments are discharged. (Completed, April 2013) PIERREFONDS-ROXBORO BOROUGH [TRANSLATION] Checking the compliance and validity of the information provided by the contractor and his subcontractors will be achieved by consulting various bodies. In order to ensure that the information is up to date, various websites will be used. As well, construction site superintendents will monitor the work site in order to validate the list of subcontractors provided by the contractor. In the event of noncompliance, corrective measures will be taken. (Planned completion: May 2013) VERDUN BOROUGH [TRANSLATION] Put in place control mechanisms to ensure receipt, from the general contractor, of complete and updated information about subcontractors, especially when an addition or amendment is made to the subcontractors during the contract. (Planned completion: June 2013) 3.2.4. Measures to Apply the Clauses 3.2.4.A. Background and Findings Monitoring must be carried out to ensure that contractors apply the clauses in the books specifications and the call for tenders documents so that the required information received and the necessary checks are completed. In the event that a contractor fails comply with his contractual obligations, there are provisions in the clauses of the books specifications to address the issue with the contractor. of is to of For example, the contractor is clearly at fault if he: o Has work performed by a subcontractor who does not hold the appropriate licence; o Contravenes laws, decrees, orders in council, regulations or managers' instructions. In the event of default, the manager's authorized representative must notify the contractor and caution him about these defaults and the deadline for remedying the situation. Auditor General of the Ville de Montr?al 445 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit In the case of the contractor's obligations to supply information on subcontractors who are performing work on work sites, we identified several discrepancies in the preceding sections, including: o The list of subcontractors not sent or not containing all the information required by law (for example, the date and amount of any subcontract above $25,000, the date and number of subcontractors' certificates of compliance); o The amended list of subcontractors not transmitted following the addition of subcontractors; o Copies of subcontractors' RBQ licences not provided. Based on the information received, it is often the same contractors and subcontractors who perform the work, so project managers do not always see the relevance of obtaining the information requested about subcontractors' qualifications and compliance. Therefore no measures have been taken to date by the business units with whom we met to point out these discrepancies to the contractors and the fact that corrective action needs to be taken regarding subcontracting. In our opinion, the business units should provide thorough monitoring of how the clauses in the books of specifications and call for tenders documents are applied. They should also ensure that actions taken are properly documented. This will make it easier afterward to strictly apply the default clauses and not to leave the impression of "tolerant" management. 3.2.4.B. Recommendation We recommend that the Direction de l'eau potable, the Direction des infrastructures, and the Direction des strat?gies et transactions immobili?res, as well as AhuntsicCartierville, Mercier-Hochelaga-Maisonneuve, Pierrefonds-Roxboro and Verdun boroughs, properly document the follow-up undertaken with contractors to obtain the information required in the books of specifications and call for tenders documents in order to be able to apply the default procedure when required. Business units' responses: SERVICE DE L'EAU - DIRECTION DE L'EAU POTABLE [TRANSLATION] The follow-up with the contractors will be entered in the minutes of work site meetings. (Planned completion: June 2013) The work site log kept by the superintendent will be updated daily to reflect the subcontractors present on the work site. (Planned completion: September 2013) Auditor General of the Ville de Montr?al 446 2012 Annual Report 5.10. Subcontracted Construction Work SERVICE DES INFRASTRUCTURES, DU TRANSPORT ET DE L'ENVIRONNEMENT - DIRECTION DES INFRASTRUCTURES [TRANSLATION] The amended compliant list of subcontractors will be entered in the file, along with copies of valid RBQ licences, certificates of compliance from Revenu Qu?bec and proof of RENA verification. (Planned completion: June 2013) SERVICE DE LA CONCERTATION DES ARRONDISSEMENTS ET DES RESSOURCES MAT?RIELLES - DIRECTION DES STRAT?GIES ET TRANSACTIONS IMMOBILI?RES [TRANSLATION] Implementation of the checklist will complete the monitoring measures already in place and standardize the way information is presented. (Planned completion: June 2013) AHUNTSIC-CARTIERVILLE BOROUGH [TRANSLATION] Preserve all traces of the follow-up undertaken with contractors on the issue of subcontractors. (Planned completion: May 2013) MERCIER-HOCHELAGA-MAISONNEUVE BOROUGH [TRANSLATION] As set out in clauses 3.3.3.7 and 3.3.3.8 of the specifications (see responses to recommendations 3.1.C and 3.2.1.B), regular follow-up will now be undertaken at each work site meeting, in addition to a thorough examination of the payments discharged and requests for payment for work performed by subcontractors, prior to authorizing such payments. All this information will be kept in the subcontracting file, as mentioned in our response to recommendation 3.2.2.C. (Completed, April 2013) PIERREFONDS-ROXBORO BOROUGH [TRANSLATION] An internal form will be used to record each level of verification of information received, and the supporting documents will be printed and kept in the relevant contract file. (Planned completion: May 2013) VERDUN BOROUGH [TRANSLATION] See to it that project managers document all follow-up undertaken with contractors to obtain the required information in order to be able to apply the relevant procedure in the event of default. (Planned completion: June 2013) Auditor General of the Ville de Montr?al 447 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 4. Appendix 4.1. Summary of the Work Contracts Examined Table A - Sample of the Contracts Selected Selection No. Nature of the work Business units Amount of the awarded contract Date the contract was awarded 1 Water mains DEP $16,420,479 2012-08-23 2 Water mains DEP $8,083,625 2012-05-17 a 3 DI Pavement and sidewalks DI 5 Pavement and markings DI 6 Drinking water plant repairs DSTI 7 Building DSTI $7,615,511 2012-08-20 8 Pavement and sidewalks Ahuntsic-Cartierville borough $2,065,159 2012-09-19 9 Pavement and sidewalks Ahuntsic-Cartierville borough $2,566,389 2012-10-04 10 Pavement repairs Verdun borough $3,433,210 2012-08-22 11 Sewer reconstruction Verdun borough $877,600 2012-06-01 12 Pavement and lateral mains Pierrefonds-Roxboro borough $601,943 2012-09-05 13 a Bridge demolition 4 Construction of an indoor pool Mercier-Hochelaga-Maisonneuve borough $12,873,530 2012-07-03 $12,996,905 2012-05-14 $8,329,798 2012-08-20 $1,974,088 2012-08-20 $9,220,535 2012-06-21 This contract is under the Direction des transports of the Service des infrastructures, du transport et de l'environnement as of January 1, 2013, following changes in the city's organizational structure. Auditor General of the Ville de Montr?al 448 2012 Annual Report Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 "Integrated Human Resources and Payroll Management System" Project 5.11 5.11. "Integrated Human Resources and Payroll Management System" Project Table of Contents 1. Introduction ................................................................................................................... 453 2. Audit Scope................................................................................................................... 455 3. Findings and Recommendations .................................................................................. 456 Auditor General of the Ville de Montr?al 451 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit List of Acronyms CSTI Comit? sectoriel des technologies de l'information STI Service des technologies de l'information SPVM Service de police de la Ville de Montr?al TM time management Auditor General of the Ville de Montr?al 452 2012 Annual Report 5.11. "Integrated Human Resources and Payroll Management System" Project 5.11. "Integrated Human Resources and Payroll Management System" Project 1. Introduction On October 8, 2003, the executive committee approved a project to implement a common and integrated platform regrouping the financial, procurement, human resources and payroll management systems. To this effect, the executive committee allocated a global budget of $53.5 million to the project. Once completed, this project--named "SIMON" (for Syst?me Int?gr? Montr?al)--was expected to: o Support proximity services provided by the boroughs; o Generate substantial savings through a reduction of payroll and accounting system management costs; o Improve services to citizens and reduce the costs of these services by providing access to standardized management data at all times and at all levels as well as by offering activity-based cost tracking to measure performance improvement; o Eliminate the duplication of activities and tasks by ultimately eliminating more than 50 major systems, streamlining decision making, reducing the workload associated with low-value added activities and automating a certain number of processes. Moreover, the main objective of the SIMON program's human resources and payroll component (the "Integrated HR-Payroll MS project") was to provide the Ville de Montr?al (the city) with an integrated management system supporting processes based on the best practices of large organizations. The initial scope of the Integrated HR-Payroll MS project included the following 15 processes as well as the management information production function, as shown in Figure 1. Auditor General of the Ville de Montr?al 453 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Figure 1 - HR-Payroll Function Processes Processes not included in the project Processes included in the project No. 1 Organizational structures No. 2 Jobs No. 3 Workforce planning No. 4 Candidate pools No. 5 Staffing No. 6 Performance appraisals No. 7 Career planning No. 8 Training No. 9 Conflict management No. 11 Group insurance plans No. 13 Time management No. 14 Salaries and benefits No. 15 Remuneration to be paid No. 16 Payroll management No. 17 Labour costs No. 10 Occupational health No. 12 Pension plan Process-managed data Data warehouse Dashboard From the aforementioned $53.5 million global budget, a sum of $25 million was to be spent on completing the Integrated HR-Payroll MS project. According to the estimated timeframe, all processes were to have been implemented by May 31, 2007. To promote the successful completion of the Integrated HR-Payroll MS project, the following orientations were determined: o As with the financial resources and procurement components, Oracle Suite products would be used; o The Oracle Suite products would be used in their "vanilla 1 " form; o The services of external Oracle Suite product specialists would be retained to facilitate the integration of these data processes in the city's IT environment; o Appropriate and sufficient internal resources would be assigned to work with the "integrator" (external resources). Between 2004 and 2006, some features related to processes of Integrated HR-Payroll MS project were implemented while the Service du capital humain was the project's proponent 1 This refers to the implementation of the original version of the software package in order to minimize change management activities. Auditor General of the Ville de Montr?al 454 2012 Annual Report 5.11. "Integrated Human Resources and Payroll Management System" Project and acting as the owner of these processes. Furthermore, the Kronos software package was selected to support the time management (TM) process. However, a moratorium on the SIMON program was announced in July 2006 and this brought all work on the Integrated HR-Payroll MS project to a sudden stop. In April 2007, work on the TM process as part of the Integrated HR-Payroll MS project (the "TM subproject") resumed under the responsibility of the Service des finances, which separated the subproject from the SIMON program. Moreover, the other processes inherent in this project remained pending for several years, although certain human resources management process functions had been developed since then as part of sectoral projects. In this regard, we note that the Service du capital humain decided in 2007 to continue developing the human resources management processes outside of the Integrated HRPayroll MS project, as indicated in the IT business plan developed accordingly. In April 2009, a summary report was produced on the experience with the SIMON program during the 2003-2008 period. This report dealt essentially with the finance and procurement functions, including the evolution of the SIMON program and lessons learned along the way. In March 2010, the executive committee was presented with a progress report on the SIMON program and emphasized the relevance of pursuing the strategy that had been put in place in 2004. To this effect, reference was made to the fact that several of the Integrated HR-Payroll MS project's initial processes as well as the complementary modules for the Finance and Procurement components of the SIMON program had not been completed. Furthermore, the report highlighted the fact that extensive planning of the overall project was scheduled for 2010. However, this planning had not been completed at the time we concluded our audit (October 2012) although the CSTI (Comit? sectoriel des technologies de l'information) at the time approved the HMRS-Payroll project and the core human resources management processes to this effect. Meanwhile, work on the TM subproject continued and was still under way at the end of our audit. 2. Audit Scope Initially, our audit dealt only with the work carried out under the TM subproject. However, considering that the TM subproject is based on the Integrated HR-Payroll MS project and that--following the moratorium announced in July 2006--the revival of this subproject Auditor General of the Ville de Montr?al 455 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit remained both uncertain and challenging at the time of our audit, we decided to extend the scope of our audit to include the project's then current status. Given that the scope of analyzing the status of the Integrated HR-Payroll MS project is in no way comparable to conducting an analysis of the TM subproject's status, we decided to tackle each of these analyses in separate audit reports. This audit report addresses the current status of the Integrated HR-Payroll MS project. In this regard, we set the following objectives for our audit: o Take stock of the moratorium announced in 2006, the measures under way, the findings already established and the contemplated scenarios; o Assess the current situation with respect to project governance and the capacity to efficiently manage human resources management processes, particularly payroll processes. We conducted most of our audit work between April and October 2012, and our audit included a global review of the key aspects of the Integrated HR-Payroll MS project since its launch in 2004. 3. Findings and Recommendations 3.1.A. Background and Findings Impact and Aftermath of the July 2006 Moratorium The Integrated HR-Payroll MS project began in 2004 and continued until the announcement of the moratorium in July 2006. An impact analysis of this moratorium on the Integrated HRPayroll MS project carried out in January 2007 highlighted the following consequences inter alia: o An additional load on the systems scheduled to be eliminated following the implementation of the Integrated HR-Payroll MS project, since those systems would need to remain operational for a longer period than initially anticipated; o The high risk associated with the availability of the necessary expertise to support and develop some of these systems. Furthermore, no vendor support would be provided beyond 2007 for the payroll systems of certain boroughs created from former suburban municipalities; o The non-use of a considerable percentage of licenses purchased and the annual recurring license support fees paid by the city. These fees represent 22% of the initial cost of these licenses or an annual expense of $2 million; Auditor General of the Ville de Montr?al 456 2012 Annual Report 5.11. "Integrated Human Resources and Payroll Management System" Project o o The risk of losing the Integrated HR-Payroll MS project's architectural and functional expertise and knowledge and apprehended challenges of rebuilding a team having the same knowledge; The need to determine the orientations concerning the moratorium in order to establish and prepare the next phase of activities. Based on our analysis of the events that followed the moratorium and of testimonies gathered during our audit, it is our understanding that the Integrated HR-Payroll MS project was ignored for several years. Indeed, at the time of our audit, the project had no official status and was no longer included in the three-year capital expenditures program (2012- 2014). Nevertheless, some of this project's processes were later completed through sectoral development whereas others were being completed or undergoing analysis and assessment. Furthermore, we could not retrace a single official statement made by the Direction g?n?rale and authorities with respect to Integrated HR-Payroll MS project reorientations or even the continuation of the project. An informal progress report on the administrative systems--including those covered by the Integrated HR-Payroll MS project--was presented to the executive committee in March 2010, but no decision regarding a proposed "future vision" for these systems followed. Substantial resources were invested to implement the Integrated HR-Payroll MS project's management model, which is partially completed but for which core deliverables have yet to have been provided or even initiated. In our opinion, departing from the integrated management model advocated by the city comprises very significant risks. Among these are the risk of a disorderly development that no longer meets needs and encourages the duplication of processes and data, thus increasing operating and maintenance costs and jeopardizing the initial objective set for the completion of the Integrated HR-Payroll MS project as well as its benefits. Progress Status and Costs At the time the moratorium was announced in July 2006, it was obvious that the Integrated HR-Payroll MS project was considerably behind schedule. Indeed, none of the 15 processes (see Figure 1) had been fully deployed, the configuration of three human resources management processes had not begun and the only payroll processes that had been deployed were those pertaining to judges, elected officials and retirees. It should be noted that work on managing the payrolls of the other city employee groups had not yet Auditor General of the Ville de Montr?al 457 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit begun and that the work on TM processes was under way only in the Lachine and LaSalle boroughs, the wastewater treatment plant and the Centre d'urgence 911. With respect to costs, from the outset, the $25 million budget allocated to the Integrated HR-Payroll MS project was never broken down into subprojects and deliverables. Furthermore, in the case of the SIMON program, expenses had been accounted for on a global basis over a period of years, thus making it impossible to differentiate the costs incurred by the Integrated HR-Payroll MS project from the subprojects and deliverables stemming from this project. Simply put, it became futile to attempt to accurately reconcile the costs incurred by the Integrated HR-Payroll MS project with the authorized budget in some detail. We were nevertheless able to estimate certain costs incurred by the Integrated HR-Payroll MS project based on an estimate of the costs incurred until the announcement of the moratorium as well as cost projections for the TM subproject as at December 31, 2012. These costs were thus estimated at approximately $4.5 million and $18.5 million respectively, representing a subtotal of $23 million out of an initial $25 million global budget. Taking into account the work required to carry out or complete the deployment of the different processes and the fact that the preliminary cost estimate for the active employee payroll process subproject represents in itself an investment of at least $20 million, the overall cost of the Integrated HR-Payroll MS project will undeniably be much higher than the initial budget. Moreover, in our search for information to establish the cost of the Integrated HR-Payroll MS project, we examined a contract awarded by the city to CGI in November 2004 in which this firm was acting as the system integrator. The object of this contract, initially valued at $10.9 million, was to provide the [TRANSLATION] "professional services required to assemble and deploy human resources and payroll work processes and all related services" and it was to end on May 31, 2007. To this initial amount of $10.9 million could be added a sum of $2.1 million to cover additional related professional services that could be required. Our audit revealed that this contract ended up costing $11.6 million and that Montr?al paid professional fees under it until the end of the year of 2010. As previously mentioned, a considerable amount of work stipulated under this contract had not yet been completed or even initiated when the moratorium was imposed in July 2006. Furthermore, the possible contribution made by CGI under this contract to the TM subproject--which began in April 2007--can be very minimal at most, given how slowly work on this subproject progressed between April 2007 and the end of 2010. This assertion is all the more justified, Auditor General of the Ville de Montr?al 458 2012 Annual Report 5.11. "Integrated Human Resources and Payroll Management System" Project given the fact that the city retained the services of this same firm under another agreement signed in 2010 for the purpose of this same project. In fact, CGI likely rendered services other than those initially provided under the contract to the city, as it appears inter alia from a legal opinion provided to the acting city manager on July 25, 2006. This opinion dealt solely with the city manager's authority to mandate CGI to provide related services for the completion of the SIMON program's finance and procurement functions (despite an initial mandate given to DMR Conseil) without issuing a public call for tenders. The opinion concludes that: [TRANSLATION] "the city has the right to mandate CGI to pursue the completion of the new integrated financial, human and material resources management system using Oracle products, regardless of the fact that the city had initially entrusted only a part of this human resources and payroll functions project to CGI and that the city manager may require work to this effect." In our opinion, this conclusion is rather surprising in the following respects: o The reason for this conclusion, according to which [TRANSLATION] "the project managers had already used up the amounts allocated by contract to CGI to pay for part of the work provided under the contract with DMR Conseil. In a context where DMR Conseil spent all of the amounts awarded under its contract and that this firm is no longer able to continue its work without additional funding, it is our opinion that nothing prevents the city from mandating CGI to complete the overall project, including the finance [and] procurement functions, through its initial contract allocations for the assembly and deployment of human resources and payroll management processes." o The contract with DMR Conseil was interrupted because of a lack of financial resources; however, most of the contract concerned work that the city intended to entrust to CGI. o The legal opinion does not address the question as to whether DMR Conseil, whose services were retained at the end of a public call for tenders process to carry out the finance and procurement functions, was entitled to the additional work entrusted to CGI without first issuing a call for tenders. o In fact, at the time the legal opinion was issued, the "main" contract, i.e., the contract with CGI, was subject to a moratorium resulting in the interruption of the work on the human resources and payroll management processes. How could a consequential amendment then be made in the absence of a main contract? We raise the question of whether or not the author had access to all of the information for the purpose of issuing his legal opinion. Moreover, regardless of whether or not this legal opinion is founded, we remain concerned by this situation--which evokes a waiver of the fundamental public administration principle according to which amounts approved for specific purposes by authorities, in this case the executive committee, cannot be used for Auditor General of the Ville de Montr?al 459 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit other purposes without said authorities' prior approval. However, we note that this situation occurred under a former administration. Lessons Learned and Revival of the Project We share the position stated in the aforementioned presentation made to the executive committee in March 2010, namely that [TRANSLATION] "despite the significant gap with respect to the target 2 , the stakeholders are more than ever before convinced of the relevance of continuing the strategy initiated in 2004." In our opinion, it is unacceptable that an organization the size of Montr?al does not yet dispose of appropriate administrative systems to manage processes that are a priori relatively simple. Consequently, we believe that the entire Integrated HR-Payroll MS project must be revived and that the results it produced in the past must be built on. In this respect, we examined the root causes of the considerable delays and cost overruns that hindered the project. Although not comprehensive, our work led us to identify the following difficulties that will be instrumental in reviving the project. We must emphasize that several of these difficulties transect the lessons learned that were identified in the aforementioned summary report as far as the finance and procurement functions are concerned. Moreover, we would like to point out that, at the time of our audit, the Service des technologies de l'information (STI) was in the process of implementing a new information technology project management framework which should, in principle, mitigate the risks of future projects being hindered by the same problems. o Customization of the Oracle Suite Products The initial intention was to complete the Integrated HR-Payroll MS project according to the following basic orientation: the Oracle Suite products were to be used in their "vanilla" form. This orientation was based on the presumption that the city's processes would need to be adapted in accordance with the Oracle products' data processes in order to minimize change management activities. We must emphasize that the best process management practices at issue are nested in the Oracle products. The many changes made to adapt the Oracle Suite products to the city's processes explain the considerable delays and cost overruns that hindered the project. 2 The presentation document mentions that the project is twice as costly to complete than anticipated and takes four times longer than expected. Auditor General of the Ville de Montr?al 460 2012 Annual Report 5.11. "Integrated Human Resources and Payroll Management System" Project Furthermore, the departures from this basic orientation resulted in the city's processes being less closely aligned with the best practices. o Poor Risk Management A detailed risk analysis was initially conducted as part of the Integrated HR-Payroll MS project. However, the mitigation measures then identified were never implemented during the project. Furthermore, except for the first year of the project, this risk analysis was not updated thereafter to take into account the evolving circumstances under which the project had been carried out. Admittedly, these deficiencies hindered the project team's capacity to foresee difficulties along the way and react promptly by implementing the necessary measures under the circumstances. o Numerous Changes to the Project's Structure and Organization The initial structure set up to manage the Integrated HR-Payroll MS project was aligned with current practices for projects of equivalent scope. However, the numerous changes made along the way hindered the proper unfolding of the project. Furthermore, the lack of resources and expertise in the business fields and affected units combined with the city's lack of experience in major business process transformation projects contributed to the aforementioned delays and cost overruns. o Lack of Leadership, Follow-Up or Accountability Over and above the aforementioned lack of follow-up with respect to risk management, we note that the summary report on the experience with the SIMON program (see above), which also applies to the HR-Payroll component, points to the absence of performance indicators which--in our opinion--are required to enable the Direction g?n?rale to ensure a rigorous control of this major project and periodic reporting to the executive committee on the project's progress. It should be remembered that one of the factors that was initially invoked to justify the need for this project was the fact that the large-scale savings that the organizational model for the "new city" was to generate stemmed mainly from the optimization and transformation of the business processes, including those under the scope of the Integrated HR-Payroll MS project. As we mentioned previously, the executive committee had allocated a $25 million budget to this project. Auditor General of the Ville de Montr?al 461 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit While we understand that the moratorium announced in July 2006 made it impossible to continue the Integrated HR-Payroll MS project, it is imperative that the Direction g?n?rale rigorously track all major projects and report periodically thereon to the executive committee. This is fundamental for enabling the authorities to make informed decisions and fulfill their governance obligations. Over and above considering the problems noted above, it is our opinion that the orientation focussing on internal development, initially adopted for the subproject to support work on the payroll process, must be re-examined as part of the revival of the Integrated HR-Payroll MS project. Indeed, that decision had been based mainly on the assumption that it would cost less to develop the project internally than outsourcing it for organizations employing more than 10,000 people (according to an analysis conducted by the STI). The actual costs incurred to date by the subprojects under the Integrated HR-Payroll MS project are much higher than initially estimated. Furthermore, the outsourcing market has evolved since 2004, specifically with respect to prices. Under the circumstances, the opportunity analysis that had been conducted at the time to determine the best way to proceed with the payroll subproject must be updated in light of how the situation has evolved. 3.1.B. Recommendation We recommend that the Direction g?n?rale take all necessary measures to: A) Revive the entire Integrated HR-Payroll MS project in line with the strategy adopted in 2004 and the lessons learned from the past, and then decide on the objectives, development model and orientations that should be emphasized accordingly; B) Update the cost estimates to complete the Integrated HR-Payroll MS project; C) Make a recommendation to the executive committee proposing an action plan to revive the Integrated HR-Payroll MS project as well as an update of the costs to complete it; D) Reassess the orientation focussed on internal development initially adopted for the payroll process; E) Ensure a rigorous management framework for the project and periodic status reports on the project to the executive committee. Auditor General of the Ville de Montr?al 462 2012 Annual Report 5.11. "Integrated Human Resources and Payroll Management System" Project Business unit's response: A) [TRANSLATION] The CSTI resumed the Integrated HR-Payroll MS project in July 2011 during a presentation in the form of a summary business case. The CSTI then authorized the project's feasibility study and business case. The project deals solely with the human resources management and payroll processes, which are essential to payroll management. As part of the implementation of the Oracle payroll system, the unmet needs cover the following activities and processes (inclusions): o Organizational structures (no. 1); o Jobs (no. 2); o Staffing (no. 5); o Salaries and benefits (no. 14); o Remuneration to be paid (no. 15); o Payroll management (no. 16); o Labour costs (no. 17, management of pay scales only); o Implementation of the foundations for business intelligence functions; o Review of the interfaces identified in the functional architecture. The following activities and processes are not included in the project (exclusions): o Workforce planning (no. 3); o Candidate pools (no. 4); o Performance appraisals (no. 6); o Career planning (no. 7); o Training (no. 8); o Conflict management (no. 9); o Group insurance plans (no. 11); o Labour costs (no. 17); o Archiving method for systems that are abandoned under the Archives Act; o Replacement or linking of peripheral mission systems used by business units to manage their everyday operations; o Implementation of new processes that will modify the data structure and impact several of the peripheral systems operated by SIMON external clients. The work required to adapt the external interfaces to the SIMON program will be carried out by the applicable system owners; o High level of availability of the SIMON technological infrastructure that must be included in a related project (management of server capacities in the three-year capital expenditures plan); o Payroll of the Service de police de la Ville de Montr?al (SPVM) not included in phase 1. The positioning of the aforementioned exclusions in the project must be updated, which will require a revision of the business needs. A cost and effort evaluation Auditor General of the Ville de Montr?al 463 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit as well as work completion schedule will be filed before the end of the current year. (Planned completion: December 2013) Figure 2 - Positioning in Accordance with the Business Needs of the HR-Payroll Function Processes Processes not included in the project Processes included in the project No. 1 Organizational structures No. 2 Jobs No. 3 Workforce planning No. 4 Candidate pools No. 5 Staffing No. 6 Performance appraisals No. 7 Career planning No. 8 Training No. 9 Conflict management No. 11 Group insurance plans No. 13 Time management No. 14 Salaries and benefits No. 15 Remuneration to be paid No. 16 Payroll management No. 17 Labour costs No. 10 Occupational health No. 12 Pension plan Process-managed data Data warehouse Dashboard Human resources processes reviewed during phase 1 of the project Payroll processes reviewed during phase 1 of the project TM process (project under way) Processes not included in the scope of the project B) With respect to the financial aspect of the project, a project case file totalling $36.9 million was presented to the CSTI on June 21, 2012. An external audit was ordered at that time, and the audit results were filed with the CSTI on October 18, 2012. Upon completion of this audit, the total was reduced by about $3 million. The report also highlighted three major project issues that had already been identified, i.e., the linking of the collective agreements to the Oracle software package, organizational capacity management as well as the convergence of three major projects in the SIMON program. This financial evaluation was developed in accordance with the scenario that had been chosen by the project's steering committee, namely: o Implementation of the human resources and payroll management processes for the employees paid by the city; o Implementation of the SPVM human resources record (excluding the phase 1 payroll component). Auditor General of the Ville de Montr?al 464 2012 Annual Report 5.11. "Integrated Human Resources and Payroll Management System" Project The SPVM's TM and Payroll components will be implemented during phase 2, which has yet to be evaluated. The financial evaluation will be reviewed and validated once again at the end of the collective agreement planning and analysis phase with the Oracle software package (scheduled for June 2013). Furthermore, the evaluation of the costs for the entire Integrated HR-Payroll MS project will be reviewed before December 31, 2013 in order to estimate the cost of implementing the processes that are not included in the scope of the project under way, which is basically limited to the payroll processes. (Planned completion: December 2013) C) Since the beginning of 2013, the project is in its planning phase (phase 3) and presentations will be made to the authorities to have the completion phase (phase 4) approved and launched in July 2013, in accordance with the major project management framework. (Planned completion: July 2013) Calendar: o Administrative systems steering committee; o Ad hoc proponent committee; o CSTI; o Major project monitoring committee; o Executive committee. (Planned completion: June 2013) D) To our knowledge, these are the different studies and analyses that have been conducted for the city with respect to payroll outsourcing: o EDS, July 2004, Comparative payroll study: outsourcing versus internal management; o Internal study, December 2004, Human resources / Payroll project - Payroll management processes - Oracle versus outsourcing; o Raymond Chabot Grant Thornton, October 2006, Opportunity analysis for optimizing payroll management; o Internal study, October 2007, Project: PAYROLL positioning - Current situation and payroll management positioning for the Ville de Montr?al; o Raymond Chabot Grant Thornton, April 2008, Benchmarking study and financial analysis of payroll management solutions - comparative analysis - outsourcing versus internal management; o Internal study, June 2012, HR-PAYROLL project case, 1.2.1. Outsource or not?, pages 6 and 7. We generally conclude that a widespread opinion in the industry is that, beyond a certain number of employees, outsourcing payroll management is no longer worth the cost. In its report, the Bureau du v?rificateur g?n?ral sets the bar at 10,000 employees or more. The only process that could be outsourced is payroll management (process no. 16). This process involves using the direct remuneration to be paid to Auditor General of the Ville de Montr?al 465 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit produce the employee payroll and making the different deductions required by law. This process includes printed cheques, pay slips, bank records and--in the case of certain external bodies--payroll accounting, tax slips and all regular and ad hoc reports. Given all of the developments made with respect to the SIMON program in recent years, it is our opinion that outsourcing payroll production no longer offers the interests or benefits that could have justified the decision to outsource in the past. Our approach remains unchanged: we want to make use of our assets and continue to build on what has already been done while managing our risks and costs. A meeting is planned with the city manager on April 26 to confirm the orientations that will be taken with respect to payroll outsourcing in the current situation. The study may be updated after this meeting. (Completed, April 2013) E) When the project resumed in 2011, it was carried out in accordance with the methodology implemented by the Division r?alisation de projets TI. This involves reporting to the executive committee on all major projects, such as the HRPayroll project, as soon as their completion phase is launched. Beyond reporting to the executive committee, the project must also be presented to the major project monitoring committee. It is also closely monitored by STI management in accordance with the project management mechanisms (project progress dashboards that are filed on a monthly basis). (Planned completion: spread out over the project's planned duration, i.e., from July 2013 to December 2016, with reporting to the executive committee twice a year) Auditor General of the Ville de Montr?al 466 2012 Annual Report Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 "Time Management" Subproject 5.12 5.12. "Time Management" Subproject Table of Contents 1. Introduction ................................................................................................................... 471 2. Audit Scope................................................................................................................... 472 3. Findings and Recommendations .................................................................................. 473 3.1. "Time Management" Subproject Governance ....................................................... 473 3.2. Development Phases ............................................................................................ 477 3.3. Security ................................................................................................................. 485 3.4. Performance.......................................................................................................... 490 3.5. Evolution of the Kronos Platform........................................................................... 492 3.6. Continuity Plan ...................................................................................................... 493 Auditor General of the Ville de Montr?al 469 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit List of Acronyms CCGPE Comit? corporatif de gestion des projets d'envergure LDAP Lightweight Directory Access Protocol CMPE Comit? de monitoring des projets d'envergure POM project organization manual SCHC CSTI Comit? sectoriel des technologies de l'information Service du capital humain et des communications STI DRPTI Division r?alisation de projets TI Service des technologies de l'information TM time management Auditor General of the Ville de Montr?al 470 2012 Annual Report 5.12. "Time Management" Subproject 5.12. "Time Management" Subproject 1. Introduction Time management (TM) was already a concern in 2000, when the Ville de Montr?al (the city) issued a first call for tenders for the purchase of a computerized work attendance management and data collection system. Thereafter, in 2004, as part of the SIMON (Syst?me Int?gr? Montr?al) program adopted by the executive committee on October 8, 2003, the city launched the Integrated HR-Payroll MS project, which was aimed mainly at providing the city with an integrated management system to support human resources and payroll management processes based on the best practices of large organizations. The processes included in the scope of the initial Integrated HR-Payroll MS project included the TM process, hereinafter referred to as the "TM subproject." In July 2006, a moratorium was announced for the SIMON program and all work on the HRPayroll MS project was interrupted. By then, the city had purchased the Kronos software package after analyses had shown that the Oracle Suite did not provide the necessary functions to enable the city to manage its TM process efficiently. At the time, work on the TM subproject was under way only in the Lachine and LaSalle boroughs, the wastewater treatment plant and the Centre d'urgence 911. In April 2007, work on the TM subproject resumed under the responsibility of the Service des finances, which separated the subproject from the SIMON program. At that time, the purpose of the project was to [TRANSLATION] "provide the city with a single time management system to enable all boroughs and city departments to manage work attendance, work schedules, time banks and time transfers between cost centres locally in order to gather payroll information while managing its time management process more efficiently 1 ." Meanwhile, the TM subproject continued and, at the end of our audit work, the subproject's two phases had progressed as illustrated in Table 1. 1 Project organization manual, Gestion de temps - Kronos, phase 1, May 20, 2007. Auditor General of the Ville de Montr?al 471 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Table 1 - TM Subproject Progress Phase 1. Pilot deployment for certain groups of employees in the Lachine, LaSalle, Ville-Marie and Mercier-HochelagaMaisonneuve boroughs as well as the wastewater treatment plant and the Centre d'urgence 911 2. Deployment for all other employee groups apart from police officers and firefightersb a b Scheduled date Actual date February 2008 May 2008a May 2009 December 2012 Partially completed and transferred to phase 2. Originally, police officers and firefighters were included in phase 2. At the end of our audit work, we were informed that the Kronos software package would be deployed for firefighters during the first half of 2013. As for the police officers, no deployment date had yet been set. 2. Audit Scope Initially, our audit dealt only with the work carried out under the TM subproject. However, considering that the TM subproject is based on the Integrated HR-Payroll MS project and that--following the moratorium announced in July 2006--the revival of this subproject remained both uncertain and challenging at the time of our audit, we decided to extend the scope of our audit to include the project's then current status. Since the scope of analyzing the status of the Integrated HR-Payroll MS project is in no way comparable to conducting an analysis of the TM subproject's status, we decided to tackle each of these analyses in separate audit reports. This audit report therefore addresses only the TM subproject. In this regard, given the delays observed with respect to the implementation of the Kronos software package, we set the following objectives for our audit: o assess the current management of the TM subproject regarding the implementation of an efficient TM system; o take stock of current risks with respect to business continuity, compliance remunerated TM framework and the efficient management of related operations. We conducted most of our audit work between October 2011 and June 2012, and our audit included a global review of the key aspects of the TM subproject since its launch in 2005. However, we updated certain data inherent to the level of advancement of the TM subproject as at December 31, 2012. Auditor General of the Ville de Montr?al 472 2012 Annual Report 5.12. "Time Management" Subproject Note that the choice of the Kronos software package to support TM process management was not included in the scope of our audit. Furthermore, we focused our work such as to evaluate whether or not the processes in place enabled the configuration of the Kronos software package in compliance with collective agreement provisions and other frameworks determining work conditions. Our intention was not to conduct an extensive evaluation of the level of compliance of the data thus configured. 3. Findings and Recommendations It must be pointed out that, at the time of our audit, the Service des technologies de l'information (STI) was in the process of implementing a new information technology project management framework. In principle, this framework should mitigate the risks of future projects being hindered by the same problems as those described in this audit report. We shall cover the application of this framework in upcoming information technology project audits. 3.1. "Time Management" Subproject Governance 3.1.A. Background and Findings A project organization that includes the required resources and an efficient decision-making structure must be combined with rigorous risk management procedures to ensure the project's successful outcome. Project Governance Structure Our audit work found many organizational changes made to the TM subproject's governance structure, those changes having stemmed from the Integrated HR-Payroll MS project. First the moratorium announced for the SIMON program in July 2006 had impacts on the monitoring of the TM subproject. At the time this moratorium was announced, the Service du capital humain was this subproject's proponent and was acting as the owner of the TM process. In April 2007, the Service des finances took over the TM subproject which it managed as both proponent and mandatary until June 2011. From that moment on, the STI took over as the subproject's mandatary, acting on behalf of the Service des finances. Second, our audit work found that the project organization manuals (POMs) for phases 1 and 2 simplified the governance structure initially developed for the Integrated HR-Payroll Auditor General of the Ville de Montr?al 473 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit MS project. We must emphasize that this structure made no provision for a coordination committee although such a committee was needed to create synergies between the project delivery teams and thus favour an efficient deployment of the TM subproject. Furthermore, we note that the steering committee's hierarchal structure does not endow members with the authority they need to intervene and make the necessary decisions to ensure that the TM subproject unfolds smoothly. During the completion of phases 1 and 2, which spanned from April 2007 to the beginning of 2011, the anticipated governance structure was modified several times as were the composition and roles and responsibilities of the steering committee and follow-up committee. Furthermore, the members of these committees met less and less often, to the point where the steering committee met only once in 2010 and 2011 and the follow-up committee held no meetings whatsoever during this period. The TM subproject participants who we met informed us that these governance gaps hindered the completion of the project in several ways and that the TM subproject was basically left unchecked during this period. However, during the course of 2011, a new governance structure was implemented for the TM subproject (see Figure 1) and a new project leader was appointed. Thereafter, we observed a gradual improvement of the situation. Figure 1 - ICT Project Governance Modela Steering committee Steering committee PGMbb/Project PGM /Project Pilot pr oject leader Chai r:r: Busi ness fi eld manager Chai Busi ness fi eld manager ST I m anager ST I m anager Rel ated fields m anager Rel ated fields m anager ICT proj ect leader ICT proj ect leader Proj ect/Pi lot l eader Proj ect/Pi lot l eader Corporate PCOc c Corporate PCO Cl ientele representati ve Cl ientele representati ve IC T pr oject leader Pil oti ng and im pl em entati on team Development team Business analy sis Planning and control ICT dev elopment Change management Infra structure Training Vendor Deployment a b c Information and communications technologies. Combined management model for the program and project. Project control officer. Auditor General of the Ville de Montr?al 474 2012 Annual Report 5.12. "Time Management" Subproject Process (or system) ownership is a universally accepted principle of sound management. In fact, the process or system owner plays an important role in the governance and completion of a project. The owner chairs the project's steering committee and specifically must: o justify the project and account for the realization of expected benefits; o authorize the implementation of the solution and organizational changes; o ensure that the project is planned such as to incorporate his or her business field's current activities. The TM process owner was never officially announced while the TM subproject was being carried out. Several arguments justify giving ownership of this sub-process to the Service du capital humain et des communications (SCHC). Indeed, the SCHC negotiates, states and enforces the rules pertaining to TM. Moreover, the SCHC played that role during the period preceding the revival of the TM subproject by the Service des finances in April 2007. However, the TM subproject has been managed by the Service des finances since it resumed in April 2007. Furthermore, the Kronos software package that supports the TM process contains parameters that are inherent to work conditions and that constitute the payroll process' main input. Consequently, the TM process is similar to a financial process and the Service des finances can admittedly claim ownership of it. Ultimately, the owner of the TM process must be officially designated in order to clear up the roles and responsibilities as well as resulting accountability issues. After this audit report was confirmed with the business units concerned, the Service des finances was appointed owner of the TM process. Risk Management and Accountability Although the POMs for phases 1 and 2 set out certain technological and organizational risks, no concrete measures were taken to mitigate them and these risks were never updated or integrated into a formal follow-up and accounting structure during the project's implementation. Moreover, the risks referred to in the phase 1 POM are the same as those referred to in the phase 2 POM. This established fact--combined with the deficiencies presented earlier in the chapter on governance--admittedly contributed to the loss of control over the TM subject's implementation timeline as well as to the difficulties experienced by the project team while searching for solutions to certain problems. Among other things, the failure to take into consideration from the outset the impacts of integrating the payroll systems of boroughs Auditor General of the Ville de Montr?al 475 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit created from former suburban municipalities into the city's system as well as the individual characteristics of their respective pension plans speaks volumes to this effect. 3.1.B. Recommendation We recommend that the Service des finances, owner of the process, and the mandatary of the "time management" subproject, namely the Service des technologies de l'information, identify their respective risk management and reporting responsibilities, integrate the subproject's current risks into a single dashboard, ensure systematic follow-up and report thereon to the steering committee and the Comit? sectoriel des technologies de l'information. Business units' responses: SERVICE DES FINANCES [TRANSLATION] At present, a monthly dashboard is sent to the Comit? sectoriel des technologies de l'information (CSTI); also a follow-up table for major projects is sent to the Comit? corporatif de gestion des projets d'envergure (CCGPE) and to the Comit? de monitoring des projets d'envergure (CMPE) by the Division r?alisation de projets TI (DRPTI) of the STI. The dashboard includes an updated risk assessment for the project. The dashboard is submitted for approval by the head of the steering committee of the Service des finances before being sent to the CSTI. Risks are systematically monitored to identify appropriate mitigation measures, which are indicated in the monthly dashboard. (Completed, April 2013) SERVICE DES TECHNOLOGIES DE L'INFORMATION [TRANSLATION] At present, a monthly dashboard is systematically sent to the CSTI; also a follow-up table for major projects is sent to the CCGPE and the CMPE by the DRPTI of the STI. The dashboard includes an updated risk assessment for the project. The dashboard is systematically submitted for approval by the head of the steering committee of the Service des finances before being sent to the CSTI. Risks are systematically monitored to identify appropriate mitigation measures, which are indicated in the monthly dashboard. (Completed, April 2013) Auditor General of the Ville de Montr?al 476 2012 Annual Report 5.12. "Time Management" Subproject 3.2. Development Phases 3.2.1. Implementation Strategy 3.2.1.A. Background and Findings The TM subproject implementation strategy was based on a gradual implementation schedule. Initially, the project was split up into two phases. Phase 1 provided among other things: o review of the architecture; o migration (seeing as the city was already using the Kronos package) of the solution, for the "blue collar" category only, to the Ville-Marie and Mercier-Hochelaga-Maisonneuve boroughs; o review of the analysis conducted to replace "GesHor" (the schedule management system currently used by the wastewater treatment plant) by Kronos; o review of the analysis conducted to replace the time recording function then handled by Kronos' Athena application; o implementation of Kronos, for the "blue collar" category only, in the LaSalle and Lachine boroughs; o implementation of Kronos, for the "permanent white collar" category only, in the Centre d'urgence 911 and the wastewater treatment plant; o training, deployment, support and acceptance of the deliverables for each borough and unit where the Kronos software package was implemented. Phase 1, which ran from April 2007 to February 2008, was meant to provide an evaluation of the total cost of implementing the Kronos software package throughout the city as well as the proof of concept and lessons learned for the implementation of the second phase. As for phase 2, the goal was to complete all of the deliverables and the implementation of the system for all city employees, with the exception of the firefighters and police officers. This second phase was scheduled from January 2008 to January 2009. The post-mortem conducted on phase 1 shed light on several major problems that had arisen during the development and implementation of the components making up the Kronos software package. Indeed, the post-mortem documents reveal that the deliverables were only partially completed, that work was not always completed according to schedules, that the total cost of the project had not been evaluated and that several planned programming operations had been postponed to phase 2. Furthermore, comments issued during the post-mortem testified to several deficiencies (e.g., the instability of the system, Auditor General of the Ville de Montr?al 477 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit last-minute modifications and duplicate data entries, which significantly increased the workload and delayed the implementation of the system). Moreover, the chosen deployment strategy--i.e., to transfer only some of the users within a same unit during phase 1 of the TM subproject--emphasized the difficulties encountered. Among other things, the strategy required a lot of training and support, practically as much as if all users had been converted to the new system. Maintaining the former TM systems for a portion of the employees also contributed to increasing the workload. In our opinion, this situation is primarily due to the aforementioned gaps in the governance mechanisms, with respect namely to project leadership and risk management, which made it impossible to efficiently handle contingencies. However, it must be noted that the corrective measures taken as of 2011 turned the situation around and that the implementation process was operating smoothly by the end of our audit work. 3.2.2. Preliminary and Functional Analyses 3.2.2.A. Background and Findings The preliminary analysis is expected to determine the processes and data for which gaps exist with respect to the software package's standard functions. A decision-making committee is then expected to determine the required administrative and operational changes to ensure that the processes comply with the package's basic functions and, if applicable, set out the mandatory changes required to the package's configuration. Although this is only a high-level summary evaluation, we were unable to track a single analysis conducted to determine the gaps between the TM processes in place and the standard functions of the Kronos software package for the TM subproject. A gap analysis is nevertheless particularly important in the case of complex projects that affect the entire organization, that are implemented within a decentralized organizational structure and that involve many linkages and dependencies. Moreover, the TM process is basically a very simple process consisting of recording the time worked by individuals, having it approved by their respective managers and transmitting the data to the payroll department. The documentation presents the time recording and schedule management sub-processes as repetitive processes that are required for payroll management yet add little value in terms of organizational management. Consequently, several administrative units must use add-on software packages to meet their needs, for management information in particular. These add-on packages required Auditor General of the Ville de Montr?al 478 2012 Annual Report 5.12. "Time Management" Subproject either an interface to connect them with the TM systems then in place or the repeated entry of the same data in the parallel and add-on systems. The Kronos software package was purchased to partially or fully address these requirements. However, no analysis or evaluation had confirmed the feasibility of this objective. At the time of our audit, a decision had not yet been made as to whether or not to use the Kronos package's modules to meet these needs. 3.2.2.B. Recommendation We recommend that the Service des finances, which owns the process, make a list of the gaps with respect to the project's initial objectives, the functions currently used, those under study and those that have been abandoned while taking into consideration the impacts of these choices. We also recommend that the Service des finances report on these items to the Comit? sectoriel des technologies de l'information to enable the latter to make the appropriate decision under the circumstances. Business unit's response: [TRANSLATION] The list of discrepancies has been completed and includes the following items: o List of parameterized Kronos functions and discrepancies with respect to the objectives of business units; o Customization of Kronos: - Customization, - Actions taken to harmonize with payroll operations, - Configuration of the interface output, - Contextual codes for the salaried employees' interface, - Contextual codes for the waged employees' interface; o List of further improvements to be made. (Completed, March 2013) 3.2.3. Acquisition of Licenses 3.2.3.A. Background and Findings According to the Aberdeen Group, an international study group, software licensing is an expense that represents between 10% and 30% of a project's total cost and is the second highest expense, after labour, incurred to implement IT projects. Furthermore, Aberdeen states that license holders must pay recurring annual software maintenance and support fees that may represent anywhere from 10% to 25% of license costs. Auditor General of the Ville de Montr?al 479 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit The following table provides a summary of the number of licenses purchased and used as part of the Integrated HR-Payroll MS project and the TM subproject. Table 2 - Licenses Purchased and Used Year Number of licenses purchased Number of licenses used Ratio (used/purchased) Oracle HR-Payroll 2008 2012 Kronos 2008 2012 254,500 280,500 177,500 179,170 110,100 43.3% 150,500 53.7% 0 0.0% 89,376 49.9% Most of the Oracle HRMS Payroll suite licenses were purchased in 2003 and a majority of the licenses used with the TM subproject's Kronos modules were acquired in 2004. A study conducted by the STI on the impacts of the July 2006 moratorium establishes the annual recurring fees paid for license support at approximately $2 million. According to the initial schedule determined for the implementation of the HR-Payroll MS project, all processes were to be completed by May 31, 2007. Given the project's current status, it is difficult to evaluate the number of licenses that will have been actually used, especially if certain modules are abandoned or replaced by new versions. The decision to purchase such a large number of licenses at the outset of the project had an important financial impact on the project's cost. In our opinion, such a decision must be re-examined for future projects. 3.2.3.B. Recommendation We recommend that the Service des technologies de l'information, in collaboration with the Direction de l'approvisionnement, assess the relevance of using a progressive acquisition strategy that takes changing needs into consideration in the calls for tenders it issues for the purchase of software package licenses to optimize acquisition and maintenance costs. Business unit's response: [TRANSLATION] Several actions have been taken: o This recommendation was integrated into the project management framework process under the supervision of STI's Section soutien aux projets to ensure that it is taken into consideration during project phases 2, 3 and 4; o The head of the DRPTI sent a message to the manager of the Direction de l'approvisionnement requesting an evaluation of the relevance of a progressive Auditor General of the Ville de Montr?al 480 2012 Annual Report 5.12. "Time Management" Subproject o o acquisition strategy for every call for tenders in all projects requiring the acquisition of a commercial software package or commercial software; The head of the DRPTI sent a message to all the section heads of the DRPTI requesting that they assess the relevance of a progressive acquisition strategy for every call for tenders in all projects; The manager of the STI sent a similar message to all members of the management committee and to the CSTI. (Completed, April 2013) 3.2.4. Parameterization and Data Conversion 3.2.4.A. Background and Findings The parameterization and configuration of the Kronos package's TM processes were to be carried out in compliance with the applicable collective agreements and employment contracts. Incidentally, in the subproject documents, we were able to retrace the analyses used to determine business needs. These analyses detail the conditions and required time management processes in accordance with agreement provisions, letters of agreement and other official documents. The SCHC is responsible for the TM process framework. However, the responsibility for the daily management and application of the process is delegated to individual departments and boroughs. The executive committee also authorized the decentralization of certain borough responsibilities regarding human resources management (during its January 26, 2005 session, decision CE 05-0122). This decentralization increased the risks of TM activities being managed in a non-uniform manner. Incidentally, based on the information obtained, the TM conditions are not always interpreted and applied uniformly by the administrative units. Generally, we noted that several "local" agreements were approved by the upper management level of the administrative units concerned and that the specific TM conditions and schedules were thus parameterized. However, we were unable to reasonably assess whether or not these specific conditions and schedules were compliant with the provisions of the collective agreements and other frameworks given the absence of comprehensive lists in that regard. Besides the fact that these "local" agreements increased the amount of programming and testing needed to configure the Kronos software package, we also noted that the configuration conditions had been parameterized without necessarily having been validated in advance with the SCHC. Consequently, in our opinion, there remains a risk that the employment conditions parameterized in the Kronos package are not compliant with the collective agreements and related frameworks. Auditor General of the Ville de Montr?al 481 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.2.4.B. Recommendation We recommend that the Service du capital humain et des communications take the necessary measures to ensure that the collective agreements and letters of agreement were properly interpreted and integrated into the Kronos software package. Business unit's response: [TRANSLATION] The SCHC, in collaboration with Service des finances, is in the process of organizing the RH-paie project. Teams are currently planning the work and identifying obligations under collective agreements, letters of agreement and current employee compensation practices in order to accurately determine project costs. When the current phase is completed in fall 2013, the project will be at the implementation stage, and the SCHC will take the opportunity to ensure that letters of agreement, collective agreements and current practices are properly applied and interpreted. A full review will be conducted before programming begins. (Planned completion: December 2014) 3.2.5. Management of Requests for Changes 3.2.5.A. Background and Findings The organization must implement a process for entering, documenting, evaluating, authorizing and managing requests for changes in accordance with project objectives. Also, it must be possible to track the status of each individual request for changes at each step of the management process. We noted that there was no formal process in place to manage requests for changes. In effect, each group (operational and IT) has its own ways of doing things. The lack of such a process leads to a disorderly management of requests for changes and inevitably increases the implementation teams' workloads as well as the risks of processing requests for changes that are not necessary, urgent or compliant or again of not processing other requests. Moreover, we noted that a few changes that were less urgent, including certain customizations, had been made. Furthermore, certain changes had not been properly documented or did not seem to have been approved at the different stages of the management cycle. Also, it seemed to us that the users were not always informed of the changes made at the implementation stage. We also noted the lack of a global registry of Auditor General of the Ville de Montr?al 482 2012 Annual Report 5.12. "Time Management" Subproject requests for changes and their respective statuses to ensure that all interventions were rigorously tracked. However, we must emphasize that, at the time of our audit, the STI was preparing to implement a framework for requests for changes based on the Macroscope methodology. 3.2.5.B. Recommendation We recommend that the Service des technologies de l'information take the necessary measures to implement, as quickly as possible, a rigorous process for managing requests for changes. Business unit's response: [TRANSLATION] A registry of requests for changes was created by the project team in 2012. Every request for changes received by email and accompanied by a completed change request form approved by the functional team is automatically added to the registry. Any request not meeting these three criteria is returned to the sender with instructions to complete the request as just described. Every month, the registry is systematically presented to the project coordination committee, which comprises the head of the steering committee, the head of the Division de la paie institutionnelle and members of the Service des finances functional team. This group must approve the registry of requests for changes and the assigned priority levels. Project deliverables are followed up in the same way. Since January 2013, a representative of the operational team has been attending meetings of the coordination committee so that follow-up will continue once the Kronos software package has been transferred to the operational team. (Completed, April 2013) 3.2.5.C. Recommendation We recommend that the mandatary of the "time management" subproject, i.e., the Service des technologies de l'information, set up a global registry of requests for changes, verify that all required authorizations were obtained and review the priorities of the changes under way. Auditor General of the Ville de Montr?al 483 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Business unit's response: [TRANSLATION] Given that the project is nearly complete, the registry comprises a list of requests for changes that have been prioritized by representatives of the Service des finances (in accordance with the evaluation criteria they prescribed on the form). The coordination committee therefore authorized the representatives of the Service des finances to reprioritize requests for changes, should the need arise. (Completed, April 2013) 3.2.6. Tests 3.2.6.A. Background and Findings The "functional tests" must be carried out by expert users in an organized environment and in accordance with a specific strategy. Processes must have been determined for each function such as to limit the extent of each test under a plan and a script 2 . IT personnel needs to provide support and coaching to the functional resources to ensure test efficiency and quality. Finally, the functional team's expert users are those who have the ability to evaluate the expected results and authorize productions. We noted the absence of a global strategy including the nature, scope and number of tests to be carried out. Furthermore, despite a very high number of requests for changes, faults and other modifications, we did not have access to a comprehensive list of them given the absence of a centralized registry. Based on the information obtained, we noted that productions were not always carried out in accordance with the best practices. For example, at the time of upgrading to version 5.1, the migration of the Kronos software package's components illustrates the testing process' shortcomings. In effect, this implementation had not been validated through functional tests, which would have made it possible--in this case--to confirm that the upgrade did not modify the processes. The implementation of this version caused errors in the calculation of time banks as at May 1, 2011 and ended up requiring a very substantial number of manual corrections. Our audit of the tests conducted by the functional team for a few requests for changes confirmed that the validations are appropriate despite the fact that the strategy and scripts are still not very well documented. However, we must emphasize that it was not always possible to retrace the authorizations for the production and follow-up of certain changes. We must also emphasize that an individual acting independently from the development 2 A test script is a detailed procedure for fully testing a function or an aspect of a function. The test plan describes what needs to be tested, whereas the test script describes how to conduct a particular test. Auditor General of the Ville de Montr?al 484 2012 Annual Report 5.12. "Time Management" Subproject must handle the transfer of components in production but that the person in charge of the functional team is responsible for the final results and must approve all implementations. We also wish to emphasize that several major IT changes were made after the Kronos software package had been implemented. These changes modified how the TM process operated. With respect to the best practices for a major IT project, it is recommended to have each component fully developed as close as possible to its final version before implementation in order to limit the changes to the production environment, since this practice increases risks considerably. 3.2.6.B. Recommendation We recommend that the mandatary of the "time management" subproject, i.e., the Service des technologies de l'information, take the necessary measures to: o develop a global strategy for carrying out tests as part of the implementation of the subsequent phases of the "time management" subproject; o ensure that the test management tools are applied (test script template among others). Business unit's response: [TRANSLATION] A global strategy will be developed for conducting tests, including follow-up. The strategy will be communicated to the principle stakeholders (Planned completion: May 2013) The strategy, including follow-up testing, will be implemented when the project is transferred to the operational team. (Planned completion: June 2013) 3.3. Security 3.3.1. Environment Management 3.3.1.A. Background and Findings The architecture of the IT environments must separate functions from tasks to ensure data integrity and accuracy in each environment. The best IT environment management practices suggest creating different environments for specific objectives. Auditor General of the Ville de Montr?al 485 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit For the TM subproject, the STI implemented an environmental architecture that meets generally recognized standards and the need to segregate functions. The environments must be accessible to certain groups according to each group's roles and responsibilities, and the transfer of IT components must be controlled to ensure data accuracy and integrity. However, in fact, we noted that components were generally transferred between environments by the development personnel who had full control over them, including those of the production environment, without any control over accesses, follow-ups or tracking of the interventions carried out. Moreover, the IT components must be developed, modified and tested in the development environment. They must then be transferred to the acceptance test environment as the project progresses. Once the acceptance tests have been approved, the components are transferred to the preproduction environment for compliance testing. Finally, once validated and approved by the expert users, the components must be transferred to the production environment. To this end, we noted certain gaps in the management of these environments and transfers. IT components are transferred from the functional testing environment to the production environment without systematically passing through the preproduction environment and without any validation by the expert users. Although the STI now validates several of these components in the preproduction environment, the expert users are ultimately responsible for verifying changes and authorizing transfers, particularly in the production environment. We also noted that the functional team is required to re-enter in the production environment specific parameters such as schedules because these parameters are not transferred to the production environment. This is not a recommended practice because not only does it require the duplicate entry of the parameters, but also these parameters are recorded and tested in the development environment, which increases the risk of errors. This task should be automated using tools such as Workforce Record Manager. Moreover, the Kronos software package is installed in an environment based on the Windows Server operating system. The project team produces documents dealing with the architecture, and the Kronos user guides in these environments (e.g., development, production) are well documented. Auditor General of the Ville de Montr?al 486 2012 Annual Report 5.12. "Time Management" Subproject This environment's security is managed--in accordance with the STI's practices--through policies and group strategies 3 in the environment of the city's Service d'annuaire informatique. We noted that most of the controls were present and that the processes were generally documented. However, we also noted the lack of an audit strategy for these environments, specifically the production environment. Such a strategy would make it possible to keep traces of risky interventions carried out on TM data or related information. 3.3.1.B. Recommendation We recommend that the Service des technologies de l'information take the necessary measures to: o control transfers of IT components between the environments; o implement the necessary tools to ensure data integrity during IT component transfers between the environments. Business unit's response: [TRANSLATION] At present, risk is managed by an internal procedure of the project team. The person (consultant) who transfers program components to the production environment is not the same person who developed them; this was done by a programmer analyst. A transfer document is systematically developed, updated and archived. Recognizing the problem, we developed and implemented a new acceptance environment. (Completed, April 2013) We are in the process of developing a procedure to ensure that all development and parameterization is systematically followed by mandatory transfer to an acceptance environment. To achieve this, the acceptance procedure that takes place in the functional team's environment will have to be reintegrated into the new environment. In this way, the acceptance process will be independent from development. We are also currently developing a procedure to set up Workforce Record Manager, which will be used for transfers to the production environment when the project is transferred to the operational team. (Planned completion: June 2013) 3 The "group strategies" (or GPOs [Group Policy Objects]) are centralized management functions available under the Microsoft Windows family. These functions are used to manage workstations and users in an Active Directory environment. Auditor General of the Ville de Montr?al 487 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.3.1.C. Recommendation We recommend that the Service des finances, which owns the process, in collaboration with the Service des technologies de l'information, as part of the "time management" subproject, determine the potential risks in order to design and implement an audit strategy accordingly. Business unit's response: [TRANSLATION] Determine potential risks and implement an audit strategy. (Planned completion: December 2013) 3.3.2. Access Management 3.3.2.A. Background and Findings Two main strategies can be used to configure access security in the Kronos software package. The first involves using the package's access management and access parameters to create an "owner"-type security. The Kronos software package was implemented with an owner-type security structure that is therefore subject to the package's configuration. The second strategy provides control using a gateway based on the LDAP 4 which the city already uses. This strategy offers several advantages, the most important being the consolidation of user information in a single registry for several systems. However, it requires that all resources be identified in the directory. During our audit, the person responsible for this activity tested the LDAP-based approach with a target group to assess the possibility of implementing this strategy for all users. However, there were no orientations or directives provided for the implementation of this strategy in the case of the Kronos software package. Application security in the Kronos software package was designed by the TM subproject's functional team by adapting the system's basic roles and responsibilities with respect to the city's situation in order to determine access profiles. Documents on the TM subproject management provided for the possibility of assigning up to 28 different access profiles to users, but the functional team reduced this number to 19, some of which are specific and temporary during the implementation of the Kronos software package. We noted that the functional team ensures regular follow-up of the most critical profiles. However, we also noted certain situations where the best practices were not applied. 4 Lightweight Directory Access Protocol. Originally, the LDAP was a protocol used to query and modify directory services. Auditor General of the Ville de Montr?al 488 2012 Annual Report 5.12. "Time Management" Subproject Indeed, the best practices recommend that high-privilege codes and profiles giving access to production environment data be assigned only to a limited number of users, that interventions using these codes be recorded in a restricted-access file and that the personnel use these privileges only as necessary to ensure data integrity, security and accuracy. The "SuperAccess" access profile that was configured in the Kronos software package authorizes unrestricted write access on all work attendance data. As with other user codes, actions leave traces, but these traces can also be modified or destroyed. We noted that this profile was assigned to seven users: four among the IT personnel and three in the functional team. In the case of this project, the number of users who have a privileged access profile appears high to us. Furthermore, the interventions made with this access profile are not documented and do not always leave traces--thus making it difficult to evaluate its relevance. The owner-type approach has impacts on security management. Under this approach, a user may have several user codes with different access rights (profiles) since codes are managed and named manually without being directly validated by the city's environment. The persons responsible for security management in the Kronos project standardized the user code format by aligning it with the city's standardized user code nomenclature. However, four members of the STI's implementation team also have another generic "administrator" user code with a "SuperAccess" profile. This is contrary to the best practices that proscribe the use of generic codes. 3.3.2.B. Recommendation We recommend that the Service des finances, which owns the process, in collaboration with the Service des technologies de l'information, as part of the "time management" subproject and once the deployment has been completed (postimplementation): o review and justify the assignment of the "SuperAccess" profile to certain users while giving consideration to the available controls; o validate the existing profiles and deactivate or destroy all access profiles that are not used or required; o prepare an action plan and an implementation schedule for using the Lightweight Directory Access Protocol directory to control access to the Kronos software package. Business unit's response: [TRANSLATION] The review of access and security profiles, as well as of the assignment of the "SuperAccess" profile, is under way. Security profiles and access Auditor General of the Ville de Montr?al 489 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit to Kronos will be controlled and monitored in accordance with the recommendations contained in the report of the Bureau du v?rificateur g?n?ral on the protection of personal information. (Planned completion: June 2013) 3.3.3. Certification Deliverables 3.3.3.A. Background and Findings In accordance with the city's applicable security frameworks, a software package that supports a business process such as TM must be independently certified by the STI's Information security and best practices section. At the time of our audit, we noted that the certification deliverables inherent to the Kronos software package had not been obtained for the version in production. Although these deliverables had been completed in 2008, the scope of the Kronos package deployment had been considerably widened since that time and several changes had been made to the package. Under the circumstances and considering the very nature of the Kronos software package, in our opinion, it is imperative to update the security certification deliverables. To this effect, we must emphasize the importance of protecting the ownership of the software package in trust, which had not been done initially. It is important to note that, in accordance with the frameworks, the project leader is required to control the filing of all deliverables and ensure that the security certifications of all new software packages installed in the city's environments have been filed. 3.3.3.B. Recommendation We recommend that the Service des finances, which owns the process, take all necessary measures to obtain the required deliverables for the Kronos software package's security certification. Business unit's response: [TRANSLATION] Certification is under way. The report has been prepared and is in the acceptance phase with the Service des finances. (Completed, April 2013) 3.4. Performance 3.4.A. Background and Findings The Kronos software package experienced major performance problems during phase 2, following its implementation in the boroughs and central departments. Version 5.1 of the Auditor General of the Ville de Montr?al 490 2012 Annual Report 5.12. "Time Management" Subproject software package was implemented by the STI but its deployment was compromised by performance issues. To determine the root causes of the problem and find solutions, the city requested the help of a Kronos specialist to carry out a technical study titled "Workforce Central Data Analysis & Empirical Sizing". The data analysis conducted as part as this study did not detect any specific problem but suggested several improvements. Finally, and according to the STI, migrating the Kronos package from version 5.1 to version 6.1 as well as increasing the servers' data processing capabilities solved the system performance problem. We also noted that certain improvements were made, including with load balancing and hyperfind query processing. However, several users we consulted pointed out to us that they still experience slowdowns and performance issues with the Kronos software package, specifically during payroll closure periods. Since no new performance data were available, we asked the project team--which supervises the performance of the database and Web servers--if problems had been experienced. According to the team's evaluations, the Kronos software package is not plagued by performance problems despite what users perceive. However, these evaluations are based only on the number of database prompts whereas other factors may have caused the performance problems. Given that the number of users integrated in the software package increased substantially, the increased use of the hyperfind function and the comments made by the users we questioned, it would be relevant to analyse and track the system's performance based on other metrics. It is important to note that the study pointed out potential problems with using the hyperfind function among other things. This function is used to enable wide-ranging queries throughout the database and operates by extracting a "view" 5 that recalls several data structures in memory. Each hyperfind query can recall data from several database tables to provide the necessary information to answer the query. The processing of the query may be slowed down if many users call up this function concurrently. 5 A view is a virtual table, i.e., it refers to the data that are not stored in a database table and in which data originating from several tables can be consolidated. Auditor General of the Ville de Montr?al 491 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.4.B. Recommendation We recommend that the Service des finances, which owns the process, in collaboration with the Service des technologies de l'information, as part of the "time management" subproject: o determine the norms and standards that would be acceptable with respect to response times in different Kronos software package use situations; o implement performance indicator tracking measures for all Kronos software package components; o assess the need to oversee the hyperfind queries to reduce the load on the Kronos software package and improve its performance levels; o evaluate the possibility of creating a data warehouse for time management and human resources data to reduce the load on the system during queries. Business unit's response: [TRANSLATION] Actions to be taken: o Determine norms and standards; o Implement indicator tracking measures; o Oversee hyperfind queries in order to increase effectiveness and efficiency; o Assess the possibility of creating a data warehouse for Kronos. These actions will be taken in collaboration with STI's Section informatique. (Planned completion: December 2013) 3.5. Evolution of the Kronos Platform 3.5.A. Background and Findings The recognized IT management reference frameworks state the importance of implementing an ongoing support, change management or improvement process for existing IT platforms. To this effect, the ITIL 6 reference framework quotes a study conducted by IBM, which found that: o 80% of incidents are detected by users; o poorly controlled changes are responsible for 85% of incidents; o users inevitably end up forming the testing team in lieu of the information technology unit; o such a situation leads to a loss of operational efficiency and higher costs for the organization. 6 Information Technology Infrastructure Library. Auditor General of the Ville de Montr?al 492 2012 Annual Report 5.12. "Time Management" Subproject We noted planning deficiencies with respect to the implementation of a management and evolution process for the new Kronos software package upon completion of the TM subproject. Such a process and the assignment of sufficient resources are essential to ensure the efficient operation of the TM system on an ongoing basis and more particularly at the time the software package or its database is updated or migrated. The resources assigned to this process may carry out the required changes in a secure and controlled change management environment and thus ensure the uniformity and compliance of processes and data after the completion of the project. The lack of a management and evolution process could have caused among other things non-compliant changes stemming from a lack of expertise, database irregularities or inconsistencies and system interruptions. Consequently, correcting the situation could be very costly. 3.5.B. Recommendation We recommend that the Service des finances, which owns the process, in collaboration with the Service des technologies de l'information, take the necessary measures to implement an ongoing support and evolution process for the Kronos software package. Business unit's response: [TRANSLATION] Processes and activities regarding payroll operations are already in place for the support and evolution of the IBM and Oracle systems. In June 2013, STI's Section informatique will add Kronos to the portfolio of software requiring ongoing support and evolution. (Planned completion: June 2013) 3.6. Continuity Plan 3.6.A. Background and Findings The organization's continuity plan records all risks and afferent impacts and should include availability and continuity plans that are specific to each applicative system operated by the organization. This plan should be used as a tool for determining the securement strategy required to counter or mitigate the consequences of the event represented by a risk. The securement strategy involves the use of the necessary preventive and curative measures as well as the participation of several departments to ensure service continuity. The TM system (Kronos) operates 24 hours a day, 7 days a week. It drives the payroll system--considered critical. Consequently, the TM system must also be considered a Auditor General of the Ville de Montr?al 493 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit critical system. We must emphasize that updating the cumulative time and time bank data for the purpose of closing payroll according to a specific frequency leaves little time for preparing the payroll. This situation is conducive to increasing risk levels. Currently, the Section s?curit? de l'information et meilleures pratiques evaluates the criticality and assigns up to 24 hours of tolerance following an interruption of the TM system regardless of the particular date or period. System availability is ensured using several measures, devices or other means designed to provide a certain level of robustness for time clocks, the system's applicative component and its database. According to the different STI units that manage components that are critical to system operations, in principle, a component may be recovered within a rather short period of less than five hours or within 24 hours in the case of major problems. However, we must emphasize that this assertion was never validated as part of a simulated recovery. In an extreme situation assuming the unavailability of time data, one of the contemplated possibilities suggests processing payroll on the basis of a previous pay period and then making corrections. However, the efforts or repercussions involved in such a strategy have not been evaluated. The STI also pointed out to us that five applicative servers are currently available in load balancing mode. The servers are housed in two different locations and the interruption of one server would not lead to the interruption of basic services to the users. However, the fact that an important application called "Connect" is installed on only one of the servers is a weak point. This could cause TM service recovery delays. Despite all of the redundancy and continuity recovery means that were implemented, we note that the absence of an overall plan and appropriate documents coupled with the lack of certain key resources could increase delays and lead to problems, mainly during the payroll closing period when the time constraints are very tight. Auditor General of the Ville de Montr?al 494 2012 Annual Report 5.12. "Time Management" Subproject 3.6.B. Recommendation We recommend that the Service des finances, which owns the process, in collaboration with the Service des technologies de l'information, as part of the "time management" subproject: o determine a formal continuity plan that is both concerted and efficient; o conduct periodic continuity tests to validate the necessary system recovery time and detect gaps that need to be corrected, including adding the "Connect" component to the other servers. Business unit's response: [TRANSLATION] Development and implementation of an efficient continuity plan in collaboration with the STI. (Planned completion: June 2013) Auditor General of the Ville de Montr?al 495 2012 Annual Report Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 Protection of Personal Information 5.13 5.13. Protection of Personal Information Table of Contents 1. Introduction ................................................................................................................... 501 2. Audit Scope................................................................................................................... 502 3. Findings and Recommendations .................................................................................. 504 3.1. Personal Information Present in Information System Environments Other than Production ..................................................................................................... 506 3.2. Security Parameters of Non-Configured Passwords............................................. 509 3.3. Discrepancy in the Review Process of Users and their Access Rights................. 510 3.4. Missing or Incomplete Access Management Procedures ..................................... 512 3.5. Discrepancies in the Physical Security of the Premises Housing Pay Records................................................................................................................. 513 Auditor General of the Ville de Montr?al 499 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit List of Acronyms CAI Commission d'acc?s ? l'information PI personal information 500 social insurance number STI Auditor General of the Ville de Montr?al SIN Service des technologies de l'information 2012 Annual Report 5.13. Protection of Personal Information 5.13. Protection of Personal Information 1. Introduction With more than 1.6 million residents and 28,000 employees, Ville de Montr?al (the city) collects and processes a considerable amount of information concerning the private life of its citizens, elected officials and employees. This information is necessary in order for the city to adequately serve the public. These activities may include, for example, processing requests from citizens (through the 311 service or borough offices) and managing employees. In Canada, privacy is a fundamental right that is protected in a comprehensive manner by federal and provincial laws. Adopted on June 27, 1975, Qu?bec's Charter of Human Rights and Freedoms makes the right to privacy an intrinsic right. The charter states, among others, the following intrinsic rights and freedoms of citizens: o the right to the safeguard of dignity, honour and reputation; o the right to respect of private life; o the right to respect of professional secrecy. A true North American pioneer in the area of access to information and protection of privacy, Qu?bec has built, over the past three decades, a legislative model that has paved the way for the implementation of similar measures throughout Canada. Personified by Qu?bec's Commission d'acc?s ? l'information (CAI), this unique model is an essential reference source for all western countries in the area of access to information and protection of private life. The CAI applies two acts: o For the public sector: the Act respecting access to documents held by public bodies and the protection of personal information 1 ; and o For the private sector: the Act respecting the protection of personal information in the private sector 2 . 1 2 RSQ, chapter A-2.1. RSQ, chapter P-39.1. Auditor General of the Ville de Montr?al 501 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit As a municipality, the city is subject to the Act respecting access to documents held by public bodies and the protection of personal information. This Act sets out two intrinsic rights: the right of access and the right of protection of personal information (PI). The Act applies to all documents whether they are recorded in writing or print, on sound tape or film, in computerized form, or otherwise. PI is defined as information that: o identifies a natural person (as opposed to a corporate body); o helps identify an individual (as opposed to depersonalized information); o is factual or subjective about a person regardless of its form or support, whether recorded or not. Given its nature, PI is confidential. The theft or disclosure of PI is often used for fraudulent purposes that can go as far as identity theft and an attack on an individual's reputation. The most sensitive types of PI are, among others, a person's: o social insurance number (SIN); o health insurance number; o date of birth; o salary and income tax statements; o banking information; o medical records; o r?sum?. As with any other information of a sensitive nature, the city must ensure the confidentiality of PI. To do so, it must put in place security measures to protect PI from theft, disclosure and unauthorized use. 2. Audit Scope The objective of our audit was to evaluate the effectiveness of the controls put in place to ensure adequate software and physical security of the PI of citizens and employees held by the city, with the exception of information related to the Service de police de la Ville de Montr?al. To this end, we investigated the following aspects: o PI management frameworks: Does the city have frameworks that define the requirements of sound PI management and that are applied to all business units? Auditor General of the Ville de Montr?al 502 2012 Annual Report 5.13. Protection of Personal Information o o o o o o o o PI inventory: Is there a complete and updated PI inventory that enables the city to draw an overall picture of the PI to be protected in order to ensure its confidentiality? Employee education: Are employees aware of the issues and risks related to PI so that they are more able to comply with the security rules regarding the protection of PI? Incident management: If a major event were to occur that could result in the mass disclosure of PI, is there an incident management procedure in place that would enable the city to react in a timely manner to limit real and potential repercussions and to take the necessary measures to resolve the incident? Logical access: Are security parameters (e.g., passwords and rights of access) configured so that only authorized individuals whose work requires them to use PI can access the information systems in which PI is processed? Physical access: Are there containment mechanisms, such as a secured vault or locked filing cabinets, to restrict access to authorized persons only to PI that is on physical supports (e.g., medical records and employee records)? PI retention: Is the PI housed solely on the production environments of information systems that are used daily by employees and managers? Instead of real PI, is depersonalized PI used in test, development and training environments to limit the risks of a breach of confidentiality? PI transmission: Is PI that is sent to third parties (e.g., the CSST, the Minist?re du Revenu) protected by security measures to safeguard the confidentiality of the information that is transferred? PI destruction: Is PI depersonalized or destroyed in such a way that it cannot be reconstituted to prevent any fraudulent use? We audited the following files since they held important and sensitive PI: o about citizens: - application files, - requests for services and complaints, - registration in recreational activities, - requests for renovation subsidies; o about employees, elected officials, judges, commissioners and retirees: - employee records, - medical records, - pay records, - pension plan records. At the same time, we selected the following administrative units for our audit of their PI management responsibilities and the volume of information they held and processed: o the Direction du greffe; Auditor General of the Ville de Montr?al 503 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit o o o o o o o the Service du capital humain et des communications; the Service des technologies de l'information (STI); the Direction des services regroup?s aux arrondissements, which reports to the Service de la concertation des arrondissements et des ressources mat?rielles; the Division de la paie institutionnelle, which reports to the Direction de la comptabilit? et du contr?le financier of the Service des finances; the Division de la gestion des rentes, which reports to the Direction de la gestion financi?re of the Service des finances; the Division de la gestion des programmes de logement, which reports to the Direction de l'habitation of the Service de la mise en valeur du territoire; the Division des ressources humaines: - of the Service de s?curit? incendie de Montr?al, - of Saint-Laurent borough, - of Montr?al-Nord borough, - of C?te-des-Neiges-Notre-Dame-de-Gr?ce borough, - of Villeray-Saint-Michel-Parc-Extension borough. Following is the list of information systems that process the PI covered during our audit: o SIMON RH (application files, basic employee records); o SIMON PAIE (the pay of judges, elected officials, commissioners and retirees); o Employeur D (medical records); o Pay (IBM); o GDC (the management of citizens' requests); o Super H (employee records); o InfoRH (data warehouse of the Service du capital humain et des communications); o Registre des postes (employee positions and salaries); o ARIEL (pension plans); o Ludik (recreational activities); o SDSR (requests for renovation subsidies). It should be noted that our audit may, in no way, be construed as a mandate to attest to the level of compliance of the city with the Act respecting access to documents held by public bodies and the protection of personal information. 3. Findings and Recommendations Overall, our audit did not reveal any major deficiencies in the control mechanisms put in place for the protection of PI held and processed by the city. Auditor General of the Ville de Montr?al 504 2012 Annual Report 5.13. Protection of Personal Information Table 1 presents the overall results of our audit based on the identified areas of risk. Table 1 - Overall Results According to the Areas of Risks Inherent riska Areas of risks PI management frameworks Disclosure of PI due to the absence of frameworks that define specific responsibilities and requirements related to protection of PI. PI inventory Breach of confidentiality of certain PI that was improperly determined during the PI inventory procedures and that was not adequately protected. Employee education Disclosure of certain PI due to the lack of knowledge on the part of employees about the behaviour to adopt to safeguard and maintain the confidentiality of PI. Incident management Inability to address and resolve major problems in a timely manner, for example the mass disclosure of PI. Logical access Breach of confidentiality of PI in the wake of unauthorized access to information systems. Physical access Breach of confidentiality of certain PI in the wake of inadequate security measures applied to physical documents and files containing PI. PI retention Real PI is used for environments other than those of production and could be stolen or divulged. PI transmission Breach of confidentiality of PI after interception during transmission between information systems. PI destruction Reconstitution and disclosure of PI that was not destroyed in a secure and irreversible way. a b c Residual riskb Highc Low Moderate Low High Low High Low Critical Moderate Critical Low Critical High High Low Critical Low Gross risk, i.e. without taking into account control mechanisms. Exposure to risk after an evaluation of the control mechanisms in place. Refer to Table 2 Section 3.1 and the following sections detail the specific deficiencies found during our audit that require corrective action. We evaluated these deficiencies based on the impact levels presented in Table 2. Auditor General of the Ville de Montr?al 505 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Table 2 - Definitions of Impact Levels Impact levels Critical High Moderate Low Definition of impact levels Direct consequence on individual security, major impact on the reputation of individuals and the reputation of the city if the PI were divulged While there is no consequence on individual security due to the large volume of PI present, the breach of confidentiality of this information would cause major harm to the city's operations and reputation. The individuals could become victims of theft or identity theft. Due to the presence of certain PI, a breach of confidentiality of this information could cause moderate harm to the reputation and operations of the city. Repercussions would be negligible on the city's operations and services. Loss of trust in the city by citizens is unlikely 3.1. Personal Information Present in Information System Environments Other than Production 3.1.A. Background and Findings Information systems generally have several distinct environments. There is the production environment, which is used by employees in the course of their work and which contains real data that is required to meet business needs. Then there are the environments that are used for other purposes, for example: o development environments: these are used by information technology specialists to develop or improve the functionalities of applications; o test environments: these are used by groups of users and computer analysts to ensure that changes made to the applications function properly; o training environments: these enable employees to acquire the expertise needed to effectively use the information systems. In environments other than production, the use of real data is not necessary, especially if that data is confidential, as is the case with PI. There are no business needs to justify its use. Good industry practices recommend that dummy records be used in environments others than production. During our audit, we concluded that real PI was being copied, in whole or in part, from production environments to various test and development environments. As well, no systematic PI deletion procedure was being applied once the test or development work was completed. The information systems involved are listed below. Auditor General of the Ville de Montr?al 506 2012 Annual Report 5.13. Protection of Personal Information o o o o o o SIMON (SIMON RH and SIMON PAIE): SIMON is the city's ERP (enterprise resource planning) system. SIMON RH contains basic employee records, as well as application files. SIMON PAIE contains 14,500 pay records of elected officials, judges, commissioners and retirees. On average, a dozen environments are used for development and test purposes. Only two of these environments have a depersonalized SIN and date of birth to avoid associating them with real natural persons. The other environments contain copies, sometimes in full, of PI originating from the production environment, the most sensitive of which are, among others, the SIN, date of birth, earnings and banking information. Super H: This application contains all the employee records, as well as those of applicants, elected officials, judges, commissioners and retirees. The test environment uses real PI originating from the product environment, such as the SIN, date of birth, salary, address and home telephone number. InfoRH: This data storehouse of the Service du capital humain et des communications contains PI for all employees, as well as applicants, elected officials, judges, commissioners and retirees. The development environment contains real PI, including the SIN, date of birth, salary, address and home telephone number. Registre des postes: This application contains job information about city employees. Real PI is copied from the production environment to the test environment. Salaries, among other information, are found here. Employeur D: This system contains the medical records of employees, elected officials, judges and commissioners when medical problems arise. The test environment uses real PI originating from the production environment. This information includes, among other things, medical information, the SIN, health insurance number, date of birth and salary. Paie (IBM): This application manages the pay of most of the city's employees and contains PI related to salaries. The development and test environments contain extracts of real PI originating from the production environment. The types of PI are, among others, banking information, the SIN, earnings, home address, date of birth and income tax statements. We estimate that the impact level is high since the city is facing the following potential risks: in allowing the use of real PI outside of production environments, the PI of all employees, elected officials, judges, commissioners, retirees and applicants could be stolen and disclosed to unauthorized individuals. With such information, malicious individuals could commit fraudulent acts, such as theft and identity theft. In all cases, this would seriously harm the city's reputation. Auditor General of the Ville de Montr?al 507 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.1.B. Recommendation We recommend that the Service du capital humain et des communications, as well as the Division de la paie institutionnelle of the Service des finances, in collaboration with the Service des technologies de l'information, put in place procedures to black out real personal information (e.g., social insurance number, date of birth) from the data of environments other than production in the information systems that they own: o the Service du capital humain et des communications: - SIMON RH, - Super H, - InfoRH, - Registre des postes, - Employeur D ; o the Division de la paie institutionnelle: - Paie (IBM), - SIMON PAIE. Business units' responses: SERVICE DU CAPITAL HUMAIN ET DES COMMUNICATIONS [TRANSLATION] SIMON RH and InfoRH: We contacted the STI to have confidential data scrambled in all the environments other than production. (Completed, April 2013) Super H and Registre des postes: The STI has taken the necessary measures to scramble data in the "test" environment. Nevertheless, we are currently in contact with them to ensure that the information elements have been taken into consideration. (Completed, April 2013) Employeur D: The request has already been made to the STI to scramble confidential data. Moreover, given that this application contains data related to accidents and illnesses, we are in the process of reviewing the information that will be considered confidential. (Completed, March 2013) SERVICE DES FINANCES [TRANSLATION] A work order has been issued to STI to implement PI block-out procedures for the pay management systems. (Planned completion: December 2013) Auditor General of the Ville de Montr?al 508 2012 Annual Report 5.13. Protection of Personal Information 3.2. Security Parameters of Non-Configured Passwords 3.2.A. Background and Findings Passwords are the first line of defence in preventing unauthorized access to information systems containing sensitive and confidential data, such as PI. Password security parameters make it possible to require users to choose robust passwords. These parameters define, among, other things, the length of the password, its complexity, its expiry deadline, and the history of recent passwords According to the STI procedure entitled [TRANSLATION] "Standard respecting access keys to computer resources", password requirements include: o an expiry deadline: 90 days; o a password length: minimum of eight characters; o a history: six recent passwords; o activation of password complexity (e.g., combination of alphanumeric characters, special characters, upper and lower case letters) During our audit, we concluded that the password safety parameters had not been activated for the Employeur D application. As for Ludik, there is no safety parameter that can be activated. The only limitation on these applications is that the user must choose a password that contains at least one character. There is no expiry deadline or history of passwords. In the case of Employeur D, the password that is assigned to the user at the time access is created corresponds to his or her name, and the system does not require that it be changed when the user first logs on to the system. We estimate the impact level to be high since the city faces the following potential risks: because there is no security requirement for passwords, malicious individuals could easily uncover them. Consequently, in the case of Employeur D, these individuals would have access to the sensitive PI of employees, elected officials, judges and commissioners who have had health problems (e.g., the SIN, health insurance number, date of birth, salary, medical record) and, in the case of Ludik, to 800,000 files on citizens, 15,200 of which contain SIN and health insurance numbers. If such information were divulged, these individuals could commit identify theft and not only harm the city's reputation but also that of its employees, elected officials, judges and commissioners. Auditor General of the Ville de Montr?al 509 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 3.2.B. Recommendation We recommend that the Service du capital humain et des communications, which owns Employeur D and Ludik, in collaboration with the Service des technologies de l'information: o configure the security parameters of passwords with, at the very minimum, the following requirements: - a minimum length: eight characters, - an expiry deadline: 90 days, - a history: six last passwords, - activation of password complexity; o change all the current passwords as soon as possible, without waiting for the 90day expiry deadline, to comply with the new parameters; o require that new users change their initial password at the time they first log on to the system. Business unit's response: [TRANSLATION] In the case of the Employeur D application, measures were taken following the audit by the Bureau du v?rificateur g?n?ral in last December. The minimum length is now eight characters with at least two numbers. The expiry deadline has been configured to 90 days. An announcement was sent to all users informing them of these changes. All passwords have been changed. (Completed, December 2012) In the case of the Ludik software package, the STI is in discussions with the supplier to determine the best method to use in order to apply the recommendations and to limit the application developments of the software package. (Completed, April 2013 [corrective strategy]; planned completion: to come [implementation of the strategy]) 3.3. Discrepancy in the Review Process of Users and their Access Rights 3.3.A. Background and Findings To ensure adequate protection of PI, managing users' access rights must not only provide requirements and security mechanisms for the creation, deletion and modification of access but also include a review process of users' accounts. Auditor General of the Ville de Montr?al 510 2012 Annual Report 5.13. Protection of Personal Information A recurring review process ensures that all employees who have left the employ of the city or who have changed jobs do not keep their former access privileges. During our audit, we concluded that there was no formal review process of users' access rights to the following information systems: o GDC; o Employeur D; o SIMON RH; o Ludik. In the case of the Registre des postes, Super H and InfoRH information systems, a review of users' access rights is conducted yearly. In our opinion, given the critical nature of the PI, this frequency of review is insufficient. We estimate the impact level to be moderate since the city faces the following potential risks: individuals who have left their job with the city could keep access rights to information systems and those who have changed jobs could keep former access rights that no longer correspond to their new duties and responsibilities. This could lead to a breach of the confidentiality of the PI held by the city. 3.3.B. Recommendation We recommend that the Direction des services regroup?s aux arrondissements of the Service de la concertation des arrondissements et des resources mat?rielles and the Service du capital humain et des communications put in place a recurring review process (at least quarterly) of users' access rights to the information systems that they own: o the Direction des services regroup?s aux arrondissements: - GDC; o the Service du capital humain et des communications: - Employeur D, - SIMON RH, - Super H, - Registre des postes, - InfoRH, - Ludik. Auditor General of the Ville de Montr?al 511 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit Business units' responses: SERVICE DE LA CONCERTATION DES ARRONDISSEMENTS ET DES RESSOURCES MAT?RIELLES [TRANSLATION] A quarterly report will be extracted by the GDC system driver (Section de l'expertise et du soutien 311) and sent to expert users in the 19 boroughs for access follow-up. Everything will be documented and posted in an internal process at the Section de l'expertise et du soutien 311. (Planned completion: May 2013) SERVICE DU CAPITAL HUMAIN ET DES COMMUNICATIONS [TRANSLATION] A procedure will be put in place between now and the end of April 2013 to validate access once every quarter. (Completed, April 2013) In the case of the Ludik software package, a procedure is in the process of being drafted to validate access once every quarter. (Completed, April 2013) 3.4. Missing or Incomplete Access Management Procedures 3.4.A. Background and Findings Management procedures are important within an organization to ensure that the various services use the same modus operandi to perform activities that meet identified business risks and, more specifically within the framework of our audit, that reduce to an acceptable level the risks associated with users' access to information systems containing PI. For example, a procedure to manage users' access would include requirements to comply with the following elements: o Request for access; o Change to access; o Review of users and their access rights; o Deletion of access. During our audit, we noted the following discrepancies: o There is no access management procedure for the SDSR application; o The access management procedure for the Employeur D application is incomplete since it does not contain a concise description of the steps to be taken when requesting the creation of access. Auditor General of the Ville de Montr?al 512 2012 Annual Report 5.13. Protection of Personal Information We estimate the impact level to be moderate since the city faces the following potential risks: there are missing or incomplete procedures that could lead to non-compliance with security requirements regarding access management. As a result, unauthorized individuals could have access to PI housed on Employeur D and SDSR. 3.4.B. Recommendation We recommend that the Service du capital humain et des communications (for the Employeur D system) and the Division de la gestion des programmes de logement, which reports to the Direction de l'habitation of the Service de la mise en valeur du territoire (for the SDSR system), define a procedure for managing access, which contains, at the very minimum, requirements to comply with the following elements: o Request for access; o Change to access; o Review of users and their access rights; o Deletion of access. Business units' responses: SERVICE DU CAPITAL HUMAIN ET DES COMMUNICATIONS [TRANSLATION] Following your recommendation, the procedure was revised. (Completed, March 2013) SERVICE DE LA MISE EN VALEUR DU TERRITOIRE [TRANSLATION] Drafting of an access management procedure for the SDSR system that provides a process to follow for access requests, amendments and cancellations, as well as for the twice yearly updating of the list of users. Dissemination by email of the access management procedure to all staff at the Direction de l'habitation and storage of the documents in the computer network's common registry. First update of the table of system users. This update should be done twice a year. (Completed, April 2013) 3.5. Discrepancies in the Physical Security of the Premises Housing Pay Records 3.5.A. Background and Findings The Division de la paie institutionnelle occupies, among others, premises on the mezzanine of the Chaussegros-de-L?ry building. In these premises, certain employee pay records are Auditor General of the Ville de Montr?al 513 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit filed in moveable (Rolodex-style) shelves. These records contain a large amount of sensitive PI, such as the first name, last name, address, SIN, banking information and date of birth of employees. Based on good industry practices and in the spirit of the Act respecting access to documents held by public bodies and the protection of personal information, offices that contain sensitive information must be protected by containment mechanisms to prevent unauthorized physical access. These mechanisms can be, for example, an area reserved for the storage of files, or doors installed to separate hallways or other means of access, such as stairs, from the offices. These doors should be fitted with card reader access locks. After visiting all the premises of the Division de la paie institutionnelle, we concluded that stairs, which begin in the mezzanine and are not an emergency exit route, lead to the upper floors that are occupied solely by employees of other city services. The stairs, which provide unrestricted access to the upper levels, have no containment mechanisms. We estimate the impact level to be moderate, since the city faces the following potential risks: by taking the stairs, employees who are not part of the Division de la paie institutionnelle could circulate freely to the pay records in the Rolodex shelves on the mezzanine. Consequently, a malicious employee could have access to PI and steal it by consulting certain pay records. 3.5.B. Recommendation We recommend that the Division de la paie institutionnelle of the Service des finances implement security measures to restrict physical access from the stairs leading to the upper floors so that only authorized employs can access the mezzanine of the Chaussegros-de-L?ry building. Business unit's response: [TRANSLATION] The Division de la paie institutionnelle will move its activities to 740, rue Notre-Dame Ouest as of summer 2013. The premises will be secured by magnetic access doors. (Planned completion: summer 2013) Auditor General of the Ville de Montr?al 514 2012 Annual Report Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 Penetration Tests 5.14 5.14. Penetration Tests Table of Contents 1. Background................................................................................................................... 519 2. Audits Scope................................................................................................................. 520 3. Results .......................................................................................................................... 520 Auditor General of the Ville de Montr?al 517 2012 Annual Report 5.14. Penetration Tests 5.14. Penetration Tests 1. Background Several business units of Ville de Montr?al and some bodies controlled by the city have systems through which they transmit critical and confidential information. Most of these systems are housed on common or individual networks that can sometimes be accessed through the Internet. In order to ensure effective safeguards are in place to adequately protect the information systems from cyberattacks, the industry strongly recommends that penetration tests be done to verify the strength of the controls applied to the various computer environments. According to the information security experts, testing the resistance of information systems to internal or external threats is of paramount importance. The term "penetration testing" refers to simulating, in a controlled and secure manner, malicious actions carried out by computer hackers to penetrate systems and networks, either through the Internet or internally, in an effort to identify any potential vulnerabilities in the computer systems, networks or software and enhance the information security. In opposition to penetration attempts by computer hackers, penetration testing is ethical because it is done with the prior consent of the entities involved. Specialists generally use the same tools and techniques as computer hackers, except that they do not cause damage to the information systems, make them unavailable, alter the information processed by these systems, or steal confidential information. The integrity, confidentiality and availability of the systems being attacked are maintained during the penetration testing. There are two main categories of penetration tests: o External penetration tests: these tests make it possible to know whether a malicious individual could, using the Internet, breach the security of the information system in order to: - obtain confidential or privileged information; - change information processed by these systems; - make the systems unavailable. o Internal penetration tests: these tests make it possible to determine whether an individual could, internally and using his usual access, compromise the security of the information system in order to carry out the same three actions mentioned under the external tests. Internal tests also allow access to and testing of information systems that are invisible from the Internet. Auditor General of the Ville de Montr?al 519 2012 Annual Report Chapter 5 Value-for-Money and Information Technology Audit 2. Audits Scope In an effort to ensure a reasonable level of confidence in the quality of existing controls and to reduce to an acceptable level the risks of cyberattacks on the information systems of some of the city's business units and bodies controlled by the city, we initiated a program of penetration testing in 2012. This program is continuing in 2013. The main objective is to test the security of computer environments that have been deemed critical and assess their resistance to a certain level of cyberattack originating externally and internally. 3. Results For obvious security reasons, we are unable to disclose, in the current annual report, the results of the penetration tests carried out in 2012. This same will hold true for the results of penetration tests carried out in 2013. Auditor General of the Ville de Montr?al 520 2012 Annual Report Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 Appendices 6 Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 Appendix 1 - Excerpts from the Cities and Towns Act 6.1 6.1. Appendix 1 - Excerpts from the Cities and Towns Act 6. Appendices 6.1. Appendix 1 - Excerpts from the Cities and Towns Act RSQ, chapter C-19 Updated to December 31, 2012 IV.1. -- Chief auditor 2001, c. 25, s. 15. Chief auditor. 107.1. The council of every municipality having 100,000 inhabitants or more shall have an officer called the chief auditor. 2001, c. 25, s. 15. Term. 107.2. The chief auditor shall, by a resolution approved by a two-thirds majority of the votes of the members of the council, be appointed for a term of seven years. The term may not be renewed. 2001, c. 25, s. 15. Ineligibility. 107.3. In no case may the following persons act as chief auditor: (1) a member of the council of the municipality and, where applicable, of a borough council; (2) the associate of a member mentioned in subparagraph 1; (3) a person who, personally or through an associate, has any direct or indirect interest in a contract with the municipality or a legal person referred to in paragraph 2 of section 107.7. Disclosure of interest. The chief auditor shall disclose in every report produced any situation that could cause a conflict between the chief auditor's personal interest and duties of office. 2001, c. 25, s. 15. Inability or vacancy. 107.4. If the chief auditor is unable to act, or if the office of chief auditor is vacant, the council shall, (1) not later than at the sitting following the inability to act or the vacancy, designate a person qualified to replace the chief auditor, for a period of not more than 180 days; Auditor General of the Ville de Montr?al 525 2012 Annual Report Chapter 6 Appendices (2) not later than at the sitting following the inability or the vacancy, or not later than at the sitting following the expiry of the period fixed under paragraph 1, appoint a new chief auditor in accordance with section 107.2. 2001, c. 25, s. 15. Expenses. 107.5. The budget of the municipality shall include an appropriation to provide for payment of a sum to the chief auditor to cover the expenses relating to the exercise of the chief auditor's duties. Amount of appropriation. Subject to the third paragraph, the appropriation must be equal to or greater than the product obtained by multiplying the total of the other appropriations provided for in the budget for operating expenses by (1) 0.17% where the total of those appropriations is less than $100,000,000; (2) 0.16% where the total of those appropriations is at least $100,000,000 and less than $200,000,000; (3) 0.15% where the total of those appropriations is at least $200,000,000 and less than $400,000,000; (4) 0.14% where the total of those appropriations is at least $400,000,000 and less than $600,000,000; (5) 0.13% where the total of those appropriations is at least $600,000,000 and less than $800,000,000; (6) 0.12% where the total of those appropriations is at least $800,000,000 and less than $1,000,000,000; (7) 0.11% where the total of those appropriations is at least $1,000,000,000. Exception. Where the budget of the municipality provides for appropriations for operating expenses related to the operation of a system of production, transmission or distribution of electric power, 50% only of those appropriations shall be taken into account in establishing the total of the appropriations referred to in the second paragraph. 2001, c. 25, s. 15; 2001, c. 68, s. 5. Duties. 107.6. The chief auditor is responsible for the application of the municipality's policies and standards relating to the management of the human, material and financial resources assigned to auditing. 2001, c. 25, s. 15. Duties. 107.7. The chief auditor shall audit the accounts and affairs (1) of the municipality; (2) of every legal person Auditor General of the Ville de Montr?al 526 2012 Annual Report 6.1. Appendix 1 - Excerpts from the Cities and Towns Act (a) that is part of the reporting entity defined in the municipality's financial statements; (b) of which the municipality or a mandatary of the municipality appoints more than 50% of the members of the board of directors; or (c) of which the municipality or a mandatary of the municipality holds more than 50% of the outstanding voting shares or units. 2001, c. 25, s. 15; 2010, c. 18, s. 20. Audit. 107.8. The audit of the affairs and accounts of the municipality and of any legal person referred to in paragraph 2 of section 107.7 comprises, to the extent considered appropriate by the chief auditor, financial auditing, auditing for compliance of their operations with the Acts, regulations, policies and directives, and auditing for value-for-money. Audit. The audit must not call into question the merits of the policies and objectives of the municipality or legal persons referred to in paragraph 2 of section 107.7. Documents and information. The chief auditor in the performance of his duties is authorized (1) to examine any document concerning the affairs and accounts relating to the objects of the audit; (2) to require from any employee of the municipality or any legal person referred to in paragraph 2 of section 107.7 all information, reports and explanations the chief auditor considers necessary. 2001, c. 25, s. 15; 2001, c. 68, s. 6. Audit. 107.9. Any legal person receiving an annual subsidy from the municipality of at least $100,000 is required to have its financial statements audited. Copy. The auditor of a legal person not referred to in paragraph 2 of section 107.7 that receives an annual subsidy from the municipality of at least $100,000 shall transmit to the chief auditor a copy of (1) the annual financial statements of the legal person; (2) the auditor's report on the statements; (3) any other report summarizing the auditor's findings and recommendations to the board of directors or the officers of the legal person. Documents and information. That auditor shall also, on the request of the chief auditor, (1) place at the disposal of the chief auditor any document relating to the auditor's audit and its results; Auditor General of the Ville de Montr?al 527 2012 Annual Report Chapter 6 Appendices (2) provide all information and explanations the chief auditor considers necessary concerning the auditor's audit and its results. Additional audit. Where the chief auditor considers that the information, explanations and documents provided by an auditor under the second paragraph are insufficient, the chief auditor may conduct such additional audit as he considers necessary. 2001, c. 25, s. 15. Audit. 107.10. The chief auditor may conduct an audit of the accounts or documents of any person having received financial assistance from the municipality or from a legal person referred to in paragraph 2 of section 107.7, as regards the use made of such assistance. Accounts and documents. The municipality and the person having received the financial assistance are required to furnish to or place at the disposal of the chief auditor any accounts and documents that the chief auditor considers relevant to the performance of the chief auditor's duties. Information. The chief auditor is authorized to require from any officer or employee of the municipality or from any person having received financial assistance any information, reports and explanations the chief auditor considers necessary to the performance of the chief auditor's duties. 2001, c. 25, s. 15. Audit. 107.11. The chief auditor may conduct an audit of the pension plan or pension fund of a pension committee of a municipality or a legal person referred to in paragraph 2 of section 107.7 where the committee requests the chief auditor to do so with the approval of the council. 2001, c. 25, s. 15. Duties. 107.12. The chief auditor shall, every time the council so requests, investigate and report on any matter within the competence of the chief auditor. In no case, however, may the investigation take precedence over the primary responsibilities of the chief auditor. 2001, c. 25, s. 15. Report. 107.13. Not later than 31 August each year, the chief auditor shall transmit to the mayor, to be filed with the council at the first regular sitting following its receipt, a report presenting the results of the audit for the fiscal year ending on the previous 31 December and indicate any fact or irregularity the chief auditor considers expedient to mention, in particular in relation to (1) control of revenue including assessment and collection; Auditor General of the Ville de Montr?al 528 2012 Annual Report 6.1. Appendix 1 - Excerpts from the Cities and Towns Act (2) control of expenditure, including authorization, and compliance with appropriations; (3) control of assets and liabilities including related authorizations; (4) accounting for operations and related statements; (5) control and safeguard of property owned or administered; (6) acquisition and utilization of resources without sufficient regard to economy or efficiency; (7) implementation of satisfactory procedures to measure and report effectiveness in cases where it is reasonable to do so. Report. The chief auditor may also, at any time, transmit to the mayor or the chair of the board of directors of a legal person described in paragraph 2 of section 107.7 a report of the findings and recommendations that, in the opinion of the chief auditor, warrant being brought to the attention of the council or the board of directors, as applicable, before the transmission of the chief auditor's annual report. The mayor or the chair of the board of directors must file the report with the council or board, as applicable, at the first regular sitting or meeting following its receipt. Copy of report. If the chief auditor transmits a report to the chair of the board of directors of a legal person described in paragraph 2 of section 107.7, the chief auditor must also transmit a copy of the report to the mayor of the municipality, to be filed with the council at the first regular sitting following its receipt. 2001, c. 25, s. 15; 2010, c. 18, s. 21. Report. 107.14. The chief auditor shall report to the council on the audit of the financial statements of the municipality and the statement fixing the aggregate taxation rate. Report. In the report, which shall be transmitted to the treasurer, the chief auditor shall state, in particular, whether (1) the financial statements faithfully represent the municipality's financial position on 31 December and the results of its operations for the fiscal year ending on that date; (2) the effective aggregate taxation rate was fixed in accordance with Division III of Chapter XVIII.1 of the Act respecting municipal taxation (chapter F-2.1). 2001, c. 25, s. 15; 2006, c. 31, s. 16; 2010, c. 18, s. 22. Report. 107.15. The chief auditor shall report to the boards of directors of the legal persons referred to in paragraph 2 of section 107.7 on the audit of the financial statements before the expiry of the time within which they are to produce their financial statements. Auditor General of the Ville de Montr?al 529 2012 Annual Report Chapter 6 Appendices Report. In the report, the chief auditor shall state, in particular, whether the financial statements faithfully represent their financial position and the results of their operations at the end of their fiscal year. 2001, c. 25, s. 15. Testimony. 107.16. Notwithstanding any general law or special Act, neither the chief auditor nor the employees under the chief auditor's direction or the professionals under contract may be compelled to give testimony relating to any information obtained in the performance of their duties or to produce any document containing such information. Immunity. Neither the chief auditor nor the employees under the chief auditor's direction may be prosecuted by reason of any act they have done or failed to do in good faith in the performance of their duties. Immunity. No civil action may be instituted by reason of the publication of a report of the chief auditor prepared under this Act or of the publication in good faith of an extract or summary of such a report. Immunity. Except on a question of jurisdiction, no recourse under article 33 of the Code of Civil Procedure (chapter C-25) or extraordinary recourse within the meaning of that Code may be exercised nor any injunction granted against the chief auditor, the employees under the chief auditor's direction or the professionals under contract acting in their official capacity. Annulment. A judge of the Court of Appeal, on a motion, may summarily annul any proceeding instituted or decision rendered contrary to the provisions of the first paragraph. 2001, c. 25, s. 15. Audit committee. 107.17. The council may establish an audit committee and determine its composition and powers. Audit committee of the urban agglomeration of Montr?al. Despite the first paragraph, in the case of the urban agglomeration of Montr?al, the council must establish an audit committee composed of not more than 10 members appointed on the proposal of the mayor of the central municipality. Two of the committee members must be council members representing the reconstituted municipalities. Those two members shall take part in deliberations and votes of the committee on any matter related to an urban agglomeration power. Auditor General of the Ville de Montr?al 530 2012 Annual Report 6.1. Appendix 1 - Excerpts from the Cities and Towns Act Opinions and information of the committee. In addition to the other powers that may be entrusted to it, the committee established in the case of the urban agglomeration of Montr?al shall submit opinions to the urban agglomeration council on the requests, findings and recommendations of the chief auditor concerning the urban agglomeration. It shall also inform the chief auditor of the interests and concerns of the urban agglomeration council with respect to the audit of the accounts and affairs of the central municipality. On an invitation by the committee, the chief auditor or a person designated by the chief auditor may attend a sitting and take part in deliberations. 2001, c. 25, s. 15; 2008, c. 19, s. 11; 2009, c. 26, s. 19. V. -- External auditor 2001, c. 25, s. 16. External auditors. 108. The council shall appoint an external auditor for not more than three fiscal years, except in the case of a municipality with a population of 100,000 or more, where the external auditor shall be appointed for three fiscal years. At the end of the term, the external auditor shall remain in office until replaced or reappointed. Information sent to the Minister. If the external auditor appointed for a fiscal year is not the external auditor in office for the preceding fiscal year, the clerk shall inform the Minister of Municipal Affairs, Regions and Land Occupancy of the name of the new external auditor as soon as possible after his appointment. R. S. 1964, c. 193, s. 104; 1975, c. 66, s. 11; 1984, c. 38, s. 11; 1995, c. 34, s. 12; 1996, c. 27, s. 12; 1999, c. 43, s. 13; 2001, c. 25, s. 17; 2003, c. 19, s. 110, s. 250; 2005, c. 28, s. 196; 2009, c. 26, s. 109. Vacancy. 108.1. If the office of the external auditor becomes vacant before the expiry of his term, the council shall fill the vacancy as soon as possible. 1984, c. 38, s. 11; 2001, c. 25, s. 18; 2003, c. 19, s. 111. Duties. 108.2. Subject to section 108.2.1, the external auditor shall audit, for the fiscal year for which he was appointed, the financial statements, the statement fixing the aggregate taxation rate and any other document determined by the Minister of Municipal Affairs, Regions and Land Occupancy by regulation published in the Gazette officielle du Qu?bec. Report. The auditor shall make a report of his audit to the council. He shall state in his report, in particular, whether (1) the financial statements faithfully represent the municipality's financial position on 31 December and the results of its operations for the fiscal year ending on that date; Auditor General of the Ville de Montr?al 531 2012 Annual Report Chapter 6 Appendices (2) the effective aggregate taxation rate was fixed in accordance with Division III of Chapter XVIII.1 of the Act respecting municipal taxation (chapter F-2.1). 1984, c. 38, s. 11; 1996, c. 2, s. 209; 1999, c. 43, s. 13; 2001, c. 25, s. 19; 2003, c. 19, s. 250; 2005, c. 28, s. 196; 2006, c. 31, s. 17; 2009, c. 26, s. 109. Duties. 108.2.1. In the case of a municipality having 100,000 inhabitants or more, the external auditor shall audit, for each fiscal year for which the external auditor has been appointed, (1) the accounts relating to the chief auditor; (2) the financial statements of the municipality and any document determined by the Minister of Municipal Affairs, Regions and Land Occupancy by regulation published in the Gazette officielle du Qu?bec. Report. The external auditor shall make a report of the audit to the council. The external auditor shall state in the report on the financial statements, in particular, whether the financial statements faithfully represent the municipality's financial position on 31 December, and the results of its operations for the fiscal year ending on that date. 2001, c. 25, s. 20; 2001, c. 68, s. 7; 2003, c. 19, s. 250; 2005, c. 28, s. 196; 2009, c. 26, s. 109. Report to the treasurer. 108.3. The external auditor shall transmit to the treasurer the report referred to in section 108.2 or, as the case may be, the report referred to in subparagraph 2 of the first paragraph of section 108.2.1. Report to the council. The report referred to in subparagraph 1 of the first paragraph of section 108.2.1 shall be transmitted to the council on the date determined by the council. 1984, c. 38, s. 11; 2001, c. 25, s. 21; 2010, c. 18, s. 23. Audits. 108.4. The council may require any other audit it considers necessary, and require a report. 1984, c. 38, s. 11. Access to books and information. 108.4.1. The external auditor shall have access to the books, accounts, securities, documents and vouchers and may require the employees of the municipality to furnish any information and explanations necessary for the performance of the external auditor's mandate. 2001, c. 25, s. 22. Auditor General of the Ville de Montr?al 532 2012 Annual Report 6.1. Appendix 1 - Excerpts from the Cities and Towns Act Documents. 108.4.2. The chief auditor shall place at the disposal of the external auditor all books, statements and other documents prepared or used by the chief auditor during the audit conducted under section 107.7 and that the external auditor considers necessary to carry out his mandate. 2001, c. 25, s. 22; 2005, c. 28, s. 49. Ineligibility. 108.5. In no case may the following persons act as external auditor of the municipality; (1) a member of the council of the municipality and, where applicable, of a borough council; (2) an officer or an employee of the municipality; (3) the associate of a person mentioned in paragraph 1 or 2; (4) a person who, during the fiscal year for which the audit is carried out, has, directly or indirectly, personally or through his associate, any participation, interest or commission in or under a contract with the municipality or in respect of such a contract, or who derives any benefit from the contract, unless his connection with the contract arises from the practice of his profession. 1984, c. 38, s. 11; 1996, c. 2, s. 209; 1999, c. 40, s. 51; 2001, c. 25, s. 23. Partnership. 108.6. The external auditor may be an individual or a partnership. The external auditor may entrust his employees with his work but his responsibility is then the same as if he had performed all the work personally. 1984, c. 38, s. 11; 1999, c. 40, s. 51; 2001, c. 25, s. 24. VII. -- Director general Status. 113. The director general is the chief officer of the municipality. Authority. The director general has authority over all the other officers and employees of the municipality, except the chief auditor, who reports directly to the council. With respect to an officer or employee whose duties are prescribed by law, the authority of the director general is exercised only within the framework of his duties as the administrator of human, material and financial resources of the municipality and may in no case hinder the carrying out of duties that are prescribed by law. Auditor General of the Ville de Montr?al 533 2012 Annual Report Chapter 6 Appendices Suspension. The director general may suspend an officer or employee from his duties. He shall immediately make a report of the suspension to the council. The council shall decide the case of the suspended officer or employee, after inquiry. R. S. 1964, c. 193, s. 109; 1968, c. 55, s. 5; 1983, c. 57, s. 50; 2001, c. 25, s. 27. Auditor General of the Ville de Montr?al 534 2012 Annual Report Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 Appendix 2 - Employees of the Bureau du v?rificateur g?n?ral as of December 31, 2012 6.2 6.2. Appendix 2 - Employees of the Bureau du v?rificateur g?n?ral as of December 31, 2012 6.2. Appendix 2 - Employees of the Bureau du v?rificateur g?n?ral as of December 31, 2012 Management personnel Support staff Auditor general Executive secretary Jacques Bergeron, CPA, CA, MBA, M. Sc. Josiane Mauriello Assistant auditors general Accounting analyst Robert Duquette, CPA, CA France Benny Isabelle Tremblay, CPA, CA Serge Vaillancourt, FCPA, FCGA Programmer Yolaine Levasseur Senior audit manager Marie-?ve Lemieux, CPA, CA, CAoIFA Audit professionals Fran?ois Arbez, CISSP, CISM, CGEIT, CISA Bernard Goyette, CPA, CGA, CMA Martine Beauregard, CPA, CGA Jocelyne Laperri?re, CPA, CA R?gent Bilodeau, CPA, CGA ?ric Laviolette, CPA, CA Johanne Boudreau, BAA Isabelle L?ger, CPA, CA, CISA Jacques Brisson, CPA, CA, CISA Chantal L'Heureux, CPA, CGA Maryse Brunetta, CPA, CGA Joanne Major, CPA, CA Khadija Chaya, CPA, CMA, MBA Victor Marchand, CPA, CGA, CISA Marie Cormier, CPA, CA Philippe Pitre, CPA, CGA Christian ?lomo, CIA, CRMA, MBA Michel Proulx, CPA, CGA Ch?rif Ferah, MBA Pierre Rochon, lawyer, MBA Andr? Gagnon, CPA, CMA, CISA Andr? Sergerie, CPA, CA Lucie Gauthier, CPA, CGA Auditor General of the Ville de Montr?al 537 2012 Annual Report Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 Appendix 3 - Information Flow Charts - Value-for-Money and Information Technology Audit 6.3 6.3. Appendix 3 - Information Flow Charts - Value-for-Money and Information Technology Audit 6.3. Appendix 3 - Information Flow Charts - Value-for-Money and Information Technology Audit Planning Annual Overall understanding of the entity Portfolio of projects Assignments Multiannual risk planning (by the BVG) Risk assessment Selected projects Notification letters and other communications to managers, interviews, document reviews, etc. Assignment planning Notice to managers no Execution of assignment? yes Assignment confirmation letter and execution Special requests Auditor General of the Ville de Montr?al 541 2012 Annual Report next... Chapter 6 Appendices Auditing and reports On-site audit Interviews, visits, findings, tests, analyses, etc. Draft audit report Action plans from the managers of the business units involved Discussions on the draft report with the managers involved ...cont'd Final report and request for action plans from the managers of the business units involved Copy to the Direction g?n?rale for follow-up on the recommendations Action plans validated by the BVG next... Discussions on the preliminary findings with the people involved Auditor General of the Ville de Montr?al 542 2012 Annual Report 6.3. Appendix 3 - Information Flow Charts - Value-for-Money and Information Technology Audit Annual reports and follow-ups ...cont'd Final report with action plans Annual report of the auditor general Periodic follow-up by the Direction g?n?rale Auditor General of the Ville de Montr?al 543 Communications with the managers involved Interim report to the city manager Recording in the audit record management system (GDV) Auditor general's annual report to councils members Periodic follow-up by the BVG Selected excerpts Annual decisionmaking record from the Direction g?n?rale to councils members 2012 Annual Report Report of the Auditor General of the Ville de Montr?al to the City Council and to the Urban Agglomeration Council For the Year Ended December 31, 2012 Appendix 4 - Accounts Statement of the Bureau du v?rificateur g?n?ral 6.4 Accounts statement of the BUREAU DU V?RIFICATEUR G?N?RAL OF THE VILLE DE MONTR?AL (HEREINAFTER REFERRED TO AS "BUREAU DU V?RIFICATEUR G?N?RAL") December 31, 2012 Deloitte s.e.n.c.r.l. 1 Place Ville-Marie Suite 3000 Montreal QC H3B 4T9 Canada Tel.: 514-393-7115 Fax: 514-390-4111 www.deloitte.ca Independent auditor's report To the Mayor, the Chairman and Members of the Executive Committee, the Members of the Council of the Ville de Montr?al, and the Members of the Agglomeration Council of the Ville de Montr?al In compliance with the provisions of section 108.2.1 of the Cities and Towns Act (the "Act"), we have audited the accounts related to the Bureau du v?rificateur g?n?ral of the Ville de Montr?al for the year ended December 31, 2012, and a summary of significant accounting policies and other explanatory information (the "financial information"). Management's Responsibility for the Financial Information Management of the Bureau du v?rificateur g?n?ral of the Ville de Montr?al ("management") is responsible for the preparation of the financial information in accordance with the recognition and measurement principles of Canadian public sector accounting standards, as described in Note 2 to the financial statements of the Ville de Montr?al, and for such internal control as management determines is necessary to enable the preparation of the financial information that is free from material misstatement, whether due to fraud or error. Auditor's Responsibility Our responsibility is to express an opinion on the financial information based on our audit. We conducted our audit in accordance with Canadian generally accepted auditing standards. Those standards require that we comply with ethical requirements and plan and perform the audit to obtain reasonable assurance about whether the financial information is free from material misstatement. An audit involves performing procedures to obtain audit evidence about the amounts and disclosures in the financial information. The procedures selected depend on the auditor's judgment, including the assessment of the risks of material misstatement of the financial information, whether due to fraud or error. In making those risk assessments, the auditor considers internal control relevant to the entity's preparation of the financial information in order to design audit procedures that are appropriate in the circumstances, but not for the purpose of expressing an opinion on the effectiveness of the entity's internal control. An audit also includes evaluating the appropriateness of accounting policies used and the reasonableness of accounting estimates, if any, made by management, as well as evaluating the overall presentation of the financial information. We believe that the audit evidence we have obtained is sufficient and appropriate to provide a basis for our audit opinion. Opinion In our opinion, the financial information related to the Bureau du v?rificateur g?n?ral of the Ville de Montr?al for the year ended December 31, 2012 is prepared in all material respects, in accordance with the recognition and measurement principles of Canadian public sector accounting standards, as described in Note 2 to the financial statements of the Ville de Montr?al. May 3, 2013 ____________________ 1 CPA auditor, CA, public accountancy permit No. A116207 ACCOUNTS STATEMENT OF THE BUREAU DU V?RIFICATEUR G?N?RAL year ended December 31, 2012 (in thousands of dollars) 2012(1) Budget $ (unaudited) Compensation of personnel Salaries Fringe benefits Professional, technical and administrative services Other operating expenses Total (1) 2012 Actual $ 2011 Actual $ 3,107 842 3,949 3,148 879 4,027 3,115 795 3,910 744 553 1,136 635 5,328 616 5,196 539 5,585 Approved budget, as modified, presented in the accounting system of the Ville de Montr?al for the Bureau du v?rificateur g?n?ral and approved by the executive committee of the Ville de Montr?al. This accounts statement of the Bureau du v?rificateur g?n?ral was prepared in accordance with the recognition and measurement principles of Canadian public sector accounting standards, according to the same accounting policies described in Note 2 to the financial statements of the Ville de Montr?al for the year ended December 31, 2012. - I 441-4 1:11. I I- - 4 .4 run Funny! flu' '1 . II 1 ll'l'