Evaluation of the Buildings and Equipment Program: Overview, scope and methods, findings and recommendations

Program overview

Program definition

The Buildings and Equipment (B&E) Program oversees infrastructure and equipment maintenance activities and capital investments delivered at the Canada Border Services Agency's (CBSA) air, land, and marine ports of entry (POEs), at the CBSA College, at Immigration Holding Centers, and at staff housing units. The B&E Program aims to ensure that infrastructure and equipment are suitable, sustainable, safe, resilient, and operational for agency staff and members of the public through appropriate asset lifecycle management activities.

Organizational structure

The FCMB is the authority responsible and accountable for making and implementing all real property-related decisions. Specific directorates involved with the B&E Program, within Finance and Corporate Management Branch (FCMB), include:

• National Real Property and Accommodations
• Legislated Facilities Program Management Office
• Major Projects Office, Custodial Ports of Entry

The Office of Primary Interest (OPI) for the B&E Program is the National Real Property and Accommodations Directorate (NRPAD), within the FCMB. NRPAD plans and delivers the B&E Program, including the provision of infrastructure, accommodation, CCTV, material management, and Environmental Operations (EO) services for the agency.

The B&E Program is responsible for three (3) main activities:

• program management and governance
• environmental sustainability
• program delivery

B&E Program priorities

The Program activities and their associated outputs yield several intermediate outcomes, which emphasize the Program's key priorities:

• Agency infrastructure and equipment are managed effectively throughout their life cycle
• Infrastructure and equipment are operational and available for use by the CBSA
• Crown-owned asset conditions are adequate; and
• Program activities are conducted in an environmentally safe and sustainable manner

Evaluation scope and methods

Relevance: Program mission and strategy

• Alignment with the CBSA mandate, priorities, and objectives, as well as with GC priorities
• Alignment with Treasury Board (TB) policy modernization requirements
• Clarity of Program objective, roles, and responsibilities

Effectiveness: Program design, achievement of outcomes, and performance measurement

• Achievement of immediate program outcomes
• External environment – challenges to and opportunities for effectiveness
• Governance, project, and change management mechanisms
• Performance measurement mechanisms

Efficiency: Resources and planning

• Resource utilization
• Planning, maintenance, and risk mitigation
• Program stability

Out of scope

The evaluation did not assess intermediate and ultimate outcomes, detection technology, CCTV, the Land Border Crossing Project (LBCP), Gordie Howe International Bridge, and other major projects delivery, rail mode, HQ accommodations and infrastructure, and vehicles and uniforms for personnel overseas.

Lines of evidence

• 24 interviews with Regions and HQ
• Review of financial and human resources data
• Review of operational data
• Document review

Scope

Covered the period of fiscal year 2018 to 2019 to fiscal year 2021 to 2022. In some cases, data beyond this period were used to mitigate historical data limitations, up to April 2023. For more details, refer to section Detailed evaluation scope in Appendix C.

Evaluation approach

As a relatively new program, the Buildings and Equipment (B&E) Program has never been evaluated before. This was considered when determining the scope of this evaluation. The main purpose of this evaluation was to assess the relevance and performance of the B&E Program in the air, land, and marine modes, as well as the impacts of the nationalization of Program activities.

Overall findings and recommendations

What we found: Overall findings

The B&E Program was established in 2019, following the Nationalization Initiative that standardized and centralized the CBSA's internal services. Despite being relatively new in its current form, the evaluation found that the B&E Program plays an essential role in the agency and has made significant progress towards maturing its function. The evaluation also found areas in which the Program can continue to mature and grow, to better support the achievement of intended outcomes.

The Program has generally been successful in achieving its immediate outcomes, with some limitations. For example, the Program has achieved or surpassed the majority of Government of Canada (GC) environmental sustainability targets. In addition, the B&E Program has ensured that most custodial buildings are in good or fair condition. However, the CBSA had the highest percentage of custodial facilities in critical condition, relative to other government departments and agencies.

Significant data gaps limit the Program in making evidence-based decisions, measuring and reporting on its performance, and in planning for present and future needs.

The Program has aligned itself with CBSA and GC priorities. However, work is needed to update tombstone Program tools and frameworks to ensure that they represent more recent Program changes and the modernization of Treasury Board (TB) requirements.

Human and financial resource planning challenges may limit the Program in achieving its objectives, while responding to new and emerging priorities. However, data inconsistencies and gaps limited a quantitative assessment of the impacts of these challenges.

Looking ahead: Key considerations

Moving forward, there are several aspects of the B&E Program's governance and organizational structure that could benefit from a review by the VP of FCMB in coordination with CBSA's Business Lines and enabling branches.

The evaluation found that there is a lack of awareness of the roles and function of the B&E Program amongst internal stakeholders. Further, the division of responsibilities for building- and equipment-related files, across the agency, is somewhat fragmented. Improvements in these areas could better support the Program in optimizing its centralized function and increasing support for clients and stakeholders.

Alignment with the Functional Management Model (FMM)

The FMM promises improved program and service delivery by enabling business line coordination and informed decision-making, driving nationally consistent results, and advancing key operational, program, and corporate priorities.

In reviewing program governance and organizational structure, the evaluation identified opportunities to clarify B&E Program roles and responsibilities and to ensure a common understanding of the Program, agency-wide, in support of the FMM.

Moving forward, the B&E Program could consider assessing the effectiveness of:

• Equipment commodities with ownership in Branches outside of the FCMB, or with no clear ownership or accountability
• Directorates outside of the NRPAD responsible for other components of the agency's buildings and equipment portfolio and projects; and
• Existing Program governance committees

Engaging equity, diversity, and inclusion advisory committees

As part of an assessment of the B&E Program's structure and governance, the Program could also consider how it – as part of the agency at-large – could formalize processes for engaging with the CBSA's offices and committees representing equity-deserving groups.

For example: The evaluation found that there is currently a lack of formalized process for engaging with the Persons with Disabilities Advisory Committee, outside of the Accessibility Office, to ensure that the Committee is fully and regularly consulted.

What we found: Performance measurement and reporting

The Program has significant data gaps and limitations, including in: obtaining data from external stakeholders; adapting to evolving reporting requirements; financial data collection and reporting; and tracking project delays.

These gaps somewhat limited the evaluation team's ability to assess the Program's achievement of outcomes. It will be necessary for the Program to address these gaps to ensure it is able to meet its performance measurement and reporting requirements.

Clear objectives and methodology for data collection, retention, management, and analysis are needed for the Program to accurately reflect its contribution to the agency mandate and its resource constraints. This would support the effectiveness of maintenance and investment activities.

To ensure it is identifying priority areas for data collection that support performance measurement and reporting, the Program would also benefit from updating its Performance Information Profile (PIP) to align with the current structure and function of the Program. Having the knowledge and expertise to carry out performance measurement and data analytics, with accuracy, will also require a strengthened focus on developing data analytics capacity within the Program.

What we found: Program management mechanisms

The Program has put a number of mechanisms in place to manage its activities and to align itself with Treasury Board (TB) requirements. However, the evaluation found that many of these mechanisms were created shortly before TB policy modernization. As a result, these tools refer to outdated requirements, and some do not refer to more recent changes in the Program's structure, roles, and responsibilities. A lack of updated mechanisms limited the evaluation in fully assessing the Program's alignment with CBSA, TB, and GC priorities.

While the Program has made progress in updating some of its materiel management guidance documents, the Program would benefit from a systematic review and refresh of its mechanisms, to ensure it is meeting its intended outcome of establishing "robust business processes supporting fixed and movable assets investment decisions" and meeting current TB requirements.

Further, the evaluation identified gaps in certain practices that could be improved, in tandem with the review and update of Program frameworks and strategies. In particular, the current, deferred maintenance strategy does not support the Program in making evidence-based decisions on deferred maintenance and asset replacement.

What we found: Resource planning

Inconsistencies in historical financial data reporting limited the evaluation in fully assessing Program resource use and in quantifying the impacts of reported human and financial resource challenges. This also presents a risk, as the Program does not have sufficient information to accurately identify the resources needed to address the demands associated with an aging real property portfolio.

The Program's reliance on over programming, and the one-year appropriation constraint, contributed to planning and resourcing challenges, affecting the Program's ability to achieve its objectives, while responding to new and emerging priorities.

Expertise in real property and materiel management is also highly sought after. Program representatives reported difficulties in retaining staff with the necessary expertise in real property and materiel management to support the delivery of Program activities. This was supported by CAS data, which suggested a turnover rate approximately double that of the agency.

While the Program supports credentialing in project management, and has established a National Training Standard for real property staff, the evaluation identified opportunities strengthen the Program's human resources strategy to ensure that gaps in training and learning, recruitment and retention, and specialized positions are proactively identified and mitigated.

Detailed findings

Theme 1: Relevance

Finding 1: The B&E Program is aligned with the stated priorities of the Government of Canada and the CBSA. The Program has also made progress in aligning with Treasury Board (TB) policy modernization requirements.

Overview

The evaluation team assessed the B&E Program's mandate, activities, and priorities against a range of Government of Canada (GC) and CBSA priorities, as well as the Program's alignment with TB policy modernization initiatives. An analysis was undertaken which compared internal program documents and ongoing efforts with GC, CBSA, and TB policy modernization corporate documents.

The evaluation team found that the Program's current, stated outcomes align to CBSA and GC priorities, and have incorporated modernized TB policy requirements. For this reason, many of these elements were assessed as part of the analyses on Program effectiveness and efficiency, and any gaps were identified therein. Additional information pertaining to priorities, corporate documents, and TB policy modernization requirements are available elsewhere in the report.

The B&E Program directly contributes to the GC's progress towards meeting the United Nations Sustainable Development Goals (SDG). This includes the Program's efforts to: reduce GHG emissions, explore sustainable materiel management strategies; and contribute to land conservation. For example, in direct alignment with the GC Greening Government Strategy, the CBSA's Rigaud Campus, Other Effective area-based Conservation Measures (OECM) parcel of land will contribute 8.45 hectares to the Canadian Protected and Conserved Area Database. This land acts as a valuable ecological corridor for wildlife to safely navigate the surrounding land.

In alignment with the Accessible Canada Act and the CBSA's Accessibility Plan, the B&E Program has successfully integrated construction requirements to increase accessibility of spaces, and toward achieving a barrier-free agency. Accessibility requirements are detailed in the NRPAD's corporate documents on design standards/guidelines. However, there are currently no approved Program Key Performance Indicators (KPI) related to Accessibility or GBA Plus. Program staff indicated that these are in development (refer to page 16).

The B&E Program increasingly aligned with CBSA's FCMB Integrated Business Plan (IBP), albeit to varying degrees across Branch priorities. The IBP priorities of "nationalization in support of the better delivery of professional services" and "improved working conditions via improvement plans," have already been realized by the B&E Program.

Areas of alignment:

• Sustainability
• GBA Plus
• Indigenous business
• Seized goods disposal
• Stakeholder engagement
• Biodiversity efforts

Finding 2: The B&E Program has taken significant strides towards implementing accessibility best practices and ensuring an inclusive work environment for all equity-deserving groups. While ongoing consultation is positive, continued prioritization of Accessibility and GBA Plus considerations will contribute to further improvement.

GBA Plus considerations

Signage: The B&E Program has made progress towards modernizing signage and ensuring inclusivity, where possible. According to Program interviewees, signage is a high priority. With respect to Indigenous reconciliation and inclusion, Program staff highlighted their engagement with Treasury Board Secretariat (TBS) on the utilization of trilingual signage. Interviewees also noted that Indigenous acknowledgement and inclusion is not limited to signage and that the B&E Program could build architectural features such as fossil walls, artwork, or video displays within custodial facilities.

CBSA uniforms: The Uniform Issuance and Policy Unit in Travellers Branch has consulted several committees representing diverse groups on policy and program related issues. Most recently, the Unit has made changes to gender binary descriptions in the Uniform Supply System (USS) and has collaborated with stakeholders on the development of new commemorative pins. The Unit recognizes that additional work is needed to address the agency's commitment to Inclusion, Diversity, Equity, and Accessibility.

Accessibility considerations

As detailed in the CBSA Accessibility Plan, the CBSA aims to be barrier-free by 2040. In order to achieve this, the agency will ensure:

• Physical spaces accessed by the public will be compliant, wherever possible, with industry standards
• An assessment of CBSA's real property portfolio will be initiated to guide investments; and
• An advancement on CBSA's National Accommodations Strategy which aims to meet the latest workplace design standards (GCworkplace) in CBSA offices, which includes design principles that incorporate barrier-free accommodations in federal workplaces

The Buildings & Equipment Program regularly consults with the CBSA's Accessibility Office (AO) on ongoing program initiatives to ensure compliance with requirements and the adoption of accessibility best practices. The B&E Program has also consulted other branches, external accessibility offices, and grassroots organizations on accessibility. Interviewees noted that consultation with some groups, such as the Persons with Disabilities Advisory Committee (PDAC), has been less frequent and remains an informal process.

Finding 3: Overall, the Program mandate, priorities, and roles and responsibilities are clearly defined in Program and CBSA corporate documents. However, there is a lack of awareness across the agency, and within the FCMB, of the function and scope of the Program following nationalization.

The B&E Program has a distinct role in overseeing infrastructure and equipment maintenance activities and capital investments at the CBSA. Its mandate, priorities, roles, and responsibilities are clearly defined.

The B&E Program was first listed in the agency's Program Inventory in fiscal year 2018 to 2019, and was linked with the agency's core responsibility of "border management." In part, the rationale for creating the B&E Program was linked to the critical role of infrastructure and equipment in agency operations. Various Program documents contain details on the Program's mandate, roles, and responsibilities, including internal, strategic documents and frameworks. However, many Program documents do not reflect the current organizational structure.

Some interviewees did not have a clear understanding of the function of the Program. This included some internal stakeholders at HQ and in the Regions, OSIs, and a few personnel within the Program OPI. A few interviewees did not know that Buildings and Equipment existed as an agency program. Some interviewees also discussed a lack of clear and concrete information on role descriptions, goals, and objectives for various areas within the Program.

Some interviewees highlighted a potential for duplication in the current Program structure, as the Program is somewhat fragmented across multiple DGs, including three directorates in FCMB, with individual planning, reporting, resource management, and control activities.

For example: The centre of expertise for Section 6, legislated facilities, is currently housed in a separate directorate, outside of the NRPAD (OPI), despite the NRPAD's responsibility for the maintenance of legislated facilities.

The current structure of the Program, and a lack of communication and awareness, may have created overlap, duplication, and confusion amongst stakeholders.

Some Program stakeholders noted that they did not always know whom to contact on specific files, which limited efficiency in seeking support on buildings and equipment-related issues.

Some interviewees also noted that stakeholders may not have a clear understanding of what the B&E Program funds, or of the delineation between the Program and equipment commodity owners outside of the NRPAD.

Many of the agency's equipment commodities are not owned by the NRPAD. For example, the agency's uniform and soft body armour are owned by the Travellers Branch. As the Material Management Functional Authority, the NRPAD works closely with the Travellers Branch for the management of these commodities. Other equipment-related files, such as radios, have no clear accountable authority within the agency's organizational structure at the time of this evaluation.

However, interviewees did not have a common understanding of the Program's functions or scope.

Theme 2: Effectiveness

Finding 4: While the existing B&E Program logic model has helped the Program to structure its activities, outputs, and outcomes, further efforts are needed to update and formalize it as part of a revised Performance Information Profile.

Evaluation methodology & contributions

As part of this evaluation, the Program requested a review of the Program's existing logic model (Appendix B), in line with its longer-term performance measurement improvement efforts. The review process consisted of:

• Identifying outputs and outcomes that had been misclassified (e.g. outcomes classified as outputs, activities classified as outputs)
• Rewording or paraphrasing activities, outputs, and outcomes to use clear, concise, and active language
• Ensuring alignment with key program documents
• Identifying outcomes that were overly ambitious, given the scope of the B&E Program; and
• Recommending where additional context should be provided to ensure a consistent understanding of the logic model

Following the completion of the review process, the evaluation team provided feedback to the Program in support of their continued development of the B&E logic model. The evaluation observed gaps, such as a lack of reporting on the Program's impacts on Inclusion, Diversity, Equity, and Accessibility (IDEA), to support GBA Plus – a reporting requirement for all GC departments and agencies in the Treasury Board Directive on Results.^{Footnote 1}

The Program's logic model has not yet been formally updated or approved.

Finding 5: While the Program's performance data and reporting have improved between fiscal year 2018 to 2019 and fiscal year 2021 to 2022, additional improvements and new tools are necessary to ensure data validity and completeness. There are data gaps related to limited information on leased and legislated facilities and to inconsistent reporting.

Accessibility: Some elements of B&E Program data are accessible. However, other data elements were inaccessible or incomplete. This was largely attributed to multiple data sources that were not integrated or consolidated into a centralized or national data repository at the Agency; insufficient information collected, systematically, by the Program, from external stakeholders on its leased/legislated facilities (e.g. from PSPC, Brookfield Global Integrated Solutions (BGIS)); and the limited (duration) of data collected by the Program on new data points and/or monitoring processes (e.g. Environmental Operations (EO) monitoring the halocarbon inventory).

Verifiability: The verification of B&E Program data is facilitated by the Program's regular monitoring, validation, and updating activities for many of its data points.

A notable shortcoming stems from NRPAD's reporting structure. Data sets are often retrieved manually, rather than through automated processes. This can create data verifiability issues. A limited data analytics capacity, within the Program, can also result in data pull errors, which can affect Program decision-making.

Consistency: The B&E Program's data is largely consistent, with data collection and monitoring processes now in place.

Instances of data inconsistencies were often tied to restrictions on data collection (e.g. data gaps for leased and legislated facilities), new best practices, new data points, changing reporting requirements, and the workload burden associated with COVID-19.

Reliability: In general, the B&E's Program data is reliable and is actively used in decision-making processes related to maintenance, renewal, and renovation and risk mitigation.

Note that data verifiability, consistency, and accessibility inform data reliability and as such, issues with each of those facets of data impact data reliability.

Despite recent improvements, the existing B&E reporting mechanisms are insufficient and require additional updates.

While some interviewees noted that performance data and reporting have improved, nearly all agreed that more improvement is needed. The evaluation observed that there are numerous data gaps due to a lack of historical data retention, limited integration, and a lack of data continuity. However, overall, data collection and reporting improved between fiscal year 2018 to 2019 and fiscal year 2021 to 2022. For example, there have been improvements in data on buildings' ages, sizes, overall conditions, and purposes. This information helps fulfill TB requirements and provides a more accurate estimation of building replacement costs.

Limited insight into external stakeholder activities yields notable data gaps, hindering monitoring capabilities and affecting decision-making processes.

Communication with external stakeholders, such as PSPC, was consistently cited as an area for improvement, amongst interviewees. Due to the lack of structured approach for obtaining information from external organizations, the B&E Program was limited in its ability to report against certain targets. For example: It is difficult for the Program to report on the value of procurement contracts awarded to Indigenous-owned businesses, due to a lack of data. This is because PSPC-led procurement comprises the majority of B&E Program procurement, for which the agency does have not data.

For data on the agency's leased facilities (i.e. PSPC custodial facilities), the evaluation team was also advised that certain information is not available in a centralized or national repository (e.g., Building Condition Reports). This includes, for example, maintenance data from third party contractors, held by PSPC. The evaluation found that cooperation with other external stakeholders, such as PSPC and BGIS, could aid the Program in improving data quality and reporting against TB reporting requirements (e.g. on building conditions and maintenance).

Incomplete or inaccessible data sets can impact the effectiveness of Program management and complicate reporting against KPIs or TB policy requirements.

Shortcomings of B&E Program data reporting:

• Time consuming & potential for inaccuracy due to manual data retrieval processes
• Limited integration
• Lack of information obtained, by the Program, from external partners
• Inconsistency in reporting metrics and gaps in data retention
• Limited data analytics capacity and heavy reliance on individuals' corporate knowledge

Ongoing efforts to improve data reporting

• An automated reporting solution is in the development phase
• The CRA is overseeing the implementation of the SAP solution for 48 Agencies, including the CBSA
• The SAP solution is expected to go live in fiscal year 2026 to 2027

Finding 6: Existing governance, program, and change management mechanisms highlight the Program's efforts to enhance its business processes and manage risk. However, the evaluation identified gaps and areas for improvement, and certain tools have not been updated in accordance with new TB policies and directives.

The B&E Program has various mechanisms in place to support robust business processes and manage risk. These include:

1. Governance mechanisms, committees, and working groups

The Program has three committees, with recurring meetings, that support project planning decisions (Appendix D). Based on a sample of Multi-Regional Meeting records, the Program leverages opportunities to discuss strategic direction, prioritization, and funding and planning considerations. The Program also presents to the agency's Financial and Investment Management Committee (FIMC) and Investment and Project Management Committee (IPMC) for funding and project-related approvals (Appendix E). However, Program interviewees highlighted gaps in governance, noting that there is room for improvement. For example, the Program's Technical Review Sub-Committee was suspended, and the Program no longer has a formalized process for consultation with technical experts.

2. Program Risk Profile and individual, project risk registers

First created in 2018, the Program Risk Profile groups risks into two broad categories: business risks and enabling risks. A total of five program risks were identified at the creation of the Risk Profile – two business risks and three enabling risks. As of fiscal year 2020 to 2021, all risks were labelled as open. The Program Risk Profile and project risk registers help the Program to identify and assign mitigation strategies to specific risks.

3. Program implementation strategies and frameworks

In fiscal year 2020 to 2021, the agency developed and updated the following major, strategic documents (up to April 2021). However, many of these documents have not been updated following TB policy modernization (May 2021):

• Materiel Management Framework
• Materiel Management Guide and Directive
• CBSA Standard on Fleet Management
• Real Property Management Framework
• Real Property Portfolio Strategy
• Departmental Sustainable Development Strategy
• B&E Performance Information Profile

These tools establish the Program's business processes and support risk management. It is important that certain documents be updated, to ensure that Program activities align with changes in TB requirements.

Example: Through the Real Property Portfolio Strategy, the Program established a framework for the management of its real property portfolio. This includes a plan for Building Conditions Reports (BCR). The agency has implemented a cyclical building condition assessment (every six years), as per TB guidance and industry best practices.

Strategies and plans: guide and support the acquisition, maintenance, and disposal of the agency's assets.

Frameworks: translate policies and directives into a concrete outline of the Program mandate, processes, portfolio, accountabilities, and monitoring and reporting requirements.

Guides, policies, directives: ensure assets are used for their intended purpose and are safeguarded and identify asset-related authorities and establish roles and responsibilities.

BCRs: Facilitate strategic planning, data-driven decision-making, and budget allocation. Support tracking of investments, compliance with TB policies, and facilitate Departmental Results Framework and MAF updates.

Note: While outside of the scope of this evaluation, as of 2023, some of the documents listed on this page were updated, and refer to the new TB policy guidance, such as the Materiel Management Guide and Directive.

Finding 7: The B&E Program has generally been successful in its continued development and updating of a range of acquisition, maintenance, and divestiture strategies related to its movable assets, including the introduction of innovative and efficient measures.

Fleet replacement algorithm

CBSA's Fleet Management team uses an algorithm based on a range of variables including vehicle age, mileage, lifetime, downtime, and fuel consumption to inform strategic vehicle replacement. Variables can be assigned different weights, as needed, enabling complete flexibility with respect to replacement as Program, CBSA, and/or GC priorities evolve.

Environmental Operations (EO) inventories

The development, monitoring, and updating of inventories pertaining to all aspects of environmental compliance and greening government efforts detailed in CBSA's 2020 to 2023 Departmental Sustainable Development Strategy (DSDS) enables strategic selection of assets requiring maintenance or replacement. These inventories are continually expanded as new assets are introduced and/or transferred from regional authorities.

Materiel management directive & guide

In an effort to standardize and streamline CBSA Materiel Management policies, processes, and procedures, a newly developed Directive and Guide were published in 2023. These tools: identify roles and responsibilities for the effective management of movable assets; and promote and support informed asset management decisions on planning, acquisition, operations, and disposal.

Seized goods disposal

The tracking of seized goods disposal began in November 2021. While historical data are unavailable, anecdotal evidence indicates a significant reduction in the amount of goods destroyed. In addition, an MOU was established between the CBSA and Public Services and Procurement Canada (PSPC) Seized Property Management Directorate (SPMD) to dispose of seized/abandoned/forfeited goods on behalf of the CBSA (refer to page 20).

Through the efforts described above, the Program has made progress toward updating its practices in alignment with the Directive on the Management of Materiel

Effective May 13 2021, the objective of the TB Directive on the Management of Materiel is to ensure that assets are planned, acquired, operated, maintained, and divested of in a manner that supports the delivery of programs and services to Canadians, while ensuring best value to the Crown. This directive details requirements, oversight mechanisms, reporting frameworks, decision-making processes, and advice on sound materiel management. In the CBSA context, this directive guides the B&E Program in its management of the agency's materiel and replaces a number of legacy policy instruments.

Finding 8: Recent developments in the tracking and disposal of seized goods better support the Program in achieving efficiencies and its environmental sustainability objectives. Additional data could further support the Program in quantifying these efficiencies.

Historical data on the disposal of seized goods is limited, as monitoring of disposal activities only began in November 2021. Despite historical data limitations, anecdotal data suggested that nearly 100% of goods were flagged for destruction in the years leading up to seized goods disposal tracking (2021). The figure below (figure 1) demonstrates a significant reduction of approximately 25% to 30%, from pre-2021 levels.

Figure 1 - Text version

Figure 1: Method of disposal by fiscal year

	Historical anecdote	2021 to 2022	2022 to 2023
Destruction	100.00%	71.76%	73.43%
Seized Property Management Directorate	0.00%	27.06%	25.09%
E-Waste	0.00%	1.18%	0.62%

Source: FCMB, NRPAD data, fiscal year 2021 to 2022 to fiscal year 2022 to 2023.

Partnering with PSPC has supported the Program in achieving its objectives for disposing of seized goods.

Prior to the agency's partnership with PSPC's Seized Property Management Directorate (SPMD), BSOs destroyed nearly all seized goods. Since partnering with SPMD, the volume of seized goods disposed via destruction has substantially decreased, creating environmental efficiencies. However, disposal of certain commodity types, such as drugs and guns, still relies heavily on destruction.

The revenue generated as a result of selling seized goods is also used to offset the cost of other disposal activities. Correspondence with the Program indicated that since the SPMD's partnership with the CBSA began in June 2022, the CBSA has avoided approximately $839,000 in costs related to the disposal of seized goods through SPMD-led sales. More detailed data on cost recovery related to the sale of seized goods were not available. Further, this does not reflect continued costs (human and financial) incurred by the CBSA to conduct disposal activities.

Finding 9: There are many causes (common and unprecedented) that can contribute to lower real property project completion rates. A lack of detailed tracking of project completion timeliness, and of the issues that can cause delays, undermines the Program's ability to effectively manage risks and forecast project timelines.

The Program completed fewer real property projects than planned, each year, between fiscal year 2020 to 2021 and fiscal year 2022 to 2023.

The number of projects completed, as planned, each fiscal year, declined from fiscal year 2020 to 2021 to fiscal year 2022 to 2023 (for more details, refer to Appendix F). In addition, the number of deferred or lapsed projects increased slightly over this period. However, without additional data or information on project completions, it is difficult to draw any conclusions on these trends. For example, they could be a result of varying project timelines and levels of complexity, or they could be associated with over-programming and the one-year appropriation constraint (Appendix A), the effects of the COVID-19 pandemic (e.g. supply chain disruptions and workforce shortages), or other challenges.

Figure 2 - Text version

Figure 2: Project^{Footnote *} status categories by fiscal year

	2020 to 2021	2021 to 2022	2022 to 2023
Complete	40%	17%	26%
Underway	43%	66%	60%
Not started	9%	7%	5%
Deferred/lapsed	5%	9%	6%
Cancelled	4%	1%	4%

Source: FCMB, NRPAD data, fiscal year 2020 to 2021 to fiscal year 2022 to 2023.

Improved tracking of common causes and costs of delays could help the Program to better anticipate and mitigate risks.

On-time, on budget, and in scope are important project management tenets for the Program, to ensure effective and timely project completion. A variety of causes can affect timely project completion, such as unanticipated land conditions and issues with contracting and procurement.

However, the Program is limited in its ability to leverage information on project completion delays, to support risk management and project timeline forecasting. Historically, the Program has not maintained a data repository on planned and actual project start and end dates. Further, there is a lack of reporting and tracking of the common, and unprecedented, causes of any project completion delays.

A lack of detailed data in this area limited the evaluation in its analysis of project completions. However, the evaluation used a case study, on the North Portal Project, to highlight the benefits of collecting detailed information on project delays and costs of delays. While this type of information is not yet reported centrally or held within a data repository, Program representatives reported that improvements are actively being made.

Case study: North Portal project

This case study highlights the difficulties the Program can face in investment planning and in managing project completion risks. Improved planning, in coordination with external partners, is needed to ensure that planned budget and timelines for infrastructure projects are adhered to more closely. Through an analysis of the North Portal Project Closure Report, this case study also highlights best practices in the tracking and reporting of causes for project delays. If applied to other projects, these best practices could support improved planning moving forward.

Context: Under the Beyond the Border (BtB) Action Plan, Canada and the U.S. committed to investing in physical infrastructure at key entry points to relieve congestion and expedite traffic across the border. This included the North Portal POE, the main Canada-U.S. border crossing in Saskatchewan.

The prime consultant contract was awarded in January 2016 and the construction contract was awarded in April 2018, prior to the nationalization of the B&E Program in 2019. The project was originally the responsibility of the Prairie region.

Implementation challenges: Multiple challenges and delays pushed the project completion date, affected the scope, and increased the budget.

The Project closure report attributes these challenges and delays to existing/unknown site conditions, difficulty in acquiring/maintaining sub trades, poor quality deliverables (inaccurate or incomplete construction documents, drawings, construction practices) and different interpretations of work requirements.

Results: Construction on the North Portal POE was completed in December 2021, and landscaping was completed June 2022. The North Portal POE's infrastructure was improved and modernized. The North Portal POE reopened, in full, in October 2022.

Project timeline:

• May 2018 Construction began, with an original planned budget of $10 million.
• December 2019 Original planned completion. The budget increased by 264% to $26.4 million, to account for delays and a planned and approved scope increase
• October 2022 Actual project completion, nearly three years later than planned.

$16.4 million increase on a $10 million project

Finding 10: The B&E Program manages client and stakeholder relationships effectively through its engagement efforts and established agreements. However, there may be opportunities to improve existing agreements to facilitate communication, information sharing, and the achievement of Program outcomes.

In overseeing infrastructure maintenance and replacement activities, the B&E Program engaged with many internal and external stakeholder groups. These stakeholders include: CBSA business line branches and operational programs, CBSA Regions, PSPC, BGIS, owners/operators, Indigenous communities, Health Canada, Canada Food Inspection Agency, and the U.S. Customs and Border Protection. The Program works closely with stakeholders in the conduct of its activities. Relationships with external stakeholders are often managed through policies, agreements, and MOUs. The FCMB has also established a Division, in a Program OSI Directorate, to oversee Stakeholder Relations and Communication.

However, the Program does not gather information, systematically, from PSPC. PSPC is a major partner of the B&E Program, managing almost all (over 90%) of the Program's contracting. Despite its shared MOU, the Program does not have a formal information sharing agreement to facilitate the transfer of data needed to support the Program's reporting and monitoring efforts. There may be opportunities for the Program to leverage existing processes, by including data sharing and reporting when outlining requirements to PSPC in the planning and development phases of projects.

Most interviewees expressed that client and stakeholder relationships are managed effectively. The relationships were described as being collaborative and positive.

BSA custodial buildings: The inventory of Federal Real Property that is under the administration of the Minister of Public Safety and Emergency Preparedness and for which the CBSA has full life-cycle responsibilities.

Leased buildings: Facilities administered by Public Services and Procurement Canada (PSPC).

Legislated facilities: Owners and operators of infrastructure that receive imported goods are required by law to provide, at no cost, space for detention and examination, where warranted, of persons or goods entering Canada. These facilities are legislated under Section 6 of the Customs Act.

The Program has faced challenges in managing its relationship with owners/operators.

The CBSA's relationship with the owners/operators of legislated facilities relies on interpretation of Section 6 of the Customs Act and evergreen policy documents, creating uncertainty and inefficiency in the maintenance of these facilities.

The Policy for legislated facilities and the Fit-Up and Funding Accountabilities (FUFA) document support the Program in managing its relationships with owners/operators of legislated facilities. In particular, the FUFA outlines construction, installation, replacement, and maintenance responsibilities, as they vary by POE.

However, Program representatives reported that the standards applied by owners/operators to legislated facilities, may not always align with CBSA standards, despite these tools. For example, certain GC priorities – such as, Inclusion, Diversity, Equity, and Accessibility (IDEA) – can be difficult to implement, without additional clarity in the roles and responsibilities and, in particular, when facilities are owned by organizations located in the United States.

Note: Challenges in managing relationships with facilities owned and operated by third parties, that are not legislated, (e.g. postal facilities), were reported. However, this facility type was not within the scope of the evaluation (Appendix C). The Program may benefit from further analysis of these issues to improve support for agency requirements in these facilities.

"We've dealt a lot with Section 6 [legislated facility] operators and our relationship is difficult to manage. I think progress is being made"

Interviewee

Example: Regional Portfolios have worked closely with owners/operators to ensure all parties are aware of all requirements, including investment planning for facility upgrades. The Program also maintains a compliance tracker, to monitor non-compliance with CBSA requirements. However, Program representatives reported that these risks could be better managed and the Program could benefit from more insight into the lifecycle maintenance requirements of these facilities. The 2020 to 2023 FCMB IBP also described legislated facilities as a risk for the Branch, noting that a lack of clarity and formal agreements for these facilities may cause funding disputes, over-specification, and inconsistencies.

Finding 11: The B&E Program has ensured that most custodial buildings are in good or fair condition. Buildings that are in poor or critical condition are concentrated in isolated and rural areas.

While the majority of custodial facilities were in good and fair condition, 40% of facilities were in critical and poor condition.

The B&E Program currently owns and operates 392 custodial buildings across Canada, of which 139 (35.5%) are in good condition. Facilities in good condition comprise the largest condition category. This suggests that the B&E Program is generally ensuring buildings are maintained.

While a range of factors can impact building condition and maintenance capabilities, the evaluation team's analysis suggested that the equitable distribution of maintenance between census classifications could be improved.

The DFRP is an actively maintained inventory, refreshing its data as new information becomes available and simultaneously removing historical data. Due to a lack of historical data, the evaluation team could not assess how building conditions have changed over time.

The vast majority of buildings identified to be in critical or poor condition fell within the isolated census classification.

Of the CBSA's 141 isolated buildings, 37 were identified to be in critical condition. This represented 26% of the isolated portfolio and is the second largest category of isolated buildings, with isolated buildings in good condition comprising the largest category (33%).

Of the CBSA's 242 rural buildings, 37 were identified to be in critical condition. This represented 15% of the rural portfolio. Buildings in poor, fair, and good condition represented 20.6%, 27.6%, and 36.4% of the rural portfolio, respectively. Buildings in critical condition comprised the smallest condition category within the rural census classification (Appendix A).

The CBSA's urban custodial portfolio is relatively minimal when compared to its rural and isolated portfolios, comprising only nine buildings. Four (4) of those buildings were in critical condition, one (1) in fair condition, and four (4) in good condition. More details are available in Appendix G.

Building Condition Report (BCR) availability

BCRs were only available for custodial facilities. BCRs for legislated and leased facilities are managed by PSPC. Though requested, PSPC did not provide this data and reported that it is not intended for building occupants. Additionally, PSPC representatives noted that the Department does not maintain a central repository of BCRs, which makes data sorting and extraction cumbersome.

Finding 12: The CBSA has the highest percentage of custodial facilities in critical condition, relative to OGDs. However, a lack of reporting and data retention prevented an analysis of historical trends.

Relative to OGDs, the CBSA has the highest percentage of its custodial facilities in critical condition.

The CBSA had the highest percentage of custodial facilities identified to be in critical condition, 16% higher than the average percentage of OGD portfolios identified to be in critical condition.

Similarly, the CBSA retained the second highest percentage of custodial facilities identified to be in poor condition, at 20%.

However, it is important to note that the Land Border Crossing Project (LBCP), managed by the Major Projects Office, Custodial Ports of Entry Directorate, is currently in the process of replacing 24 POEs identified to be in the worst condition. Further details on the replacement of these facilities, including target completion date, were not available to the evaluation team and the LBCP was out of scope for this evaluation.

Overall, it must be noted that it is difficult to draw meaningful comparisons between the CBSA and OGDs, given the diverse nature of the agency's real property portfolio. However, a number of data gaps further limited the analysis of building conditions. For example, the DFRP is an actively maintained inventory, refreshing its data as new information becomes available and removing historical data. Financial data benchmarking, in support of this analysis, was also limited by insufficient OGD financial data and the Program's current inability to report on financial information at the building-level.

Finding 13: The Program is limited in its ability to assess asset deterioration in the intermediate and long term due to the deferred maintenance backlog, increases in replacement value, and a lack of reliable data.

It is difficult to calculate an annual variance of accumulated, deferred maintenance for custodial facilities. Program financial data is limited and often lacks continuity between fiscal years.

The evaluation team reviewed the Real Property Financial Submissions for the Directory of Federal Real Property (DFRP) and the Management Accountability Framework (MAF) to determine percentage increases in deferred maintenance, over time. However, the data available on replacement and deferred maintenance costs were unreliable. Several reporting templates were used across the fiscal years examined, which limited comparability. The B&E Program indicated that this was often a function of changing TB reporting requirements. A lack of continuity in reporting methods can yield significant issues with respect to program management and decision-making on maintenance and replacement opportunities.

While the B&E Program's deferred maintenance values have increased, year-over-year, Program representatives indicated that deferred maintenance and replacement value calculations only account for annual increases due to inflation, indexed with construction costs. Historically, the Program has not been able to calculate other changes in deferred maintenance (e.g. reductions in backlog, "new" deferred maintenance).

In their work to renew the Program's maintenance strategy, analyses of deferred maintenance and replacement costs may also benefit from leveraging data on replacement costs by asset categories. In tandem with other criteria, this information could better support prioritization and proactive, evidence-based decision-making on the maintenance and replacement of agency assets (Appendix H).

What is deferred maintenance?

Deferred maintenance is the practice of postponing maintenance activities (e.g. repairs on real property) due to limitations in available resources, such as time, staffing, or funds.

"…The way we're working right now, we're just reactive. We address what is crumbling first."

Interviewee

" […] we're not being preventative. We're reacting."

Interviewee

Finding 14: The B&E Program has successfully met the majority of its environmental sustainability targets for CBSA custodial facilities and assets. Where targets have not yet been met, significant progress has been made and Program staff have confirmed their ongoing efforts towards new and more ambitious targets.

Halocarbon target: This target has been met. 100% inventory verification of halocarbons by 2023

Storage tank target: This target has been met. 100% of storage tanks with an audit completed by 2023

Potable water target: This target has been met. The removal of 8 drinking water advisories by 2023

Contaminated sites target: This target has been met. 1.5 million in funding by 2025

Asbestos target: This target has been met. 100% of sites assessed for asbestos

Radon target: Progress has been made. By 2023, test 80% of all CBSA facilities for the presence of radon. Currently: 97% of custodial facilities

GHG target: Progress has been made. 40% reduction of GHG emissions below 2005 levels by 2030, with aspiration for achievement by 2025. Currently: 14.5% below 2005 to 2006 base year levels. Note: In 2021, the Greening Government Strategy target was revised to 40% below 2005 levels by 2025

Fleet acquisition target: This target has been met. 75% of new light-duty unmodified administrative fleet vehicle purchases are zero-emission vehicles or hybrids by 2030 with supporting charging station infrastructure at 21 custodial ports of entry by 2023.

Go to Appendix I for more details on fleet acquisition.

Finding 15: The Program has demonstrated an ability to address urgent and emerging priorities. However, adapting to meet external challenges and opportunities as they arise, without a planned and dedicated source of funds, may detract from the Program's regular activities.

Various external environmental challenges and opportunities affect B&E Program activities, and numerous examples of how the Program has and continues to respond and adapt were identified. Such as:

• Changes in legislation, regulations, and GC/OGD priorities
• Operational requirements, technological innovations, international agreements
• Canadian economy (e.g. inflation) and the Budget
• Climate change and natural disasters, public health and pandemics

Example: Addressing urgent needs during the COVID-19 pandemic

The agency established an Emergency Management Committee (EMC) and a COVID-19 task force in 2020 to address unexpected expenses related to the pandemic. The EMC managed consolidated requests to the FIMC and Executive Committee for emergency funding requirements, and the B&E Program supported the provision of Personal Protective Equipment and infrastructure fit-ups.

While some departments requested emergency funding from Treasury Board Secretariat (TBS), the CBSA did not request incremental financing for the emergency and related B&E costs were absorbed by the agency.

Example: Addressing changing operational environments due to climate change

In response to recent natural disasters that have affected CBSA frontline operations and assets, the Program has planned to complete a climate change risk assessment and is ensuring that climate change preparedness is considered in new building design requirements.

Resources for responding to emerging challenges and opportunities

Generally, an over-reliance on over-programming (Appendix A) has affected the Program's ability to dedicate funds for unexpected events.

Many interviewees noted that, oftentimes, changing priorities require a diversion of resources and efforts from regular Program activities. Pivoting to address emerging needs (e.g. health and safety) can affect the Program's ability to complete its core activities.

B&E Program funding can also depend on the changing priorities of the agency's operational program areas that request and transfer funds to the Program. That said, the agency's cost centre "FIMC Reserve" is intended to fund unexpected events and large-scale emergencies will often result in dedicated funding provisions from the GC.

Theme 3: Efficiency

Finding 16: The National Training Standard (NTS) for Real Property and Accommodations Specialists helps to standardize training needs and expectations for many B&E Program employees. However, low levels of participation in training may affect the Program in meeting client and stakeholder needs, as well as implementing GC priorities, such as Inclusion, Diversity, Equity, and Accessibility (IDEA).

In 2021, the B&E Program established a new NTS, which outlined training requirements by job classification. However, data reflected low participation rates in the identified training.

Participation in Equity, Diversity, Inclusion and GBA Plus-related courses, including those mandatory for the agency and those in the Program NTS, was low. Of three GBA Plus-related courses in the Program NTS, the evaluation found that none of those required to participate in two courses on GBA Plus had done so, and 38% of those required to participate in the course, "Aboriginal considerations in procurement" had done so.

Benchmarking with OGDs

The CBSA is unique, amongst the OGDs surveyed, in having established a standardized and formal training program for its real property personnel.^{Footnote 2}

Certification and specialization

While not required, Project Management Certification is supported by the Program and identified as an asset qualification when hiring.

In the EO Unit, interviewees indicated that the majority of staff belong to the PC job classification, which requires specialization in science. EO also provides staff with training, relevant to their specialized portfolios, and staff participate in technical working groups with subject matter experts. The specialization and ongoing development of staff supports the Unit's in achieving its objectives. This may be a best practice, that other areas of the Program could leverage.

Some interviewees indicated that workload can make it difficult to find time to complete training. A few interviewees also noted that there is a need to better address the training needs of staff in Regional Portfolios. The evaluation observed a lack of clarity, amongst Regional Portfolio staff, on what training options were available to them, as HQ staff located in the Regions.

Finding 17: The B&E Program has expanded its staffing complement each year, over four fiscal years. However, Program interviewees and HR data revealed challenges in recruitment and resourcing to meet the demand for specialized skills and expertise. Data inconsistencies limited a quantitative assessment of the impacts of these challenges on staffing levels.

The B&E Program encountered difficulties in addressing human resources gaps due, in part, to a lack of professional job classifications and difficulties in recruitment and retention.

Real property and materiel management activities require a specialized and sought-after skill set, which can make it difficult for the Program to recruit employees with the necessary expertise to support its activities. Further, the B&E Program had an average attrition rate (11%) approximately double that of the agency (5.6%), from fiscal year 2018 to 2019 to fiscal year 2021 to 2022. It is expected that nearly 21% of B&E Program staff may be eligible to retire within the next five years.

Interviewees highlighted challenges in recruiting and retaining qualified staff

Some interviewees indicated that there is a need for highly specialized positions to manage some B&E Program functions. It can be difficult to create these positions, as well as recruit and retain qualified employees to fill them (e.g. engineers). In addition, a few regional interviewees expressed concerns over the limitations that language requirements can pose for recruiting and retaining non-supervisory personnel with the necessary expertise.

Staffing the B&E Program

The number of B&E Program staff has increased over time, as has the Program's scope and level responsibility. Given the discrepancies between IBP reported, MAF reported, and CAM data on salary and non-salary expenditure, the evaluation could not accurately assess whether the Program's current staffing level meets the Program's needs, mitigates attrition, and is proportionate to the continued growth in the Program's roles and responsibilities within the agency.

The number of B&E Program employees and the use of overtime (OT) (in hours) have increased. B&E Program OT has increased at a higher rate than the agency average. While the increase in B&E Program OT was highest between fiscal year 2018 to 2019 and fiscal year 2019 to 2020, the year-over-year increases have since been relatively proportionate to the increase in staff.

Finding 18: Aggregate, B&E Program full-time equivalents (FTEs) increased over four fiscal years, with a notable shift towards consolidating Program FTEs within the NRPAD. This is an indication of B&E Program maturation and centralization.

Figure 3 - Text version

B&E Program FTEs. By directorate and fiscal year

	2018 to 2019 Total	2019 to 2020 Total	2020 to 2021 Total	2021 to 2022 Total
NRPAD	21.51 (32%)	53.34 (54%)	62.88 (66%)	72.53 (74%)
Other FMCB	17.70 (26%)	19.05 (19%)	9.73 (10%)	9.88 (10%)
Other Branch	28.07 (42%)	25.58 (26%)	22.95 (24%)	15.40 (16%)

Source : FCMB, NRPAD FTE Data, fiscal year 2018 to 2019 to fiscal year 2021 to 2022.

The number of B&E Program FTEs increased from 67.28 in fiscal year 2018 to 2019 to 97.81 in fiscal year 2021 to 2022.

The NRPAD FTEs allocated to B&E Program activities increased over time from 21.70 (32% of Directorate) in fiscal year 2018 to 2019 to 72.53 (74%) in fiscal year 2021 to 2022.

The nationalization initiative, centralization of buildings and equipment-related activities, and maturation of the Program likely explain the notable increase in NRPAD FTEs and the decrease in other FCMB directorates and other Branch FTEs.

Go to Appendix L for more details.

Finding 19: While the MAF-reported B&E Program budget and expenditures have increased year-over-year, inconsistencies in how financial data are reported limited an analysis of the Program's funding model.

Historically, financial data reporting is inconsistent, limiting an analysis of the B&E Program funding model.

The B&E Program relies on numerous streams of funding to support its activities. These include its A-base (core) and B-base funding. In the past, it has also included Accelerated Infrastructure Program 3 (AIP3) funding, internal bid funds (e.g. FIMC reserve), and funds transferred from other agency programs (e.g. TB submission funds).

The way that the Program reports on its financial data for the MAF has changed year-to-year. For example, in some years, the Program did not disaggregate its financial data by A-Base and B-Base funds. Internal, CBSA data inconsistencies, along with insufficient OGD data, limited financial benchmarking and an analysis of the Program's funding profile over the last four years.

Other data sources, such as financial data reported in the FCMB IBP and in the CAM, did not align with MAF-reported data, limiting analysis (Appendix M). The Program is aware of these discrepancies.

Many interviewees noted that the current funding model does not support the Program in addressing its priorities.

Figure 4 - Text version

minus−13.6 million average variance
40.05 million increase in expenditures (fiscal year 2018 to 2019 to fiscal year 2021 to 2022)

(in millions $)

	2018 to 2019Footnote *		2019 to 2020		2020 to 2021		2021 to 2022
	Budget	Expenditures	Budget	Expenditures	Budget	Expenditures	Budget	Expenditures
O&M	38.78	36.85	42.34	40.44	55.96	50.71	69.41	61.83
Capital	63.70	31.39	16.77	15.28	18.88	17.85	49.45	46.46

Source: FCMB, NRPAD, MAF reported data, fiscal year 2018 to 2019 to fiscal year 2021 to 2022