Evaluation of the Beyond the Border Action Plan - Shiprider Program

National Program Evaluation Services
Internal Audit, Evaluation and Review
Royal Canadian Mounted Police

December 29, 2018

Access to information assessment

This report has been reviewed in consideration of the Access to Information Act and Privacy Acts. The asterisks [***] appear where information has been removed; published information is UNCLASSIFIED.

Table of contents


Acronyms/definitions

BtB
Beyond the Border
CBP
Customs and Border Protection
CBSA
Canada Border Services Agency
FTE
Full-Time Equivalent
HRMIS
Human Resources Management Information System
NHQ
National Headquarters
NPES
National Program Evaluation Services
OAG
Office of the Auditor General of Canada
O&M
Operations and Maintenance
PMF
Performance Measurement Framework
PRIME
Police Records Information Management Environment
PROS
Police Reporting and Occurrence System
RCMP
Royal Canadian Mounted Police
TB
Treasury Board
TEAM
Total Expenditures and Asset Management
U.S.
United States
USCG
United States Coast Guard

1 Executive summary

Introduction

This report presents the results of the Treasury Board mandated evaluation of the Beyond the Border Action Plan - Shiprider Program conducted by National Program Evaluation Services (NPES) of the Royal Canadian Mounted Police (RCMP) in 2017-18. The evaluation examined the period from April 1, 2012 to March 31, 2017 and included RCMP National Headquarters (NHQ) and full-time Shiprider locations in British Columbia (Surrey, Victoria), and Ontario (Kingston, Niagara and Windsor).

Program Profile

The Beyond the Border Action Plan - Shiprider Program (Shiprider) was established in 2012-13 with the expected result of preventing criminals from leveraging the Canada-U.S. border to commit transnational crime. Shiprider is considered a unique and specialized program that through a cooperative approach, combats cross-border criminality on shared Canadian and U.S. waterways. Shiprider removes the international maritime boundary as a barrier to law enforcement by enabling seamless continuity of enforcement and security operations across the border. It involves vessels crewed by specially trained and designated Canadian and U.S. law enforcement officers who are authorized to enforce the law on both sides of the international boundary line. The RCMP's role is to pursue national security and transnational investigations, and to make it possible to interdict individuals attempting to illegally cross the border by water. The RCMP and United States Coast Guard (USCG) are the Central Authorities for Shiprider.

What we examined

The objective of the evaluation was to examine the relevance and effectiveness of Shiprider. The following evaluation questions were developed in compliance with the Treasury Board Policy on Results (2016) and in consultation with senior program management within the RCMP:

  1. Is there a continued need for Shiprider?
  2. To what extent is Shiprider aligned to federal government and departmental priorities?
  3. Is Shiprider being delivered as intended?
  4. To what extent is Shiprider performance data being used to inform decision-making?

The evaluation used a theory-based approach and the triangulation of multiple lines of evidence to corroborate its findings. Lines of evidence included a document and literature review, data analysis and 55 key informant interviews.

What we found

  1. There is a continued need for Shiprider to address cross-border criminality.
  2. There are no known programs that duplicate Shiprider.
  3. Shiprider is aligned with federal government and RCMP priorities and strategic outcomes.
  4. The Shiprider mandate is well defined and understood.
  5. The evaluation did not find sufficient evidence to demonstrate how Shiprider was progressing towards an intelligence-led delivery approach.
  6. Shiprider roles and responsibilities are defined and generally understood.
  7. The partnership between the RCMP and USCG is essential to the delivery of Shiprider.
  8. The evaluation could not determine whether the resources were sufficient to deliver Shiprider.
  9. Key Shiprider performance data were not available to measure the delivery of Shiprider.

Recommendations

  1. Review and revise the Shiprider delivery approach to ensure the program is being delivered as intended and is aligned with current Federal Policing priorities.
  2. Collect performance data that adequately measures the overall Shiprider expected results and supports decision-making.

2 Introduction

2.1 Purpose of the evaluation

This report presents the results of the Treasury Board mandated evaluation of the Beyond the Border Action Plan - Shiprider Program (referred to as Shiprider throughout the report) conducted by National Program Evaluation Services (NPES) of the Royal Canadian Mounted Police (RCMP). The objectives of the evaluation were to examine the relevance and effectiveness of Shiprider.

The evaluation commenced in December 2017 and concluded in December 2018 with a presentation to the Performance Measurement and Evaluation Committee. The report received the Commissioner's approval on December 29, 2018.

2.2 Evaluation scope and context

The Shiprider evaluation was undertaken in line with the Treasury Board Policy on Results (2016), which requires departments to measure and evaluate performance, using the resulting information to manage and improve programs, policies and services. The evaluation examined a five year period between April 1, 2012 and March 31, 2017 and included RCMP National Headquarters (NHQ) and full-time Shiprider locations in Surrey, Victoria, Kingston, Niagara and Windsor within E and O Divisions. Surge operations that took place outside of the full-time Shiprider team locations were not included in the evaluation.Footnote 1

Previous reviews

The Beyond the Border (BtB) Action Plan was created in 2011 to establish a new long-term partnership between Canada and the U.S. to enhance security and accelerate the legitimate flow of people, goods and services across the border.Footnote 2

In 2016, the Office of the Auditor General (OAG) completed an audit of the BtB Action Plan, inclusive of Shiprider. The purpose of the audit was to determine if participating Departments and Agencies were achieving results towards enhancing security and accelerating the legitimate flow of people, goods and services across the border.Footnote 3 The OAG report found that federal organizations had mostly implemented the security initiatives but could not show that they improved security at Canada's borders.

With respect to Shiprider, the OAG found that the initiative met its commitment to deploy four Shiprider teams and develop several performance indicators to show increased participation of certified RCMP officers, patrol hours, and arrests since 2012-13. However, performance indicators to demonstrate that law enforcement was more effective as a result of Shiprider were not developed, and therefore it was not possible to determine if Shiprider improved security at Canada's borders.Footnote 4 As a result, the OAG recommended that the RCMP develop performance indicators that clearly measure the security benefits of the initiative and measure and report accurate and reliable results against baselines and targets to be able to assess the security benefits achieved. The RCMP agreed to the recommendation to develop a logic model and Performance Measurement Framework (PMF) for the BtB initiatives, inclusive of Shiprider, no later than December 2016.Footnote 5

As of March 31, 2017, the logic model and PMF for the BtB Action Plan initiatives, inclusive of Shiprider, had been developed and were considered closed by the RCMP's Departmental Audit Committee. Although Shiprider performance indicators were developed as part of the PMF, Shiprider performance data collection was not scheduled to occur within the evaluation reference period of April 1, 2012 to March 31, 2017. As a result, performance data from the PMF was not available for use in this evaluation.

3 Program overview

3.1 Program description

In 2011, the Prime Minister of Canada and the President of the United States (U.S.) issued a joint declaration on a shared vision for perimeter security and economic competitiveness.Footnote 6 This initiated the BtB Action Plan, inclusive of Shiprider, to combat cross-border criminality in Canada and U.S. shared waterways. Shiprider was officially established in 2012-13 with the expected result of preventing criminals from leveraging the Canada-U.S. border to commit transnational crime.

Shiprider is a bi-national law enforcement initiative permitting specially trained and cross-designated Canadian and U.S. law enforcement officers to jointly crew each other's vessels and patrol common and undisputed waterways along our shared border, while under the direction and control of an officer of the host country.Footnote 7 Shiprider removes the international maritime boundary as a barrier to law enforcement by enabling seamless continuity of enforcement and security operations across the border.Footnote 8 The RCMP's role is to pursue national security and transnational investigations, and to make it possible to interdict individuals attempting to illegally cross the border by water.Footnote 9

The RCMP and the U.S. Coast Guard (USCG) are the Central Authorities for Shiprider.Footnote 10 Support is also provided by other Canadian and U.S agencies such as the Canada Border Services Agency (CBSA) and U.S. Customs and Border Protection (CBP). These agencies supported Shiprider by sharing intelligence, helping to develop a shared understanding of the security environment, and informing investments for the strategic deployment of resources along the border.Footnote 11

Shiprider locations were selected based on joint threat and risk assessments and are currently considered to be the highest-risk marine regions at the border.Footnote 12 In 2012, Shiprider consisted of two teams, one in Surrey and one in Windsor. It later expanded with *** additional teams in Niagara and Kingston in 2015, and *** in Victoria in 2016.

3.2 Program resources

The funding for Shiprider prior to April 2015 was coordinated and held nationally under the responsibility of the Federal Coordination Canada-US unit at NHQ. On April 1, 2015, the main share of the Shiprider funding envelop was transferred from NHQ to O and E Divisions.

Shiprider employs approximately 45 full-time equivalent (FTE) positions. The table below provides a snapshot of where these Shiprider resources were located as of March 31, 2017.

Table 1: ExpendituresFootnote 13 and ***Footnote 14 by Shiprider teams for 2012-13 to 2016-17.
Location 2012-2013 2013-2014 2014-2015 2015-2016 2016-2017 ***
Surrey $562,519 $944,899 $1,318,327 $1,888,510 $3,678,894 ***
Victoria ***
Windsor $296,624 $1,143,826 $1,131,742 $1,144,881 $1,085,797 ***
Kingston $1,065,799 $1,271,212 ***
Niagara $850,489 $1,651,135 ***
National Headquarters $2,355,905 $978,155 $1,122,218 $811, 656 $508,223 ***
TotalFootnote 15 $3,215,048 $3,066,880 $3,572,287 $5,761,335 $8,195,261 ***

4 Evaluation methodology

4.1 Evaluation approach and design

A theory-based approach was used for this evaluation. Extensive use of triangulation was undertaken as an analytical method, where multiple lines of evidence helped corroborate findings. The evaluation focussed on the relevance and effectiveness of Shiprider, as well as considered areas of interest identified by RCMP senior management. Qualitative and quantitative information was used to determine findings and recommendations for improvement and to help inform senior management decision-making.

4.2 Data sources

The following lines of evidence were used to inform the findings and recommendations.

Document and Literature Review. Relevant internal and external documentation such as foundational documents, performance related reports, previous assessments, management reviews, business cases/plans, operational documentation, policies, and other applicable information.

Data Analysis. Available financial, administrative, performance measurement and statistical data from the following systems and information sources:

  • Enforcement occurrences from the Police Reporting and Occurrence System (PROS) and
  • Police Records Information Management Environment (PRIME) databases;
  • Full-Time Equivalent (FTE) information from the Human Resources Management Information System (HRMIS);
  • Financial information from the Total Expenditures and Asset Management (TEAM) System; and
  • Enforcement activities from monthly/annual reporting logs.

Key Informant Interviews. A total of 55 interviews were conducted with a sample of key Shiprider personnel and law enforcement partners to obtain their views on Shiprider and to validate and supplement information gathered by other lines of evidence.

Figure 1: Percentage of interviewees by category – Text Version
Percentage of interviewees by category
Interview guides # of interviewees
Divisions 23, 42%
External Partners 18, 33%
Division Senior Management 6, 11%
Public Safety Canada 3, 5%
Border Integrity Operations Centre / Marine Security Operational Centre 3, 5%
National Headquarters 2, 4%

4.3 Methodological limitations and mitigation strategies

A key limitation of the evaluation was the lack of performance data. In addition, data such as occurrences in U.S. waterways were not collected by the RCMP, consequently the evaluation only provided a Canadian perspective. In order to mitigate these limitations, the evaluation used multiple lines of evidence and triangulation of data. This approach was taken in order to demonstrate reliability and validity of the findings and to ensure that conclusions and recommendations were based on objective and documented evidence.

5 Findings

5.1 Continued Need for Shiprider

Finding: There is a continued need for Shiprider to address cross-border criminality.

Law enforcement agencies working at and near the Canada – U.S. border are increasingly confronted with responding to and investigating criminal activity that extends beyond their respective national boundaries. Threat assessments conducted by Canadian and U.S. law enforcement agencies identify organized crime as the most prevalent threat encountered at the shared border. This includes significant levels of contraband trafficking, ranging from illegal drugs and tobacco, to firearms and human smuggling.Footnote 16

The results from document review and interviews found that Shiprider is a unique and specialized program that through a cooperative approach, combats cross-border criminality on Canadian and U.S. shared waterways. Shiprider serves as a best practice in the area of cross-border maritime law enforcement, matters of national security, transnational criminal investigations as well as enhancing the partnership between Canada and the U.S.

A review of intelligence reportsFootnote 17 suggests an increasing threat to Canada and U.S. shared waterways posed by the smuggling of drugs and illegal goods through small vessels and containers. Along with increased threats, Shiprider has seen an increase in occurrencesFootnote 18 identified in the PROS and PRIME systems. Occurrences (electronic police file/reports) are generated as a result of Shiprider operational activities including, but not limited to patrols, investigations and intelligence. Occurrences depict the number and type of files with which Shiprider operations are involved. When data from PROS and PRIME were examined, the number of occurrences increased from 68 occurrences in 2012-13 to 963 occurrences in 2016-17, for a five year total of 2,382Footnote 19 Shiprider occurrences generated (See Figure 2).

While the actual number of occurrences is known, it was not possible to determine a direct correlation between the increase and a reduction in cross-border criminality, due to the influence of other factors. For instance, the number of Shiprider locations have increased from two in 2012 to five in 2016. The increase in Shiprider locations may have contributed to the increase in the number of occurrences generated (See figure 2).

Figure 2: Number of Shiprider Locations and Occurrences by year – Text Version
Number of Shiprider Locations and Occurrences by year
2012-13 2013-14 2014-15 2015-16 2016-17
Shiprider locations 2 2 2 4 5
Occurrences 68 558 448 368 963

When interviewees were asked about the continued need for Shiprider, all interviewees (100%) stated there was a continued need. However, 44% of this group were also of the opinion that the need for Shiprider teams depended on geographic location, as the amount of vessel traffic and known criminal activity differs from one location to another. For example, the Victoria Shiprider team was established in August 2015 to enhance investigative and patrol capacity in locations where significant smuggling threats were occurring and to provide a mechanism for increased collaboration with law enforcement partners.

In addition, interviewees stated that seasonal differences based on geographic location should also be a key consideration in determining the need for Shiprider teams. For example, E Division teams conducted patrols all year round whereas O Division teams did not patrol during the winter season (approximately four months) but instead focused their efforts on Shiprider surveillance of waterways and providing operational law enforcement assistance to internal RCMP units and external partners (e.g. CBSA, local police forces of jurisdiction).

An increase of occurrences does reveal a continued need for Shiprider to collect, manage, share and analyze cross-border police information to enhance law enforcement's investigational and maritime awareness capabilities.

Finding: There are no known programs that duplicate Shiprider.

The cross-border capabilities of officers under Shiprider, who are authorized to exert their authority on both sides of the border, is what sets Shiprider apart from other marine programs. Based on international and domestic documents and interviews, there are no known programs that duplicate Shiprider due to the cross-border capability. Cross-border criminality seeks to exploit vulnerabilities in geography and gaps in enforcement. By authorizing Canadian and U.S. officials to operate on either side of the border, the USCG and RCMP have developed an innovative means of securing both sides of the border without violating the sovereignty of either nation.

While there is no duplication of Shiprider activities, there are a number of domestic and U.S. partners that complement Shiprider activities. For example, when interviewees were asked what programs complemented Shiprider activities, the most cited programs in E Division were CBSA and various E Division marine groups, while O Division most often cited the RCMP Marine Security Enforcement Team and CBP. Shiprider's collaborative effort with many of its partners continues to enhance marine security by fostering a law enforcement relationship for all marine units in their respective locations. Shiprider continues to demonstrate a need to combat cross-border criminality, particularly in the absence of any comparable marine program.

5.2 Alignment to federal government and RCMP strategic outcomes

Finding: Shiprider is aligned with federal government and RCMP priorities and strategic outcomes.

Shiprider supported the federal government's roles and responsibilities of a "safe and secure Canada" by maintaining the safety and security of Canada and its citizens through crime prevention, law enforcement, the securing of Canadian borders, and emergency preparedness.

A review of key documents linked the relevance of Shiprider to the mandate of the RCMP. Shiprider is aligned with the RCMP's strategic outcome of "criminal activity affecting Canadians is reduced" by enforcing federal laws and securing Canada's borders between ports. Shiprider also aligns with the Federal Policing Strategic Plan 2014-2017Footnote 20 by contributing to intelligence and enforcement efforts to the highest threats at home and abroad.

In addition to priorities related to law enforcement, the BtB Action Plan stated that its initiatives, including Shiprider, established a long-term partnership between Canada and the U.S. that is built upon a perimeter approach to security between the two countries.Footnote 21

5.3 Program delivery

Finding: The Shiprider mandate is well-defined and understood

The mandate for Shiprider was outlined in several Shiprider foundational documents, such as the Framework Agreement between Canada and the U.S.Footnote 22 and the BtB Action Plan.Footnote 23 The mandate described in the documents can be summarized as being "to jointly identify, assess, and interdict persons and organizations involved in any criminal offence related to cross-border law enforcement through the coordination of cross-border policing programs and activities." The mandate was well-defined and understood by almost all interviewees (96%) as being "to combat cross-border criminality through partnership between Canada and the U.S."

Finding: The evaluation did not find sufficient evidence to demonstrate how Shiprider was progressing towards an intelligence-led delivery approach.

Although the mandate was well-defined and understood, it was not clear if Shiprider was being delivered as intended. Foundational documentsFootnote 24 identified that Shiprider was to be delivered as an intelligence-led operation. The majority of interviewees (69%) stated Shiprider was being delivered as it was intended based on the mandate; however, the mandate does not define the delivery approach as being intelligence-led. For example, interviewees stated that Shiprider addressed cross-border criminality and enhanced the partnership between Canada and the U.S. However, interviewees (25%) who did not agree that Shiprider was being delivered as intended attributed this to Shiprider not being intelligence-led. In addition, a number of interviewees described the lack of developing, receiving, and sharing of intelligence as a barrier for Shiprider.

While Shiprider indicated within some foundational documents the desired state was to be intelligence-led, the evaluation did not find sufficient evidence to demonstrate how Shiprider was progressing toward this desired state.

Finding: Shiprider roles and responsibilities are defined and generally understood.

Shiprider foundational documents defined the roles and responsibilities of the RCMP and the USCG as the central authorities responsible for coordinating the implementation and ongoing operations of Shiprider.

The roles and responsibilities of RCMP Regular Members dedicated to Shiprider teams in the Divisions were defined within RCMP work descriptions.Footnote 25 Based on the work descriptions for "Operations Constable", "Team Leader", and "Supervisor", the most commonly listed roles and responsibilities included:

  • Conducting patrols;
  • Providing operational assistance;
  • Gathering and analyzing intelligence; and
  • Providing guidance on investigations.

The roles and responsibilities of NHQ were defined in foundational documents as supporting operations by providing guidance, oversight, and standardization of policies and practices for all Shiprider operations. The NHQ unit served as the policy center for Shiprider operations and addressed issues related to legislation, procedures, protocols, training and strategic coordination with partner agencies.Footnote 26 On April 1st, 2015, the main share of the Shiprider funding envelope was transferred to O and E Divisions, resulting in a shift of responsibilities.Footnote 27 However, the evaluation did not find evidence to indicate the nature of the change in roles and responsibilities from NHQ to the Divisions.

The majority of interviewees indicated roles and responsibilities were understood (detachment - 81%, division - 71%, and NHQ - 75%). Interviewees who stated roles and responsibilities were not well understood or unclear attributed this to a lack of clarity in, or unclear communication of, Shiprider objectives and priorities. For example, interviewees explained that it was difficult to determine which activities should be the focus, for example, enforcement-related or intelligence-related activities.

In order to examine whether the defined roles and responsibilities were put into practice, (an indication of understanding the roles and responsibilities), a sample of weekly reports from E and O Division were reviewed and compared with the other lines of evidence. The types of Shiprider activities interviewees most often cited, as well as reported on in weekly reports, were generally consistent with the activities defined within RCMP work descriptions (Table 2). Based on this comparison, conducting patrols was prioritized as the number one activity and consistent throughout, while gathering and analyzing intelligence was less prevalent. Discrepancies in the listed activities may be a result of factors related to geographic location, as described previously in the report, which may have influenced the nature and frequency of activities performed at each location.

Table 2. Comparison of Shiprider activities listed by interviewees, weekly reports, and work descriptions.
Most often cited activities by interviewees Most reported activities in weekly reports Most commonly listed responsibilities in work descriptions
1 Conducting patrols Conducting patrols Conducting patrols
2 Gathering and performing intelligence-led analysis Maintenance of equipment and administration tasks Providing operational assistance
3 Offering education and outreach initiatives to partner agencies and the public Investigations, intelligence, outreach and partnership, and training Gathering and analyzing intelligence

In order for Shiprider to be delivered in a manner that contributes to preventing criminals from leveraging the Canada-U.S. border, roles and responsibilities should be both defined and understood at all levels of Shiprider. From the evidence gathered, roles and responsibilities were varied but defined and generally understood by Shiprider personnel.

Finding: The partnership between the RCMP and USCG is essential to the delivery of Shiprider.

The BtB Action Plan and other documentationFootnote 28 stated partnership with the U.S. continues to be a priority for the Government of Canada, the RCMP, and other U.S. stakeholders in the delivery of Shiprider. This partnership is essential not only because the two countries have a shared border but also because Canada and the U.S. are allies and economic partners with interests in keeping their respective societies free from cross-border criminality.

By strengthening border integrity through the deployment of integrated border law enforcement operations, Shiprider demonstrates that it is a committed and reliable security partner. Shiprider involves specially trained and designated Canadian and U.S. law enforcement officers working together in joint teams to enforce the law on both sides of the border. Importantly, all operations are conducted under the discretion and control of law enforcement officers of the "host" country, assisted by the law enforcement officers of the "visiting" country, and in accordance with the domestic laws of the host country.Footnote 29

The Framework Agreement provided further details on commitments involved with the partnership between the RCMP and USCG. For example, the requirement for RCMP and USCG officers to obtain cross-designation is a key element to Shiprider. Since 2012, 97 RCMP and 354 USCG officers have successfully obtained their cross-border maritime law enforcement officer designation. In order to conduct patrols on both sides of the border, both RCMP and USCG cross-designated officers must be present on the boat during patrols as per the guiding principles outlined in the Framework Agreement. This ensures officers are able to secure both sides of the border without violating the sovereignty of either nation. Without a partnership, this would not be possible.

The partnership between the RCMP and the USCG is essential in conducting Shiprider operations, as both parties must be present during the hours of patrol. For example, in 2016, according to division statistical reports, presentations, and weekly reports, approximately 2,705 hours of Shiprider patrols took place.Footnote 30 The evaluation was unable to determine if the patrol hours were adequate to deliver Shiprider.

RCMP and USCG interviewees described their Shiprider relationship as both positive and effective, and communicated that it allowed them to benefit from each other's unique areas of expertise.

Finding: The evaluation could not determine whether the resources were sufficient to deliver Shiprider.

Lessons learned from previous Shiprider pilot projects provided a solid understanding as to the requirements for successful operations, including the resources required.Footnote 31 Since 2012-13, Shiprider FTEs were consistently less than what was forecasted (Figure 3). While in earlier years Shiprider FTEs were relatively aligned, the discrepancy becomes wider beginning in 2015-16.

Figure 3. Comparison of the forecasted FTE count with the actual FTE count for ShipriderFootnote 32 – Text Version
Comparison of the forecasted FTE count with the actual FTE count for Shiprider
2012-13 2013-14 2014-15 2015-16 2016-17
Forecasted FTE 15 30 30 63 63
Actual FTE 11.79 25.37 25.74 21.7 42.77

While the number of Shiprider FTEs were consistently less than what was forecasted, it was not possible to interpret whether more or less FTEs would have had an impact on the delivery of Shiprider. In addition, when interviewees were asked what challenges had an impact on the delivery, over half (53%) cited the lack of personnel as a challenge to operations.

Between 2012-13 and 2016-17, salary funding was forecasted at $25.1M and operations and maintenance (O&M) at $22.5M. Final spending on salaries during the reference period was $13.71M, a difference of $11.39M. Final spending on O&M was $4.16M, a difference of $15.34M. While there has been a gradual increase in expenditures (salary and O&M) for Shiprider, the expenditures were less than what was forecasted (Figure 4). Again, it was not possible to interpret whether more or less funding would impact the delivery of Shiprider.

Figure 4. Comparison of the forecasted salary/O&M expenditures with the actual salary/O&M expenditures for Shiprider – Text Version
Shiprider Salary Funding Comparison
2012-13 2013-14 2014-15 2015-16 2016-17
Forcasted Salary Spending 1.9 3.8 3.8 7.8 7.8
Actual Salary Spending 1.03 2.26 2.49 3.47 4.46
Shiprider O&M Funding Comparison
2012-13 2013-14 2014-15 2015-16 2016-17
Forecasted O&M Spending 3.5 3.1 3.1 7 5.8
Actual O&M Spending 1.04 0.74 0.98 2.04 2.36

It should be noted that in 2015-16, the main share of the Shiprider funding envelope was transferred from NHQ to the Divisions. Consequently, FTE salary actuals were not captured for E Division in 2015-16 potentially causing a gap.

5.4 Performance data

Finding: Key Shiprider performance data were not available to measure the delivery of Shiprider.

As stated previously, following the OAG recommendations, a PMF for the BtB Action Plan, inclusive of Shiprider, was approved on March 31, 2017; however, Shiprider data collection for the PMF was not scheduled to occur until April 2018, which is outside of the evaluation reference period.Footnote 33

Although Shiprider PMF data collection did not occur during the reference period, some performance data was being collected by Shiprider teams, such as number of patrol hours, patrols, number of boardings, etc. Almost all interviewees (95%) indicated data was being reported through activity logs/reports on a daily/weekly basis and/or PROS and PRIME; however, there was no evidence that this information was used to inform decision-making.

The evaluation found that data collection didn't measure the intended delivery and overall result of Shiprider. It should be noted that indicators were significantly different between O and E Division. For example, since 2015 E Division has been collecting data on 10 indicators while O Division has been collecting data on 18 indicators, none of which are the same with the exception of patrol hours. One exception noted was that within the same Division, indicators and reporting were consistent. For example, E Division has been collecting the same performance data, such as boardings, warnings, and arrests, since 2015 and O Division since 2016.

The most cited challenge with data collection and reporting was that the data collected did not measure how the overall program was being delivered. This is consistent with the finding from the OAG's Fall 2016 Report on the Beyond the Border Action Plan, which stated there were no performance indicators developed to measure intended results of the program. The OAG report found that the RCMP was not able to show that law enforcement was more effective as a consequence of Shiprider. The OAG recommended the development of performance indicators that clearly measure the security benefit of Shiprider and measure and report accurate and reliable results. As of March 31, 2017, a logic and PMF for the BtB Action Plan, inclusive of Shiprider were approved with data collection beginning April 2018. In line with the OAG findings and recommendations, Shiprider performance indicators and the lack of performance data continues to hinder the ability to measure the delivery of Shiprider.

6 Conclusions and recommendations

The evaluation found that there is a continued need for Shiprider, as it is a unique and specialized program with the expected result of preventing criminals from leveraging the Canada-U.S. border to commit transnational crimes. The need is dependent upon geographical locations, amount of vessel traffic and known criminal activity. Shiprider was aligned with federal government and RCMP priorities and strategic outcomes as it contributes to the safety and security of Canada and supports Canada's relationship with the U.S.

The partnership between the RCMP and USCG is essential to the delivery of Shiprider, as both parties must be present during patrols in order to combat cross-border criminality. Due to a lack of data, the evaluation was unable to confirm the strength of the relationship, although RCMP and USCG described their partnership as positive and effective and explained it allowed them to benefit from each other's unique areas of expertise.

The evaluation was unable to determine if Shiprider was being delivered as intended. While Shiprider has a well-defined and understood mandate, documents and interviews indicated that Shiprider's delivery approach is to be intelligence-led. The evaluation did not find sufficient evidence that Shiprider was progressing towards this desired state.

A lack of available performance data limited the ability to tell a complete performance story, including whether resources were adequate for the successful delivery of Shiprider. This is consistent with the findings from the OAG's Fall 2016 Report on the Beyond the Border Action Plan. The impact of Shiprider and its contribution to the prevention of cross-border criminality was unclear.

Shiprider is regarded as a best practice internationally in combatting cross-border criminality and fostering strong partnerships. In order to determine if Shiprider is being successfully delivered, it is important that Shiprider develop adequate performance measurement tools.

Based on the findings of the evaluation, it is recommended that the Deputy Commissioner of Federal Policing:

  1. Review and revise the Shiprider delivery approach to ensure the program is being delivered as intended and is aligned with current Federal Policing priorities.
  2. Collect performance data that adequately measures the overall Shiprider expected results and supports decision-making.

7 Management response and action plan

7.1 Management response

Senior officials responsible for the implementation of the Shiprider initiative reviewed the evaluation and accepts the findings and recommendations proposed by National Program Evaluation Services. The conclusions of this report will serve as a catalyst to modernise the current program and help evolve it to a higher level of maturity, aligning it with RCMP Federal Policing Core Responsibilities and Priorities.

7.2 Action plan for NHQ – BI

Recommendation Lead / area of responsibility Planned action Diary date

Recommendation #1

  1. Review and revise the Shiprider delivery approach to ensure the program is being delivered as intended and is aligned with current Federal Policing priorities.

FPCO-BI

In consultation with Shiprider units and stakeholders, review the intended delivery approach; including topics such as roles and responsibilities, resourcing and intelligence in comparison to the current divisional and national framework

July 2019

FPCO-BI

Based on consultations with Shiprider units and stakeholders, revise the delivery approach to align it with current FP Priorities and Core Responsibilities.

December 2019

FPCO-BI

Communicate the new delivery approach to all Shiprider units, affected divisions and external partners.

March 2020

Recommendation #2

Collect performance data that adequately measures the overall Shiprider expected results and supports decision-making.

FPCO-BI

Collect and analyse the Shiprider performance data as outlined in the existing Performance Measurement Framework.

May 2019

FPCO-BI

Based on collected performance data, identify gaps and assess whether the performance indicators are effective in measuring the expected results of Shiprider

October 2019

FPCO-BI

In consultation with Shiprider units, develop and implement performance indicators that align with the revised delivery approach, effectively measure the expected results and supports decision making regarding Shiprider.

December 2019

FPCO-BI

Communicate any new reporting requirements to all Shiprider units, affected divisions and partners.

March 2020

Date modified: