4.0 Performance, efficiency and economy

4.1 Achievement of expected outcomes

The Performance Information Profile (PIP)^{Footnote 8} for the Canada Remembers Program was revised in 2020 and presented to the Performance Measurement and Evaluation Committee (PMEC) in November 2020. The performance measures in the PIP are being collected by the Commemoration Division and key measures are publicly reported in the Departmental Results Reports.

The information presented in the PIP comes from internal data and the following two surveys that collect VAC client satisfaction and Canadians’ opinions:

• VAC National Client Survey (survey of representative sample of VAC clients, last conducted in 2020); and
• Attitudes Towards Remembrance and Veterans’ Week (survey of general Canadian population, conducted annually).

4.1.1 Program effectiveness

The objective of the Canada Remembers Program is to ensure those who served in Canada’s military efforts are honoured and the memory of their sacrifices and achievements is preserved.

The 2020 Attitudes Towards Remembrance and Veterans’ Week survey noted 92% of Canadians considered it important that VAC recognize and honour Canadian Veterans and those who died in service through the presentation and care of memorials, cemeteries, and grave markers. Table 3 below depicts one of the indicators for this outcome (the remaining indicators relate to the time it takes VAC to issue and ship medals through the Honours and Awards component of the program and were not in scope of this evaluation).

Table 3: Immediate outcome performance measure

Source: Attitudes Towards Remembrance and Veterans’ Week Surveys 2017, 2018, 2019, and 2020

The target satisfaction level of 65% has not been met over the past four years. The indicator is broad and encompasses multiple components. It is difficult to determine what Canadians are not satisfied with in relation to memorials, cemeteries and grave markers and determining what is needed to improve the level of satisfaction.

There is evidence this immediate outcome is being achieved. There are several indicators to measure this immediate outcome. Table 4 below presents the results of the measures for this outcome.

Table 4: Immediate outcome performance measures – survey results

Sources: Attitudes Towards Remembrance and Veterans’ Week Surveys 2017, 2018, 2019, and 2020, and internal reporting.

*Impacted by COVID-19

The performance measures show there is strong support for Veterans and that their contributions and sacrifices are valued by Canadians. The only indicator that did not meet its target each year was the orders for learning and historical resources and products (target is 15,000). In 2020-21, orders for learning and historical resources were impacted by COVID-19. Key informant interviews and a review of documents indicated many orders for large groups were not made in 2020 as public gatherings were not permitted (including some educators who were teaching remotely). Also, Veterans’ Week sample packs (promotional envelope containing a copy of each Veterans’ Week resource) were not distributed to schools in 2020 which, in turn, impacted demand for products.

The Attitudes Towards Remembrance and Veterans’ Week Survey is used to measure two indicators related to this outcome. A measure of youth participation in Encounters with Canada’s Canada Remembers Theme Week and visits to VAC’s main memorial and cemetery web pages are the other indicators used. The Encounters with Canada indicator is in the process of being removed from the PIP as Encounters with Canada permanently shut down its operation in January 2021 as a result of the impacts from COVID-19.

The survey asks two questions to Canadians: one being the extent to which Canadians say they are proud of the role that Canada’s military has played in peacekeeping missions and conflicts like the World Wars, the Korean War, and the war in Afghanistan. In 2020, 81% of Canadians responded they were proud of the role of Canada’s military. The second question asked Canadians how much they agreed or disagreed with the statement, “Veterans Affairs Canada's remembrance program effectively honours Veterans and those who died in service and preserves the memory of their achievements and sacrifices.” Over the period of the evaluation scope, 70 to 74% of Canadians surveyed agreed with this statement. The performance target for this indicator is 80%. Results increased with the age of the survey respondents, with 64% of Canadians under 45 years of age in agreement and 83% over 65.

There is an opportunity to review the indicators to ensure the most relevant ones are used to assess outcome achievement with consideration of measures to better reach younger Canadians.

All of the program activities and initiatives conducted by the Commemoration Division contribute to the Ultimate Outcomes of the Canada Remembers Program. VAC uses three indicators to measure achievement of this outcome.

The table below depicts the indicators, targets and results for this ultimate outcome.

Table 5: Ultimate outcome performance measures

There is a high level of interest and participation by Canadians in commemorative activities. The evaluation team noted through interviews and document review that there is an opportunity for the Program to review the indicators to ensure the best measures and targets are being used. For example, a measure that includes the full suite of social media channels managed by VAC would better reflect the scope of Canadians engaged in commemoration.

The instruments used to measure this outcome are the National Client Survey, visitation statistics and visitor satisfaction surveys for the Vimy and Beaumont-Hamel memorials sites, and web statistics for the National Virtual War Memorial. The results for the past four years were:

Table 6: Results for the ultimate outcome “Veterans and those who died in service are honoured”

*COVID-19 greatly impacted the number of visitors to Vimy and Beaumont-Hamel in 2020-21

**Indicator was added in 2019 but is not yet reflected in the PIP. Data is from visitor experience surveys that will be conducted every 3 years at the Vimy and Beaumont-Hamel memorials

The decrease in client’s satisfaction with VAC’s commemorative initiatives should be noted. The evaluation team analyzed data from the 2020 National Client Survey to get a further breakdown of satisfaction with how VAC’s commemorative initiatives honour those who served. The analysis of the question “I am satisfied with the way VAC’s commemorative initiatives honour those who served our country and preserve the memory of their achievements and sacrifices” is presented in the following chart:

Chart 1: 2020 National Client Survey Results: satisfaction with commemorative initiatives

The target of 80% was reached only for Veterans who were age 90 and older. Satisfaction is lower for younger Canadian Armed Forces Veterans, with a 60% satisfaction rate for Veterans aged 18 to 39. The satisfaction rate for Indigenous Veterans was 63%, lower than the 71% satisfaction rate for non-Indigenous Veterans. The evaluation team recognizes the Program has taken steps to increase programming and awareness of modern-day Veterans and underrepresented groups, through increased promotion of their achievements and sacrifices. However, work remains to be done to reach the target level of satisfaction for the above noted groups. The next client survey (scheduled for summer 2022) will have further results to help inform decisions and the direction and adjustments needed to for future programming.

The evaluation team also reviewed the 2019 Visitor Experience Survey report of visitors to the Vimy and Beaumont-Hamel Memorials. Key findings from the report were:

• 22% at Vimy and 13% at Beaumont-Hamel were Canadians.
• Over 90% were very satisfied with their visit; and
• More than half felt smartphone self-guided tours or more interpretive panels were important for future onsite experiences.

These results indicate very high satisfaction with European memorial sites. Also, they support evidence that an expansion to offer more technology and interpretive signage is desirable.

4.1.2 Gender Based Analysis plus considerations:

Gender-based analysis plus (GBA plus) is an analytical process used to assess how different groups of women, men and gender-diverse people experience policies, programs and services based on multiple factors including race, ethnicity, religion, age, and mental or physical disability. According to Treasury Board Secretariat (TBS), differential impacts on diverse groups of people are to be considered for policies and programs^{Footnote 11}. As of 2020 TBS requires the GBA plus indicators to be incorporated into PIPs. With the exception of youth, the Canada Remembers Program PIP has yet to reflect other demographic intersections and corresponding data specific to GBA plus. At the same time the evaluation team understands VAC is working to collect more GBA plus data through the client survey and the 2021 Census.

While performance information is being captured and recorded, there is an opportunity to revise the measures, baselines and targets to incorporate GBA plus data. Also, the PIP should reflect changes to programming that place greater emphasis on more modern missions, highlight diversity in service and interactive digital experiences to support VAC in meeting its outreach and diversity, equity and inclusion objectives.

4.2 Program efficiency

4.2.1 Commemorative Partnership Program

The CPP consistently has a high demand for funds. The table below outlines the number of projects the CPP funded, as well as its expenditures and reach. Additional funds were received in 2019-20 and 2020-21 to support the Highway of Heroes Tree Campaign, and in 2020-21 additional funds were received to further support the Juno Beach Centre.

Table 7: CPP Expenditures, Projects and Reach

* Excludes funding provided to the Invictus Games that was paid through the CPP.

A significant portion of the projects funded through the CPP are funded as grants, especially projects funded through the Community Engagement portion of the program. Grants are a more efficient way to administer a program. Projects funded as grants are mostly lower dollar (e.g, $10,000 or less) or deemed lower risk. Data analyzed by the evaluation team shows the majority of Community Engagement initiatives are funded through grants and Community War Memorial initiatives through contributions. See Tables 8 and 9 below for a break down of approved projects that were grants versus contributions. There was a decrease in the number of grants in 2020-2021 as more higher dollar projects were approved and funded as contributions. CPP staff noted in interviews they receive applications for more higher dollar and more complex initiatives (e.g., more digital and more technology) versus local community initiatives than previously. More on this in section 4.2.3.

Table 8: Approved community engagement projects - grants and contributions

Table 9: Approved community war memorial projects – grants and contributions

4.2.2 Service standards

The Commemoration Division has committed to meeting service standards for making application decisions. Service standards were met during the last two years of the evaluation period. See Table 10 below for details.

Table 10: Application decision service standards for the Commemorative Partnership Program

*For both program components the service standard was not met for 2017-18 and for the first three quarters of 2018-19 the standard was not met for the community war memorial program. Demand for funding exceeded the resources available. In an effort to manage this situation, decisions were delayed to allow time for a more thorough analysis of applications and to determine how to maximize the funds available.

4.2.3 European Operations

VAC is responsible for the stewardship of 14 National Historic Sites in Europe (nine in France, four in Belgium and one in the United Kingdom). These sites were created to commemorate historical events of particular importance to Canadians and Newfoundlanders, and to honour the sacrifices of those who served during the First and Second World Wars. Two of the sites in France have been designated as Canadian National Historic Sites–the Canadian National Vimy Memorial and the Beaumont-Hamel Newfoundland Memorial. These sites are the only National Historic Sites of significance outside of Canada, that is those with visitors and programming. Before the COVID-19 pandemic, Vimy and Beaumont-Hamel ranked in the top ten most visited National Historic Sites. Vimy was second only to the Quebec Fortifications and Beaumont-Hamel was ranked seventh. Together, these sites attracted an annual average of 932,349 visitors over the time period under evaluation (See Appendix B for site visit data).^{Footnote 12}

VAC’s 2020-21 Departmental Results Framework (DRF)^{Footnote 13} lays out European Operations’ (memorials) core responsibility as the conservation of Canada’s 14 war memorials overseas which includes:

• Making ongoing capital investments for all 14 memorial sites;
• Developing a comprehensive visitor experience strategy; and
• Developing a forest management plan for Vimy and Beaumont-Hamel.

Given the COVID-19 travel restrictions, the evaluation team did not visit the memorial sites. Due to this limitation, the evaluation team relied on internal planning and operational documents and key informant interviews to understand European Operations’ roles, responsibilities and core functions, which include:

• Preserving and conserving the memorial sites, which encompass monuments as well as nearly 150 hectares of forest, battlefields, and original First World War tunnels and trenches;
• Curating the experience of close to 900,000 visitors, including continual rotation of museum-grade exhibits and development and delivery of interpretive products and experiences; and
• Managing real property that includes 18 buildings, horticulture and maintenance equipment, 10 fleet vehicles, 60-year old septic fields, IT infrastructure, etc.

To ensure a global view of site needs, European Operations restructured its organization. Previously, there were two managers, one for Vimy, and one for Beaumont-Hamel and the other sites. The managers looked after both site maintenance and visitor experience. There is now one manager for visitor experience and one for site maintenance for all 14 sites. This organizational structure allows for a more global and cohesive view of each of the key objectives— conservation and visitor experience.

4.2.3.1 European Operations funding: ongoing capital investments and conservation

VAC’s 2018 Overarching Commemoration Evaluation stated there is a need for “stable long-term funding to allow for efficient planning and implementation . . . to ensure the ongoing commemorative integrity of the Canadian National Vimy Memorial and the Beaumont-Hamel Newfoundland Memorial.” The evaluation team learned from key informant interviews that ongoing stable funding remains a challenge as noted in the last departmental evaluation. Capital investments are needed to conserve the memorials and support infrastructure.

European Operations has taken measures to obtain some additional, necessary long-term stable funding via an increase in the directorate’s budget. A long-term cost analysis of operations, maintenance, and preservation requirements associated with the 14 overseas memorial sites has been completed. Also, they have completed detailed conditional surveys and continue to conduct annual inspections of the sites.

European Operations has triaged site maintenance and interpretative requirements across a five-year forward-looking maintenance cycle and manage a $4 million operating budget for the maintenance, interpretation and administration of the memorial sites. Key informants indicated this amount is not sufficient to maintain and conserve the cultural integrity of the memorial sites, which also encompasses provision of a positive, culturally rich and immersive visitor experience. A review of the European Operations’ 2021-22 Operational Plan notes funding gaps remain. As a result, maintaining standard operations while actioning other key priorities, will continue to be difficult.

Key informants stated VAC needs to be planning 10 or 15 years in advance and that it is hard to plan without secured capital investment. From a project management standpoint, interviewees stated it is difficult to procure services, expertise, and resources through the annual departmental business planning exercise. For European Operations, the annual departmental funding approval cycle has the complication of procuring international services and contracts. For such large scale and culturally sensitive sites, key informants noted that planning two to three years out with secured capital investment for 10 years would better enable conservation of the sites and, in some instances allow for needed upgrades to the infrastructure.

Conservation of the memorials ensures their overall cultural integrity. This is outlined in the Commemorative Integrity Statements (CIS) for Vimy Ridge (2005) and Beaumont-Hamel (2019). For their purposes, Parks Canada considers any Commemorative Integrity Statement older than 10 years outdated.^{Footnote 14} Signed-off by the Minister for VAC and the Minster responsible for Parks Canada, these foundational documents were developed with the intent to guide the management of these National Historic Sites. It was noted by the evaluation team that the Management Plans for National Historic Sites under the administration of Parks Canada are accessible online while VAC’s Management Plans are not online.^{Footnote 15} The evaluation team noted in reviewing Vimy’s Commemorative Integrity Statement the historical value of the preserved trenches and tunnels are documented as significant vestiges and evidence of the First World War.

There is a dual role in managing the memorial sites. It is important to balance both the real property obligations with conservation of cultural resources as stated in the Commemorative Real Property and Other Holdings policy^{Footnote 16}. Specifically, the policy states that VAC must, “ensure maintenance, preservation and presentation of Canada’s international monuments and memorials and battlefields.” Through key informant interviews, the evaluation team noted that work is underway to strengthen working relationships with VAC’s real property group in head office. Also, the evaluation team observed the importance of strengthening working relationships with other areas of the department, including procurement, policy and IT to support commemorative and real property initiatives.

Also, as stated VAC’s 2020-2030 Strategic Plan for Commemoration VAC plans to strengthen relationships with federal partners to align programs, policies and procedures for greater efficiency. For example, VAC can work more closely with:

• Parks Canada who is the Government of Canada expert in cultural resource management and interpretation;
• Global Affairs Canada possesses expertise in the Foreign Service Directives for public servants working outside Canada and is also the common service organization for real property outside Canada;
• The Department of National Defence’s Directorate of History and Heritage possesses expertise and delivers programming related to military history as well as military medals and decorations; and
• The Department of Canadian Heritage leads the promotion of Canada’s heritage and culture, including collections management for museums and the preservation and conservation of cultural objects and artefacts.

4.2.3.2 Increasing operating costs for European Operations

European Operations relies heavily on operating funds and key informants noted that for many years they have not a sufficient level of stable funding to meet operational requirements. European Operations’ annual operating budget for VAC’s memorials in Europe is approximately $4 million while spending levels for 2018-19 to 2020-21 have averaged $4.6 million per year. The majority of the annual budget is committed to non-discretionary expenses like utilities and sewage; information technology infrastructure; vehicle and machinery maintenance; non-negotiable foreign service employee allowances; security contracts, cleaning, maintenance and infrastructure repair. The Government of Canada invested $5 million to construct a new Visitor Education Centre at Vimy. This was completed in 2017 to coincide with the 100th year anniversary of the battle of Vimy Ridge. VAC was not allocated any additional funds to operate the Centre and maintain, upgrade or develop new exhibits.

The Commemoration Division also relies heavily on operating funds to deliver core elements of the Canada Remembers Program, including European Operations. This presents a risk that core programming could be affected if departmental operating funds are reduced. Also, since operating funds are requested through annual internal integrated business planning, funding levels to meet conservation and visitor experience objectives could be reduced based on other departmental needs.

4.2.3.3 Visitor experience

A high-quality visitor experience is a key objective of VAC’s European Operations. A strong visitor experience is recognized as an essential, baseline industry standard for historically and culturally significant sites and museums in Canada and around the world. At VAC’s two National Historic Sites, this encompasses everything from how guests are greeted to signage, the exhibits and experiences they encounter during their visit, as well as available special events, digital and multimedia supports, and educational programming. At VAC’s smaller memorial sites it may be as simple as ensuring bilingual interpretive signage is visible to explain the significance of the site.

In 2019, a visitor experience survey was implemented at the Vimy and Beaumont-Hamel sites to gain knowledge about the people who visit and about their experiences. Some highlights from visitors surveyed at Vimy are outlined in Table 11.

Table 11: 2019 Visitor Experience Survey Results – Vimy and Beaumont-Hamel

The percentage of survey respondents who took a guided tour was further explored in the survey. There was very high satisfaction for the tour content, length and delivery (i.e., overall results were 4.9 out of 5). While the uptake on the guided tours is moderate at Vimy and low at Beaumont-Hamel, there were a variety of reasons noted for this as outlined in Table 12.

Table 12: Reasons noted for not taking a guided tour

At both Vimy and Beaumont-Hamel, survey respondents had high levels of agreement (i.e., Totally Agree) that as a result of their visit they had a better appreciation of the sacrifice and contribution of those who served and of the significance of the sites: 84% at Vimy and 86% at Beaumont-Hamel. This is evidence that the memorial sites are offering a highly valuable experience and are important to bring awareness and appreciation of the sacrifices and achievements of Canada’s military.

Visitors were also asked about ways to enhance their experience at the sites. The top two responses for both Vimy and Beaumont-Hamel in terms of level of importance were self-guided tours with smartphones, and more interpretive panels. These items were also noted in the Innovation Hub’s work with European Operations in 2020 and in the 2020-2030 Strategic Plan for Commemoration. The evaluation team noted through a review of other World War commemorative sites in France, such as the Sir John Monash Centre (Australian National Memorial) and the British Normandy Memorial that as a result of substantive investments these two locations now offer a visitor and education experience that is modern, ever evolving and functional, including updated interactive systems that guide and educate visitors as they make their way around the sites. Without a new and continued investment, Vimy and Beaumont-Hamel is at risk of falling behind for not having a more modern visitor education experience.

4.2.3.4 Forestry management plan

A forest management plan for Vimy has been an ongoing topic for departmental discussion since 1985. In 2021, a partnership with Natural Resources Canada (NRCan) was implemented to support the maintenance, replanting and long-term sustainability of wooded areas and forests at Vimy and Beaumont-Hamel─290 acres. This partnership with NRCan supports the project entitled, Forest Conservation – Canada’s FWW Memorials in Europe and is guided by an inter departmental Memorandum of Understanding. VAC will receive $2.1M from 2021-2022 to 2025-2026, with additional funding of $1.6M to be tabled in Parliament in 2026. VAC and NRCan will revitalize the long-term stewardship of Canada’s First World War memorial sites in Europe through the maintenance and conservation of forests and wooded areas. This will include the maintenance, replanting and long-term sustainability of wooded areas and forests at Vimy and Beaumont-Hamel according to best forestry practices. Key attention will be given to the conservation of historic battlefields and commemorative view planes (ex: treed alleys and lanes) as defined in the Commemorative Integrity Statements for each memorial site.

Key informants stated the project is well underway:

• Forestry equipment has been purchased
• In the process of hiring the permanent team of foresters via the Commonwealth War Graves Commission (CWGC)
• Developing the forest management plan for the next 30 years with partners (CWGC and the Office national des forêts).
• Begun clearing small parcels at Vimy.
• Prepared the terrain and replanted more than 650 trees
• Hired a company to undertake a flora and fauna inventory of the Vimy site.
• Put in a place a five-year contract with a depollution (demining) company to work in areas where there will be planting to ensure there are no unexploded ordinances.

4.2.3.5 Comparison with Parks Canada

As of 2020, there are 999 designated National Historic Sites^{Footnote 17} and of these Parks Canada administers 174^{Footnote 18}. The protection and presentation of these 174 sites are the mandate for Parks Canada^{Footnote 19}. The remainder of the more than 800 sites are administered by various organizations including other federal government departments, provinces, territories, and community groups. As mentioned above, VAC manages the only two National Historic Sites outside of Canada—both rate amongst the top ten most visited National Historic Sites when compared to those under Parks Canada administration. At Parks Canada, they have organized administration of National Historic Sites under two sub programs^{Footnote 20}:

1. National Historic Site Conservation^{Footnote 21}
2. National Historic Site Visitor Experience

These organization of these two sub-programs reflect the same strategic priorities held by VAC’s European Operations directorate. Each of the Parks’ National Historic Site sub programs have logic models with outcomes and performance indicators specific to the administration of National Historic Sites. This level of accountability that is built-in to the governance of a sub-program maps how investments and resources are linked to activities to achieve the desired results.^{Footnote 22}

The National Historic Site Conservation sub-program has aimed at ensuring that:

• interventions likely to impact on cultural resources are assessed and mitigated;
• condition of built assets at national historic sites is maintained or improved; and
• national historic sites administered by Parks Canada are managed in ways that leave their commemorative integrity unimpaired.^{Footnote 23}

The National Historic Site Visitor Experience sub program has outcomes aimed at ensuring that:

• an increase in the number of Canadian and International visitors to National Historic Sites;
• visitors to National Historic Sites connect to and learn about the cultural heritage of the site; and
• Visitor Experience assets are maintained and improved.^{Footnote 24}

Comparatively, VAC’s National Historic Sites fall under the overarching Canada Remembers Program and just one PIP and logic model. There is only a single indicator on visitation rates linked to its two National Historic Sites of significance. In 2020-21, an outcome on visitor experience was added to the Departmental Results Framework with results from the annual visitor survey included as an indicator. It has yet to be added to the PIP. The evaluation team noted that there are no indicators specific to conservation.

The evaluation team compared full-time equivalent staffing levels at VAC’s two national historic sites of significance (Vimy and Beaumont Hamel) with two Parks Canada national historic sites with similar visitation rates (Halifax Citadel, Nova Scotia and Fort Langley, British Columbia). Maintenance workers and students were excluded from the comparison as they are typically part-time or under contract as in the case of maintenance work for VAC. This comparison used the VAC organizational chart for European Operations and the Government Electronic Directory site for Parks Canada. Overall, VAC has seven^{Footnote 25} full time equivalent (FTE) staff focused on their two national historic sites while Parks Canada has 22 at the two comparative sites. Additionally, Parks Canada has regional field units that offer further support and expertise to the sites, totaling an additional 65 staff.^{Footnote 26} See Table 13 below for details.

Table 13: Parks Canada and VAC national historic sites staffing comparison based on similar visitation rates

*see Annex A for detailed list of most visited national historic sites.

**Additionally, these Parks' National Historic Sites have field unit staff that support them. For example, the Mainland Nova Scotia field unit has 33 FTEs that supports a total of five parks and National Historic Sites which includes the Halifax Citadel. The Coastal British Columbia field unit has 32 FTEs that supports a total of four parks and National Historic Sites which includes Fort Langley.

4.2.4 Learning and public information

VAC engages school aged children, youth, educators, and the general public through learning and public information resources. These include Veterans’ Week print and online materials. Canadian teachers can order print materials for their classrooms that include lesson plans. As well, resources can be found on the VAC website and VAC social media channels. The VAC website features an online home learning space that includes lesson plans, and an online library featuring personal recollections of those who served (Heroes Remember).

The 2018 - 2020 Attitudes Towards Remembrance and Veterans’ Week surveys asked Canadians to rate the level of priority the Government of Canada should place on remembrance initiatives to engage Canadians. Table 14 outlines the results for several initiatives that VAC undertakes annually.

Table 14: Public information related survey results

Source: Attitudes Towards Remembrance and Veterans’ Week Surveys 2018, 2019, and 2020

The continued need for the learning and public information component was highlighted in interviews with VAC staff and through feedback from educators. Annual educator evaluation forms reported that 99% of those surveyed plan to use VAC-produced learning resources again in the future. Table 15 depicts results from annual educators surveys.

Table 15: Feedback from educators on Veterans Week

The Learning and Public information section provides free, bilingual Veterans’ Week learning materials for educators and youth. Table 16 shows the types of products produced and the volume of each per year since 2019.

Table 16: Learning and public information resources

In interviews with staff who produce the learning and education content it was noted advances have been made to improve the availability of content on VAC’s website, including the addition of a Classroom Materials section on the Remembrance homepage. Some materials are still only available via DVD. Teachers have consistently responded that they want to continue with the printed education literature mailouts as it is something that creates interest with the youth and something that can be shared with their families.

4.2.4.1 Future commemorative programming for learning and public information

The 2020-2030 Strategic Plan for Commemoration outlines an objective to provide innovative learning materials to help educators teach youth about the role of Canada’s military personnel in the world and their achievements and sacrifices. To meet this objective, the plan identifies the following initiatives:

• Develop, test and release The Canadian Armed Forces around the World multimedia learning resource. This tool will help teachers, students and the public learn more about Canada’s military policing and humanitarian efforts during the past seven decades;
• Conduct an environmental scan and user research to determine needs and priorities of Canadian teachers. Best practices will be used to create learning products that meet teachers’ current and anticipated needs;
• Engage with education students in universities with the aim of instilling an interest in, and capacity to integrate, remembrance activities into their future teaching activities.
• Design and test new learning products. VAC will work with partners to take advantage of resources and expertise; design, develop and test of new products; and share them for feedback from the broader community of educators;
• Partner with universities and leaders in science and technology to develop innovative, experimental tools and approaches to bringing remembrance into Canadian classrooms; and
• Expand the capacity for virtual tours of memorial sites as only a small percentage of youth in Canada are able to visit sites in person.

To help refine the 2020-2030 Strategic Plan, in 2020 VAC consulted with educators regarding the future of remembrance in the classroom. The report of the consultations noted that educators practice traditional remembrance activities and mark major military anniversaries on the dates they occur (e.g., Remembrance Day). The findings in the report affirmed that most educators prefer printed materials (88%) and hands-on learning activities (86%).

The report made several recommendations, including:

• Continue providing printed materials and other learning resources (either hardcopy or downloadable) related to remembrance activities prior to Remembrance Day and other meaningful dates throughout the year, while introducing new technologies gradually.
• Consider simplifying, streamlining or providing options for customizing the Canadian Armed Forces Around the World multimedia resource kit so that educators can use the material in a wide variety of settings with different age groups and abilities. Also, provide options for distributing learning materials to communities and schools that do not have full access to all technologies and for students who prefer a hands-on experience and are primary school age.
• Promote learning activities that will engage students (e.g., animation, films, media, short activities, the use of artefacts, activities with movement, photo galleries, pedagogical games, tactile resources, infographics, escape room activities, a remembrance app, etc.).
• Promote materials that teach children about the contributions of Indigenous peoples, racialized peoples, LGBTQ2+, and other minorities.
• Develop materials for educators to help them teach students about sensitive topics such as the Holocaust, the genocide in Rwanda and ethnic cleansing.

Opportunity:

Prioritize and action the findings from consultations with educators to modernize and update the educational content in line with the objectives in the 2020-2030 Strategic Plan.

4.2.5 Innovation and technology

In 2019 VAC established an Innovation Hub to lead the department in innovation and human-centred design. The Commemoration Division worked with the Innovation Hub on two projects related to CPP and to European Operations.

4.2.5.1 Commemorative Partnership Program

Innovation Hub work on CPP focused on ways to reduce the time and effort required to process and report on applications and support the intake and assessment of “complex” applications. This work resulted in the following recommendations:

• improve the application form to ensure applicants only submit complete applications digitally
• implement a new software solution to manage application intake through to evaluation and reporting
• triage applications to streamline time and effort to invest in the evaluation of higher risk and higher cost applications
• streamline internal review process
• prioritize and consolidate Quality Control efforts for the most important content and plan for one round of Quality Control early in the process so it can be easily leveraged confidently later on, and
• streamline the decision-making process based on application risk to speed up administration and reduce response times.

Since the completion of work with the Innovation Hub, the CPP team has been working to move the program online, from application to reporting. This work is underway and is expected to be completed by the end of 2022; therefore, the evaluation team was not able to assess the improvement to the delivery of the CPP. Work to implement the other recommendations remains outstanding.

Advances in innovation and technology are also occurring with regards to commemorative initiatives that are supported throughout the CPP. In interviews with staff, the evaluation team heard VAC is receiving more applications for CPP funding that are using technology in their remembrance initiatives. Staff indicated that these are often higher dollar initiatives and are more complex than traditional projects. This presents an increased level of difficulty to assess for value and with a small budget to work with this has added pressure to meet demand.

4.2.5.2 European Operations

In completing work with the European Operations directorate, the Innovation Hub recognized going to the sites in Europe where Canadians fought and are memorialized is a powerful and important experience. However, many Canadians have not and will not travel to Europe to see the sites in person. The 2019 Vimy and Beaumont-Hamel Visitor Survey stated 22% of visitors to Vimy and Beaumont-Hamel are Canadian. As a result, the Innovation Hub team, in collaboration with European Operations staff recommended bringing the Vimy and Beaumont-Hamel experience home to more Canadians through digital platforms. A second opportunity was validating a concept involving the Student Guide Ambassador Program. This concept would engage former guides in European Operations activities in support of bringing the commemorative sites home to Canadians. This could include sharing past experiences, participating in live Q&As, engaging with students in the classroom and joining online tours.

The Innovation Hub’s work was seen as first steps and key informants noted that further discussion was needed before proceeding further. Key informants also noted that an additional investment in resources and expertise would be needed to action these two key recommendations.

4.2.5.3 Learning and public information

In interviews with staff who produce the learning and education content it was noted that advances have been made to improve the availability of content on the department’s website, including the addition of a Classroom Materials section on the Remembrance homepage. During COVID-19 the ‘Home Learning Corner’ was created to assist teachers, educators and parents who homeschool. There was an increase in orders from homeschoolers, which typically involve requests for a small number of individual copies of resources (as opposed to class packs of 25 or 30). The team pivoted to smaller requests by reducing the size of the packages and they plan to continue to provide these smaller packages.

Staff in the Learning group stated in interviews that they recognize the need for new technology and to move things to the “cloud” for example. This would allow them to provide more innovative content. Some materials are still only available via DVD, while other materials are available on a USB stick. There is an opportunity for the Learning and Education program to incorporate new and more technology in the delivery of resources to schools and the Canadian public, while maintaining the print materials.

There is an opportunity for continued and new efforts to modernize and update educational content in a format that is compatible with multi-media sources.

4.2.5.4 Ceremonies and events

The COVID-19 pandemic led to significant changes in how VAC supported and led commemorative events in Canada. Most in-person events were cancelled in 2020 and 2021. Ceremonies and events staff, along with colleagues in Communications Division adapted to deliver virtual events, both live and pre-recorded. Staff who led the events worked with Information Technology staff to ensure major milestones and anniversaries were commemorated and available for Canadians to view and participate in. Interviews with staff noted that additional expertise was needed to conduct the virtual events (e.g., knowledge and expertise in the use of the technology). In 2021, smaller scale in-person events were held in some provinces where permitted. Staff noted in interviews that if a return to in-person events at the same level as before the pandemic was to happen and that virtual events would also continue, then additional resources would be needed to ensure success.

The 2020-2030 Strategic Plan for Commemoration states VAC must continue to embrace new technologies to help meet the evolving needs and expectations of Canadians. The Plan also recognizes improvements so VAC can be the first stop for people seeking information on military history, planning a visit to an overseas memorial, or planning a commemorative event. Staff noted in interviews that to meet these goals, additional expertise in Information Technology and capacity to incorporate the new technologies into existing programming will be required.

4.2.6 Commemoration communications

4.2.6.1 Advertising and social media

VAC’s Communications Division has a significant role in promoting and communicating remembrance to Veterans and Canadians.

VAC uses advertising in traditional media along with advertising across social media platforms to reach Veterans and Canadians. Social media is being used to promote the stories, sacrifices and achievements of all Veterans and those who died in service, and has recently placed more emphasis on modern day and underrepresented Veterans. Social media content that VAC produces consistently draws interest from the Canadian public, especially during Veterans’ Week and other significant milestones that are recognized.

VAC runs an advertising campaign each November focused on remembrance. Communications Division leads this initiative with support from the Commemoration Division. The evaluation team reviewed results of the 2020 advertising campaign. The objectives of the campaign were to:

• engage Canadians of all ages in acts of remembrance;
• focus on younger Canadians and new Canadians; and
• drive target audiences to VAC’s social media sites or the campaign landing page: veterans.gc.ca/CanadaRemembers

The target audiences were:

• Canadians aged 18 to 54 years, with emphasis on young Canadians (18 to 34),
• Canadian Veterans and their family members,
• CAF members and their families, and
• Canadian Veteran community stakeholders and partners.

The campaign included a variety of initiatives across several types of media, including:

• Television
• Digital media
  • Advertising on popular Canadian websites
  • YouTube homepage videos and ads before videos start
  • Connected TV (i.e., device that supports video content streaming)
  • Spotify
• Social media
  • Twitter - brand reminders, promoted poll, website clicks, in-stream video
  • Snapchat – commercials, filters, lens
  • Facebook and Instagram - newsfeed and stories

Results for the 2020 Remembrance Day campaign indicated^{Footnote 27}:

To support the advertising campaign, VAC also employed an outreach strategy in which prominent Canadian organizations and individuals were engaged to share the campaign message among their audiences and followers (for no cost).

Results indicate an increased appreciation and interest for the efforts of Canadian Veterans and those who died in service to Canada. Through social media, VAC engages Canadians and provides a forum for the public to show and share their appreciation and interest in the efforts of Canadian Veterans. The evaluation team reviewed VAC’s social media content. There is evidence of engagement by Canadians through their comments, “likes” and sharing of the posts. VAC has been a leader in terms of government departments using Facebook and Facebook is by far VAC’s most popular social media platform with nearly 900,000 followers of the Canada Remembers page. This compares to approximately 80,000 followers on Twitter and 23,000 on Instagram. Comments on the Canada Remembers Facebook were reviewed and were positive, respectful, and appreciative of Canadian Veterans and their sacrifices.

VAC’s Communications Division also collaborates with Commemoration Division for ceremonies and events. During the COVID-19 pandemic, Communications Division worked with Commemoration Division to implement virtual events and ceremonies and to move more activities into the digital world, both live and pre-recorded. Interviews with staff noted that the expansion into virtual events presented an opportunity to reach more Canadians through new ways of promoting remembrance content. Staff also noted that more Information Technology expertise is required to conduct high quality virtual and digital events.

4.3 Program economy

4.3.1 Operating costs and level of administrative effort

A breakdown of Canada Remembers funding by program/initiative is provided in Table 17 below:

Table 17: Canada Remembers Program expenditures

* Excludes funding provided to the Invictus Games that was paid through the CPP.

** Excludes the Métis Veterans Recognition Payment.

***In addition to the memorials managed directly by VAC, the preservation of memorials and maintenance of cemeteries/graves of Canadian and the former Dominion of Newfoundland forces who died in the two World Wars and who are buried outside of Canada is provided by the Commonwealth War Graves Commission (CWGC). Canada is one of the six partner countries (Australia, India, New Zealand, South Africa and the United Kingdom) providing funding to the CWGC. The funding provided is proportional to the number of graves with Canada sharing approximately 10% of CWGC costs (currently this amounts to approximately $11 million). In addition, VAC has an Agency Services Memorandum of Understanding with the CWGC for the horticultural and maintenance requirements of VAC’s memorial sites in Europe. The Commission for the United Nations Memorial Cemetery Korea (UNMCK) provides cemetery and grave maintenance for Canada's war dead buried in Korea. The Commission for the UNMCK consists of 11 member nations (Australia, Canada, France, Netherlands, New Zealand, Norway, South Africa, Turkey, UK, USA and Korea) whose fallen during the Korean War are interred at the UNMCK.

The table depicts fluctuations between operating and maintenance and human resource expenditures. The evaluation team reviewed annual expenditures and found that in 2019-20 there was a significant increase in funding for cemetery and grave marker maintenance and for overseas events commemorating the 75th anniversary of the Second World War. In 2020-21 expenditures were impacted by the COVID-19 pandemic. The Commemoration Division relies heavily on operating funds to deliver core elements of the Canada Remembers Program (e.g., overseas memorials, ceremonies and events, learning and public information, and cemetery and grave maintenance). This was noted in the 2018 Overarching Commemoration Evaluation, and the situation remains. The previous evaluation also noted that this presents a risk that core programming could be affected if departmental operating funds are reduced. Further, to achieve the goals of the 2020-2030 Strategic Plan for Commemoration, investments will be needed and stable long-term funding would help ensure the Canada Remembers Program is able to continue to meet its mandate.

Recommendation #1

The Assistant Deputy Minister, Commemoration and Public Affairs Branch:

1. Develop a costed operational plan that clearly identifies resources required and technical tools/enhancements to support the delivery of initiatives within the Commemoration Strategic Plan or Vision. The plan should include European Operations requirements for conservation and visitor experience.
2. Implement the operational plan and where required, pursue:
   1. increasing physical and technical capacity to better manage and deliver projects and initiatives; and
   2. securing stable long-term funding to efficiently plan and implement day-to-day work in Canada Remembers including European Operations.

Management response: Veterans Affairs Canada agrees with the recommendation.

Recommendation #2

The Director General, Commemoration Division review and revise the Canada Remembers Program Performance Information Profile (including the logic model) to ensure:

1. Activities, outcomes, targets, tools and data sources are updated, and align to the strategic vision for Commemoration;
2. Outcomes and performance indicators are developed specific to overseas memorial sites (National Historic Sites), including cultural integrity, conservation and visitor experience; and
3. Performance indicators measure the program’s success in supporting reconciliation efforts and incorporate Gender Based Analysis plus.

Management response: Veterans Affairs Canada agrees with the recommendation.

Recommendation #3

The Director General, Commemoration Division strengthen internal partnerships (real property, procurement, IT, and policy) to establish clear roles, responsibilities and accountabilities to better support the Commemoration Division, including the European Operations Directorate.

Management response: Veterans Affairs Canada agrees with the recommendation.