4.0 Program effectiveness

4.1 Achievement of expected outcomes

Performance Information Profiles for the Funeral and Burial Program and the Canada Remembers Program were last amended in 2017. The performance measures identified in both profiles are being collected and monitored by VAC Commemoration division.

The Performance Information Profiles primarily use two surveys to collect VAC client satisfaction and Canadian public opinions:

• VAC National Client Satisfaction Survey (survey of representative sample of VAC clients)
• Attitudes Towards Remembrance and Veterans Week 2020 Survey (survey of general Canadian population);

The National Client Satisfaction Survey targets VAC clients, but does not target Funeral and Burial benefit recipients. The survey also does not capture feedback from the entire Canadian Veteran population (who also may be eligible for the program at their time of death). Given this information, the survey does not provide a fulsome picture to measure performance for the Funeral and Burial Program.

4.1.1 Funeral and Burial Program effectiveness

The objective of the Funeral and Burial Program is to ensure Veterans who die as a result of their service or without financial means receive assistance for a dignified funeral and burial.

The desired outcome of access to financial assistance for a dignified funeral and burial is based on whether Veterans’ families/estates are aware of the program and therefore have access. The supporting performance measure for this outcome within the Performance Information Profile is a survey question in the VAC National Client Satisfaction Survey.

The 2020 survey asked respondents, “Please tell me how much you agree with the following statement: “I am satisfied that the estates of Veterans have access to financial assistance through VAC’s Funeral and Burial Program.” As can be seen in Table 8, of the 2,460 respondents, 67% agreed/strongly agreed with this statement. This is down from 75% in 2017-18, and below the target of 70%. The shifting population base from older Veterans who are more likely to be satisfied (80% for those 90 years and older), to younger Veterans who are generally less satisfied (59% for those under 40 years old) may be responsible for this downward trend. It should be noted that while the level of satisfaction is lower than targeted, there are few respondents reporting dissatisfaction with access to financial assistance through the Funeral and Burial Program. As shown in Table 8, only 10% of survey participants responded unfavourably, while 23% of responders remained neutral.

Source: 2020 VAC National Survey Data Tables and 2020 VAC National Client Survey Final Report

4.1.1.1 Program awareness

The Last Post Fund has a role in promoting the Funeral and Burial Program. Interviews with Last Post Fund staff and their annual report indicate that there are challenges to ensuring Veterans and their next of kin are aware of funeral and burial benefits. Since the 2017 evaluation, Last Post Fund staff have indicated that they are focussing their marketing strategy to improve awareness of program benefits. Raising awareness of the Last Post Fund brand is a primary objective in their Strategic Action Plan. The Last Post Fund’s communication/marketing approach includes print ads and articles in Canadian Funeral and Legion magazines, community newsletters, posters in CAF regiments across Canada, posters/cards in homeless shelters, e-newsletters with Last Post Fund members, and a social media presence (2,000+ Facebook followers). Volunteers across Canada aid in promoting benefits to local Veteran organizations and attending funeral industry trade shows.

Following a recommendation from the 2017 VAC evaluation, VAC developed reports detailing VAC clients who had an active homeless indicator or War Veterans Allowance (low-income) indicator at the time of death. Clients with these indicators have an increased likelihood of Funeral and Burial Program eligibility. These reports are forwarded to the Last Post Fund for follow-up with the Veterans’ next-of-kin/estates to make them aware of the program eligibility. These reports will be discussed in more detail in section 5.0, Program efficiency.

Between 2017-18 and 2020-21 an average of $5.7 million was provided to Funeral and Burial recipients. The evaluation determined that means-tested Funeral and Burial benefits were paid within 30 days 90% of the time during the period of the evaluation (performance target is 80%).

In 2020-21 a new indicator was included in the Performance Information Profile to assist in measuring this outcome: the number of matter of right payments issued within the 30-day turn-around time. Performance reporting shows that 100% of matter of right reimbursements were made within 30 days of the Last Post Fund receiving supporting documentation.^{Footnote 16}

To determine whether the financial support offered through the Funeral and Burial Program is meeting the needs of recipients, the evaluation team spoke with Last Post Fund and VAC Program staff and reviewed Last Post Fund client survey responses. No concerns were raised. The Last Post Fund survey does not have a question regarding satisfaction with financial support, however, survey comments from many respondents expressed thanks and gratitude for the level of support they did receive.

The ultimate outcome is measured using the annual Attitudes Towards Remembrance and Veterans’ Week Survey. Canadians are asked if, “It’s important that VAC recognize and honour deceased Canadian Veterans by providing funeral and burial assistance”. Results show consistently positive feedback with 90+% of participants from 2017-18 through 2020-21 completely or somewhat agreeing with this statement.

4.1.1.2 Benefit recipient feedback

Since 2018, the Last Post Fund has been surveying applicants regarding their service experience. There is a good response rate for the survey (40-46% for approved cases and 13-23% for denied cases). Just over half of respondents indicate that they heard about the Funeral and Burial Program from VAC. Comments received through the survey responses indicate general satisfaction with the Funeral and Burial Program. Some comments specifically highlight sentiments of honour:

“We were so happy and proud that this service is available as the funeral costs were a hardship and the help really went a long way in helping my mother with the costs. Your assistance is appreciated more than words can say.”

“Thank you for your help and assistance in helping to provide my Dad a funeral of dignity and respect for a member of the armed forces.”

“So glad you are there to help in a difficult time and to help with a proper funeral and resting place. Thank you from me and my family.”

The evaluation team noted that there is an opportunity to collect additional information on the success of the Funeral and Burial Program through the Last Post Fund’s client service survey. The Survey is currently focussed on collecting feedback on the client service experience with Last Post Fund. With a high response rate (43%), and engagement from survey participants, there is potential to capture feedback from program recipients such as:

• Satisfaction with the level of funding to support a dignified burial for the Veteran;
• Satisfaction that the Veteran was recognized and honoured for their service; and
• If the funding hadn’t been available, would this have impacted the ability to have a dignified burial for the deceased Veteran.

4.1.2 Cemetery and Grave Marker Maintenance Program effectiveness

The Cemetery and Grave Marker Maintenance Program falls under the general Canada Remembers Program Performance Information Profile. Canada Remembers Program activities ensure that those who served in Canada’s military efforts are honoured and that the memory of their sacrifices and achievements is preserved. Specific activities include ceremonies and events, honours and awards, and the presentation and perpetual care of memorials, cemeteries and grave markers.

The Cemetery and Grave Marker Maintenance Program publicly acknowledges Veterans who died through the care and maintenance of military grave markers across Canada.

The evaluation finds that VAC Commemoration staff are committed to ensuring that the program effectively honours Veterans and those who died in service and that the memory of their achievements and sacrifices is preserved.

Results from the Attitudes Towards Remembrance and Veterans’ Week 2020 Survey indicate that 64% of Canadians are satisfied with “how Veterans Affairs Canada recognizes and honours Canadian Veterans and those who died in service through the presentation and care of memorials, cemeteries and grave markers”. Satisfaction rates for respondents to this question ranged between 59-64% during the evaluation scope period. The target for this performance measure is 65%. There is no additional information to draw any conclusions on this statement. The question is not specific to cemeteries and grave marker maintenance (e.g., there is reference to memorials as well). Therefore, the evaluation team is unable to determine to what degree cemeteries and grave marker maintenance contributes to or contradicts this satisfaction rate.

Again, the immediate outcome is tied to the overall Canada Remembers initiatives and the evaluation team is not able to isolate the performance measures specifically to the Cemetery and Grave Marker Maintenance Program.

The Attitudes Towards Remembrance and Veterans’ Week Survey is used to measure the immediate outcome of the program. The survey determines the percentage of Canadians who agree that VAC’s remembrance program^{Footnote 17} effectively honours Veterans and those who died in service and preserves the memory of their achievements and sacrifices. Over the evaluation scope period, 70-74% of Canadians surveyed agreed with this statement. The performance target for this indicator is 80%.

The value Canadians (including family members) place on the maintenance of Veteran graves is evidenced by the significant increase in calls to the public grave marker maintenance reporting phone line (Table 7 – Total enquiries for grave marker maintenance from general public, section 3.1.2.2). Enquiries have more than tripled over the period of the evaluation.

The Department uses web analytics to measure web page activity within the VAC website. Over the period of the evaluation, between 3 million and 3.5 million visits were made per year to VAC’s main memorial and cemetery web pages. The evaluation team notes that visits have decreased between 2017-18 (3,554,090) and 2020-21 (2,949,115). Web activity will fluctuate in correlation with significant anniversary events (e.g., the 100th anniversary of Vimy Ridge in 2017). Though the Canada Remembers Performance Information Profile uses overall web page visits as an indicator of success for the Cemetery and Grave Marker Maintenance Program, it may not be the best measure as the overall numbers do not correlate to click-throughs to the Program’s web page. Table 9 below, shows a more accurate representation of the number of visits to the Grave Marker Maintenance web page.

Source: VAC Communications

Suggestions from VAC’s National Client Survey responders (2,618) on how to raise awareness of VAC commemorative initiatives include the use of social media (55%), local organizations and community groups (41%), and through schools (40%). Though comments are general in nature to commemorative initiatives, the top two suggestions could reasonably be applicable to the Cemetery and Grave Marker Maintenance Program.

The Cemetery and Grave Marker Maintenance Program contributes to the overall Canada Remembers intermediate outcome through the preservation of Veteran graves and two departmental cemeteries.

The 2017 evaluation found that the Cemetery and Grave Marker Maintenance Program had a significant maintenance backlog and insufficient funding allocated to address the repairs. Additional funding provided in 2018 has enabled the program area to conduct a project to reduce the backlog. As of January 2022, the project is on target to complete the backlog repairs as well as additional repair work discovered during the course of the project. The results of the backlog project are further elaborated on later in this section.

Ultimate outcomes are generally described as effecting a change in the state of a target population. The Cemetery and Grave Marker Maintenance Program contributes to the overall Canada Remembers ultimate outcome by providing locations for family and the public to visit and remember deceased Veterans. Cemeteries provide an opportunity for the families and the public to honour Veterans on their own time and in numerous locations across the country.

In 2020-21, 71% of Canadians surveyed through the Attitudes Towards Remembrance and Veterans’ Week 2020 Survey indicated that they make an effort to demonstrate appreciation by wearing a poppy, watching a remembrance ceremony on TV, attending a ceremony, participating via social media, or participating in another activity to remember. Visiting a cemetery is not specifically mentioned in this survey question.

The VAC 2020 National Client Survey collected data on the satisfaction with VAC’s commemoration initiatives^{Footnote 18} and the way they honour those who served. Overall, 71% of respondents were satisfied (down 5% from 2017-18). Younger Veterans (under 40) and Indigenous responders were less likely to be satisfied, while older Veterans, women, and French-speaking responders were more likely to be satisfied with VAC commemorative initiatives. The evaluation team is unable to provide additional information as to the reason for the difference in opinions, or to what extent cemeteries and grave marker maintenance may affect the response rates. The target for this measure is 80%.

Though not specific to Cemetery and Grave Marker Maintenance, 73% of Canadians surveyed through the Attitudes Towards Remembrance and Veterans’ Week 2020 Survey agree that VAC’s remembrance program effectively honours Veterans and those who died in service. The target for this measure is 80%.

Both measurements have consistent results from 2017-18 through 2020-21. The evaluation team is unable to speak to the rationale of Canadian and Veteran opinions on remembrance initiatives, and to what degree cemeteries and grave makers contribute to these rankings. However, the level of public opinion on the satisfaction with VAC’s commemorative initiatives indicates that there is room for improvement in informing Canadians on the work completed in the area of cemetery and grave marker maintenance, especially in the last four years.

4.1.2.1 Service standards

Internal reporting indicates that public inquiries regarding the need for care and maintenance to Veteran graves are being responded to within VAC’s five-day service standard almost 100% of the time. Additionally, the current service standard for required grave marker maintenance action is that work be completed within 12 months of the request. Internal reporting shows this standard was met 90-100% of the time between 2017-18 and 2020-21 (see Table 10).

Source: VAC Commemoration Division

4.1.2.2 Success of the grave marker repair backlog project

The Program Area’s commitment to success is evident in the work over the last four years to eliminate the backlog of grave marker repairs, especially during a worldwide pandemic. Project funding of $24.4 million enabled Commemoration to bring on additional temporary staff and provide additional funding to the Commonwealth War Graves Commission to perform grave marker inspections and repairs.

Interview results and project reporting indicate that the backlog of repairs is being sufficiently addressed. Additional funding provided to VAC for the project is allowing the Program Area to address approximately 57,000 repairs plus an additional 37,000 maintenance repair issues discovered due to the increased project inspections. For comparison, the total number of repairs completed in the previous evaluation four-year period (2012-13 to 2015-16) was 13,388.

The additional inspections and corrective actions could not have occurred within the pre-project program funding level ($1.25 million). Though the types of repairs were not tracked for the project, interviews suggest early intervention may have averted future maintenance issues (i.e., fixing a problem before it grows larger and becomes a costly expenditure). The additional repair work completed with the funding provided could be considered a positive unintended impact, as it was not part of the original project plan.

The backlog project is focusing on identifying and completing repair work. On a go forward basis, collecting and reviewing additional performance information could support program management and decision making and inform future evaluations. For example:

• Number of grave markers inspected;
• Number of grave markers requiring repair work; and
• Repair type (e.g., cleaning, alignment, replacement).

Details such as those noted above could help inform project cycles and resource needs. The backlog project is further elaborated on in section 5.2.1, Grave marker maintenance backlog project.

Opportunity – There is an opportunity for VAC to track and measure the success of the Cemetery and Grave Marker Maintenance Program separately from the overall Canada Remembers Program. As the backlog project concludes, measuring targeted performance information would be helpful for future program management and decision making, as current measures are general in nature and encompass overall remembrance initiatives.

4.2 Unintended impacts

The deadline for submitting a means tested funeral and burial application may unintentionally pose a barrier for some families/estates attempting to access the Funeral and Burial Program. According to the 2005 Veterans Burial Regulations, no financial assistance is payable [for last sickness, funeral, burial and cremation costs] “unless an application is made within one year after the date that the remains of the deceased person become available for interment”. In contrast, an application for funeral and burial benefits due to a service-related condition (matter of right) has no deadline.

A review of other VAC programs and benefits indicates application deadlines range from six months to “no deadline”. Many VAC programs provide benefits to Veterans on a go-forward basis and recipients do no have out-of-pocket expenses. However, VAC programs and benefits that have a reimbursement component similar to the Funeral and Burial Program (like Treatment Benefits and the Veterans Independence Program) generally give applicants 18 months to apply for reimbursement.^{Footnote 20}

The evaluation reviewed practices outside the Department for guidance on what might constitute a more suitable application deadline for the Funeral and Burial Program. An examination of foreign departments of Veterans affairs programming revealed that families/estates in other allied countries have between three months (United Kingdom) and an unlimited time (New Zealand) to request reimbursement for funeral and burial expenses.^{Footnote 21} Contrast this with the most recent Last Post Fund annual report wherein it was recommended that the application period be increased to at least three years to provide a more compassionate approach.

According to interviews with Last Post Fund stakeholders and ad-hoc statistics captured by the organization, there are approximately 50 applications received annually that are beyond the one-year deadline. It was also noted that there are instances where grieving families struggle with managing paperwork and applications in the first-year post death. Interviews indicate it is still too early to tell whether the current COVID-19 pandemic will have any affect on late applications in 2021-22. Counsellors with the Last Post Fund indicated minimal impact on funerals due to the pandemic, with some celebrations for life being impacted (due to restrictions on attendance numbers).

Given that the Last Post Fund favourable decision rate is approximately 50% for the means-test application stream, the evaluation team estimates that an additional 25 applications may be approved annually if the one-year deadline is relaxed. Considering the maximum eligible amount of $7,376, the total financial impact of such a change has an upper estimate of $184,400 per year.

Opportunity - The evaluation finds that there is an opportunity to better align the Funeral and Burial Program application deadline with other VAC reimbursement programs. Additional program costs would be (comparatively) low, while the benefit to individual Veterans’ families and next of kin could be impactful. This opportunity should be taken into consideration the next time revisions are made to the Veterans Burial Regulations.