The Funeral and Burial Program and the Cemetery and Grave Marker Maintenance Program were last evaluated in 2017 as part of the Evaluation of Commemorative Benefits and Services. It should be noted that the remaining components of the Canada Remembers Program will be evaluated concurrently as part of the Evaluation of Canada Remembers and is scheduled to be completed later in 2022.
2.1 Evaluation scope
The evaluation was conducted from April 2021 to December 2021. The evaluation covered the time period from 1 April 2017 to 31 March 2021 (though more current information and statistics may be represented in this report when possible).
The Canada Remembers Honours and Awards Program was included in preliminary scoping for this evaluation as it was evaluated in conjunction with the Funeral and Burial Program and the Cemetery and Grave Marker Maintenance Program during the last evaluation cycle. Due to low risk (low yearly operating expenditures) and no identified concerns, the Honours and Awards component was not fully evaluated as part of this evaluation.
Analysis in relation to the maintenance of cemeteries and graves for war dead overseas was not performed. The contract with the Commonwealth War Graves Commission for War Dead graves is a joint agreement across partner governments including Australia, Canada, India, New Zealand, South Africa and United Kingdom (Canada’s funding portion represents approximately 10.07% overall). Assessing the value for money or service offered through this contract may be better suited as a joint initiative across partner governments rather than an internal evaluation.
2.2 Evaluation objective and questions
Treasury Board of Canada Secretariat’s Policy on Results stipulates that all ongoing grants and contribution programs that average $5 million per year in actual expenditures must be evaluated every five years. The policy also indicates that all organizational spending and programs in the Program Inventory must be considered within departmental evaluation planning based on risk and need. Further, the Directive on Results indicates that
“…evaluations of ongoing programs of grants and contributions with five-year average actual expenditures of $5 million or greater per year required to fulfill the requirements of the Financial Administration Act section 42.1, shall include an assessment of relevance, effectiveness and efficiency”.
The past performance of the programs was reviewed along with current activities to determine program relevance, effectiveness, and efficiency. The evaluation team also explored identified areas of priority including:
- results of the grave marker maintenance backlog project;
- future operation/plans for the Cemetery and Grave Marker Maintenance Program post-backlog project;
- Funeral and Burial Program processes and changes resulting from the 2017 evaluation;
- Funeral and Burial Program reach; and
- on-going cemetery operations at God’s Acre Cemetery and Fort Massey Cemetery.
Upon completion of an initial review, evaluation questions were developed to help assess program relevance and program performance (including in the areas of efficiency and economy, and effectiveness). A list of evaluation questions, highlighting the key lines of inquiry, can be found in Table 2 below. For further details and methodologies, see Appendix A - Evaluation Matrix.
| Relevance |
| To what extent do the programs continue to address a demonstrable need, now and in the future? |
| To what extent do the programs align with Government of Canada priorities and with federal roles and responsibilities? |
| To what extent are the programs responsive to the needs of recipients and target audiences? |
| Performance |
| Have appropriate outcomes been identified and measured to determine program success? |
| Are there opportunities to improve the efficiency and economy of the programs or ways to improve the effectiveness of the program? |
| Are there any unintended impacts resulting from the programs (positive or negative)? |
2.3 Evaluation methodologies
The research methodology incorporated multiple lines of evidence, ensuring reliability of collected information and reported results. The lines of evidence used to evaluate the Programs’ relevance, performance, efficiency and economy are shown in Table 3.
Table 3 – List of methodologies
| Methodology | Source |
|---|---|
| Departmental Documentation and Secondary Research Review< | The following Departmental documents were reviewed to understand objectives/intent of the programs, their authorities and requirements, complexity, context and any key issue areas: departmental planning documents; mandate letters; previous audits and evaluations; memorandums of understanding; policies; business processes; records of decisions; strategic documents; performance reports; research papers; survey results; and client/public correspondence. |
| Non-Departmental Document Review | Various non-departmental documents were reviewed such as: parliamentary reports and transcripts; legislation and regulations; budget speeches; industry research; media documents; and Speeches from the Throne. |
| Interviews | Interviews were conducted with VAC staff, including senior management, head office and field office employees, and Departmental subject experts. Interviews were also conducted with representatives from the Last Post Fund and the Commonwealth War Graves Commission. |
| Statistical Analysis | Statistical analysis was conducted on VAC financial and operational data for the evaluation period. Financial and operational data provided by the Last Post Fund and the Commonwealth War Graves Commission was also analyzed. |
| Observation | Observation of completed and yet to be completed grave marker maintenance was conducted at several sites in Prince Edward Island. Further observation was not possible due to departmental travel constraints related to COVID-19 pandemic safety protocols. A virtual observation of the means-tested application assessment process was provided by the Last Post Fund. |
| File Review | Reviews were completed to assist in evaluating the performance, economy, and efficiency of the Funeral and Burial Program. The reviews tested the accuracy of information from both VAC and Last Post Fund systems and was used to determine if potential clients for the program are being correctly identified. The review was also used to determine if corrective actions implemented as a result of the 2017 Evaluation of Commemorative Benefits and Services have had a positive effect. |
2.4 Considerations and limitations
The evaluation identified the following considerations and limitations:
- For consideration, when analyzing data for this evaluation it is important to remember:
- some Veterans are VAC clients at the time of death (in receipt of VAC benefits or services) and therefore client information is available for analysis; and
- some Veterans are not VAC clients at the time of death (not in receipt of VAC benefits or services) and therefore there may be no information available in the VAC system.
- Data on the Funeral and Burial Program is contained in a database administered by the Last Post Fund whereas Veteran client demographic and death details are housed on VAC’s Client Service Delivery Network. The evaluation team was unable to fully match Last Post Fund data with VAC data due to reporting time frames, system limitations, and differences in clientele. Therefore, the evaluation team was only able to fully analyze a subset (2,452) of the Funeral and Burial program recipients (those who were VAC clients before receiving Funeral and Burial benefits).
- VAC systems do not currently contain any information on Funeral and Burial recipients, and the Last Post Fund has limited reporting capabilities for socio-economic information. Additionally, the 2020 Canadian Census analysis, which includes a Veteran component, was not available at the time of the evaluation. These factors limited the evaluation team’s ability to conduct a fulsome Gender Based Analysis (GBA+).
- Travel restrictions imposed due to the COVID-19 pandemic limited the evaluation team’s ability to conduct on-site field work. In lieu of in-person interviews, the evaluation team connected with interviewees via audio and/or video software.
- The 2017 Evaluation of Commemorative Benefits and Services team conducted a visual inspection of graves in select locations in Prince Edward Island, Halifax, and Quebec and noted the conditions of each grave inspected. Due to COVID-19 pandemic travel restrictions, the current evaluation team could not conduct follow-up inspections to determine whether grave conditions had improved, deteriorated, or remained the same in Halifax or Quebec. A limited sample was reviewed in Prince Edward Island.
- Observation of Last Post Fund frontline staff interacting with Funeral and Burial clients could not be conducted due to COVID-19 travel restrictions. While audio recordings of interactions were available when the Funeral and Burial program was last evaluated, recordings for this evaluation period were not available due to a change in the Last Post Fund telephone service provider.
- Access or observation of the Last Post Fund data system was not possible due to COVID-19 travel restrictions, limiting the evaluation team’s ability to determine the efficacy of the Last Post Fund decision-making processes. To mitigate, several file reviews were conducted by the evaluation team. In addition, Last Post Fund staff provided a virtual walk through of the decision-making process involving a variety of case scenarios.
- The evaluation team did not speak directly with survivors or estates that received reimbursement of funeral expenses through the Funeral and Burial Program. To mitigate, the evaluation team examined the results of surveys conducted by the Last Post Fund of those that applied to the program (both approved and disallowed). Interviews were also conducted with Last Post Fund front-line staff to gauge the needs and views of program applicants/recipients.
- Funeral and Burial matter of right decisions are rendered by VAC’s disability adjudication unit and payments are administered by the Last Post Fund, while the overall program is managed by the Commemoration Division. The evaluation team was unable to observe program processes and how the different areas interact. As a mitigation, the evaluation team examined business processes and conducted interviews with relevant parties to determine whether the process for matter of right payments was working efficiently. Additionally, the evaluation team reviewed case files and requested ad-hoc reports from Last Post Fund staff.
The above noted information should be taken into consideration when reading the evaluation report.