This evaluation had been conducted in alignment with the 2016 Policy on Results where departments periodically evaluate organizational spending on the programs and services of the department.
In addition, this evaluation has been conducted as an identified area of risk/need. During the preparation of VAC’s 2020-25 Departmental Evaluation Plan, there was interest from VAC Senior Management in having the Audit and Evaluation Division evaluate the review and appeal process. Furthermore, an evaluation of VAC’s Ombud’s Office (2019) noted complexities and challenges in the review and appeal process and recommended that VAC review the various complaint and redress streams available to Veterans.
2.1 Evaluation Scope and Questions
The evaluation focussed primarily on the time period of April 1, 2015, to March 31, 2020. However, with the onset of the global COVID-19 pandemic, the Evaluation team also looked at some of the impacts of the pandemic on redress processes.
Departmental Reviews can be initiated directly by the Veteran and/or with the support of the Bureau of Pensions Advocates or the Royal Canadian Legion. For the purpose of this evaluation, the evaluation team focussed on BPA’s involvement in the redress process and not redress applications initiated by Veterans themselves or with the help of the RCL or with private representation.
Although Additional Pain and Suffering Compensation (APSC) decisions can now be reviewed and appealed at VRAB, they could not be for the majority of the time period covered by the evaluation and have therefore been omitted from scope as the numbers would be too low to be significant at this time.
The objective of this evaluation was to ensure the Disability Benefits redress process is relevant, meeting Veterans’ needs, while determining if there are opportunities to improve its effectiveness or efficiency.
Preliminary interviews, document review, and initial data analysis identified areas of priority and focus for this evaluation. Upon completion of an initial review, evaluation questions were developed to help assess program relevance, performance, efficiency and economy, and effectiveness in order to achieve the evaluation objective. Specific questions assessed during the evaluation are identified in Table 1.
| Evaluation Questions |
|---|
|
2.2 Multiple Lines of Evidence
Multiple lines of evidence have been used to support the evaluation findings. The methods undertaken to support these lines of evidence are identified in Table 2.
| Methodology | Source |
|---|---|
| Departmental Documentation and Secondary Research Review | Departmental documentation/information has been reviewed to understand the program objectives/intent, authorities and requirements, complexity, context and any key issue areas. Documents included departmental planning documents, policies, mandate letters, business processes, records of decisions, strategic documents, performance reports, research papers, survey results, and correspondence. |
| Non-Departmental Document Review | Various non-departmental documents were reviewed, including, Parliamentary reports, Budget Speeches/Plans, Speeches from the Throne, and documentation related to redress processes for other countries and jurisdictions. |
| Interviews |
Interviews were conducted with staff in the following areas:
Note: For the purpose of conducting this evaluation, VAC or its interviewees did not attempt to collect, use or disclose personal information of Veterans. |
| Statistical Analysis |
Statistical analysis included:
|
| File Review | A file review was completed to assist in evaluating the performance and efficiency of the Disability Benefits Program – Redress Process. A random sample of 371 files with redress decisions were reviewed, providing a confidence level of 95% +/- 5%. No BPA client files subject to solicitor-client privilege were part of this review. |
2.3 Considerations, Strengths and Limitations
- The evaluation team initially identified the review period as April 1, 2015 – March 31, 2020, however, as a result of the global Covid-19 pandemic, the evaluation team has included some of the impacts of that pandemic on the redress process in an effort to provide a fulsome and updated review of the process.
- As a result of restrictions imposed by the global COVID-19 pandemic, the evaluation team relied on virtual interviews and observations in lieu of in-person meetings and discussions.
- The evaluation team did not speak directly with Veterans who accessed the redress process. As a mitigation strategy, the team spoke directly with BPA employees and VRAB Members and staff who communicate with Veterans. In addition, the team reviewed data received from BPA’s client satisfaction questionnairesFootnote 5 and VRAB’s Review Hearing Exit Surveys.
- The Redress process is currently undergoing a migration process from its existing system of the Client Service Delivery Network (CSDN) to a new system called GCcase. In addition, VRAB has their own client management system which is separate from the CSDN (the VRAB Scheduling Application). Furthermore, BPA has not been able to secure its own client management system and, prior to the pandemic, relied almost exclusively on paper files. As a result, it is a challenge to track the whole process from start to finish given the different systems involved.