This evaluation was conducted in response to the Treasury Board of Canada’s requirement to ensure there is adequate evaluation coverage of risk and organizational spending. It was conducted in accordance with the directive and standards specified in their Policy on Results.
2.1 Evaluation Scope
The objective of this formativeFootnote 4 evaluation was to look for alignments between the existing ToD medical impairment criteria for the PSC, and the health-related assessment criteria of the APSC and CRB programs. These alignments could:
- identify opportunities to be more efficient;
- determine if unintended impacts are occurring among these VAC programs; and
- inform/promote equitable program access for all VAC clients.
The evaluation will support program and policy decision making and inform the implementation of new departmental initiatives.
This evaluation focused on CAF Veterans assessed with more significant levels of disability and impairment. To explore the potential alignments of the program assessment criteria through data analysis, the evaluation team set a disability percentage of ≥40% assessment for a single disability condition as the evaluation threshold. This threshold was used for analytical purposes only to explore relationships, and does not suggest a standard for alignments based on assessment level. It is recognized that each program may benefit from examining alignments at a higher or lower threshold based on program intent and criteria.
2.2 Evaluation Questions
Three questions (see Table 1)—based on information collected from a planning phase staff survey in fall 2019 and preliminary staff interviews—guided the evaluation.
| Recognition, Health, and Income Replacement Programs for Canadian Armed Forces Veterans with more significant disabilities/impairments |
|---|
|
To focus on the evaluation objective, meet timeline targets, and comply with available resources, a number of areas were excluded from the scope of the evaluation. These include:
- The delivery of specific program benefits/services (including VAC providers); Disability Benefits/Pain and Suffering Compensation and claims processing;
- Case Management function;
- Assessment of specific Treatment Benefit grids;
- CAF Long Term Disability Insurance Plan (previously called SISIP);
- CAF to VAC transition process;
- Program relevance; and
- Program logic models and progress towards achieving program outcomes.
2.3 Multiple Lines of Evidence
Multiple lines of evidence have been used to support the evaluation findings. The methods undertaken to support these lines of evidence are identified in Table 2.
| Methodology | Source |
|---|---|
| Documentation Review and secondary research review |
|
| Non-Departmental Government Document Review |
|
| Consultations | To support this evaluation, consultations with various stakeholders were conducted. These key consultations began early in the evaluation process and included:
|
| Literature Review |
|
| File Reviews (conducted by VAC employees and in adherence to privacy and personal information safeguard requirements) |
|
| Statistical/Program Data |
|
| Survey |
|
2.4 Considerations, Strengths and Limitations
The evaluation team acknowledges the following with respect to the evaluation and findings:
-
On 1 April 2019, VAC implemented significant changes through the Pension for Life (PFL) initiative. PFL included the launch of new programming (PSC, APSC and the Income Replacement Benefit) and the implementation of a new client relationship management system called GCcase. APSC administrative client data was not available in GCcase for the evaluation period due to ongoing system development. To mitigate, the evaluation team used data from the program’s precursor, the Career Impact Allowance (CIA)Footnote 6, as a substitution to explore the relationship between the PSC and APSC.
CIA program data was determined to be a suitable substitution for the APSC as:
- Like the APSC program, the CIA had the same definitions of permanent and severe impairments laid out in the program regulations; and
- The CIA measured the extent of the impairment using the same three grade levels now used by the APSC program.
The key differences between the CIA and the APSC are the different program intents (economic vs. non-economic compensations) and that in order to be eligible for the CIA Program, the Veteran had to apply for and be participating in the Rehabilitation Program. As the CIA was an economic benefit, the department assessed the Veteran’s earning capacity and considered the number of years the Veteran had left to serve in the CAF as part of the grade level determination as well as any medical and physical impairments. It is important to note that earnings capacity and years left to serve are not considerations in the APSC grade level determination. Additionally, while participation in the Rehabilitation Program is not required to gain access to APSC, a barrier to re-establishment must be identified for APSC entitlement.
- Any potential changes to the assessment process for any program can have significant cost implications that require approval from the Minister of Finance. As a result, any potential changes that are explored by the Department will require an accrual costing completed by the Office of the Chief Actuary and if new funding is required, any proposed changes must be submitted to the Department of Finance for consideration as part of the Federal Budget process.
- The implementation of the GCcase system required considerable input from and impact on operational decision makers. Evaluation requests to operational staff were concise so as to minimize impact on their already busy workload.
- At the same time as this evaluation, Strategic Policy began an initiative which was to consider the overall structure of VAC programming. As such, the efficacy of the general structure of VAC programming was not examined.