Veterans Affairs Canada - May 2022
Evaluation results
Relevance and Need
ETB is largely meeting Veterans’ expectations, while Veterans identified unmet needs associated with the CTS program. Eligibility is appropriate for both programs, although there are areas of confusion amongst potential clients. Improvement could be made to ETB short course funding.
CTS and ETB generally work well together, particularly for Veterans who access education planning services through CTS. The complementary nature of the programs is reinforced in program policies and other documentation.
Effectiveness
VAC front-line staff have a general knowledge of the CTS and ETB programs. There is an opportunity to improve communications with front-line staff around the programs.
Veterans surveyed indicated that their awareness of the CTS and ETB programs while they were serving was limited. The evaluation found that being able to access ETB does not impact Veterans’ decision to leave the Canadian Armed Forces.
The evaluation finds the ETB program is being delivered effectively. Eighty percent of Veterans surveyed indicated the training funded by the program met their needs, with 59% reporting that ETB provided them with the skills and knowledge they needed to engage in activities important to them.
Approximately 85% of survey respondents who participated in the ETB program indicated that they had completed their course of study or were still enrolled. Most indicated a satisfactory outcome from the program.
Client satisfaction with CTS suggests that delivery is not meeting needs or expectations. Overall, 56% of CTS clients surveyed (including clients who received both CTS and ETB) indicated satisfaction with the services received from the third-party service provider (55% indicated satisfaction with VAC services).
Most clients surveyed report their overall health and wellbeing, their adjustment to life after service and satisfaction with their life as good to excellent. Clients who only accessed ETB are more likely to report better outcomes in all these areas than clients who only accessed CTS.
Efficiency
Program expenditures for both CTS and ETB are on the rise historically and are forecast to increase in the coming years. Increased client uptake and expenditures are more pronounced for ETB in the forecast data.
Program data does not allow for easy measurement of program outcomes; the evaluation relied on the survey of clients to assess outcome achievement. There is an opportunity to improve data capture and integration the next time system changes are made.
Recommendations
| Recommendation 1 | Management responses and action plans |
|---|---|
|
It is recommended that the DG-SDPM improve communications about the programs among Members and Veterans, including:
|
With respect to Recommendation 1A, the Director General, Service Delivery and Program Management will work in collaboration with Communications, Centralized Operations Division (COD), Policy, Business Integration Directorate (BID), National Learning Unit (NLU), National Client Center Network (NCCN), and Field Operations (FO) to:
With respect to Recommendation 1B, the Director General, Service Delivery and Program Management will work in collaboration with, Communications, National Client Center Network (NCCN), Transition, Field Operations (FO), and National Learning Unit (NLU) to:
|
| Recommendation 2 | Management responses and action plans |
|---|---|
|
It is recommended that the DG-Policy undertake efforts to review the maximum amount available for ETB short courses and consider whether costs of licensing exams and fees should be included. |
With respect to Recommendation #2, the Director General, Policy and Research will:
|
| Recommendation 3 | Management responses and action plans |
|---|---|
|
It is recommended that the DG SDPM explore client satisfaction results and associated performance data to identify areas for further improvement, and gaps where further data collection is required. |
VAC’s current contract with a national service provider for Career Transition Services is due to expire in 2022 and VAC is in the process of re-procurement.
|
Program Description
- The Career Transition Services Program (CTS) works to ensure that eligible Canadian Armed Forces (CAF) Members, Reservists, Veterans, spouses, and survivors have access to services that will help develop the skills and tools they need to effectively search, apply for, and obtain employment.
- Services are delivered by a contracted service provider and focus on three broad categories: 1) labour market information; 2) career counseling; 3) and job-finding assistance.
- The Education and Training Benefit (ETB) is designed to help Veterans successfully transition from military to post-service life, achieve their education and post-military employment goals, and better position them to be more competitive in the civilian workforce.
- The ETB provides a taxable benefit of up to $86,671.95 (while a Veteran is participating in a formal program) to cover tuition, fees, supplies, and some incidental and living expenses.
- Veterans may be funded for most types of formalized educational programs and can also receive funding for one-time courses aimed at self-fulfillment and personal interest/development.
About the Evaluation
Scope and Methodology
- The CTS and ETB evaluation was conducted by Goss Gilroy Inc. on behalf of VAC between February 2021 and October 2021. It was conducted as part VAC’s 2020-25 Departmental Evaluation Plan in accordance with Treasury Board’s Policy on Results.
- The evaluators assessed the programs’ relevance, effectiveness, and efficiency over the period April 1, 2018, to March 31, 2021.
- The evaluation also considered:
- Alignment between the CTS and ETB programs;
- Application of a Gender-based Analysis Plus (GBA+) lens to the evaluative process to assess how diverse groups of women, men and non-binary people served by VAC experienced the programs; and
- Understanding the extent to which the programs are producing outcomes that contribute to the well-being of eligible Veterans.
Constraints and Limitations
- The evaluation team conducted an on-line survey of participants in the CTS and ETB programs. Only Veterans that opted-in to be contacted on their My VAC Account were included in the sampling frame.
- Data collection, including the survey, occurred over the summer months at a time when potential respondents may have been less available or inclined to participate.
- It is possible that there are differences between stakeholders who participated in data collection and those who did not. For example, clients who had either very positive or very negative experiences may have been more enthusiastic about participation. This means that findings from individual lines of inquiry are not, for the most part, generalizable to the whole population. To mitigate, alternative lines of evidence were used to support survey findings.
- VAC data files did not contain information on indicators related to eligibility decisions, or program utilization over time, other than application processing data. To mitigate, data from the third-party service provider’s Three-Year Annual Report was used to supplement VAC data.