Evaluation of the Veteran and Family Well-Being Fund Summary

Recommendation #1:

It is recommended that the Assistant Deputy Minister of Strategic Policy, Planning and Performance determine if changes are needed to the intent and objectives of the Fund to align with its evolution and departmental priorities. Where appropriate, take the necessary steps to:

1. Modify the stated objectives of the Fund; and
2. Seek opportunities to increase core funding levels and/or staffing complement to address the Fund objectives and maintain the cumulative workload.

The Veteran and Family Well-Being Fund was originally established with the intent to fill a gap in supporting innovation and research in Veteran health and well-being. There have now been a number of application rounds, providing an additional baseline of information relating to the types of projects funded, as well as the needs from organizations focusing on Veteran and family well-being. In an effort to ensure continued alignment with departmental priorities and needs, VAC will assess if modifications to the intent and the objectives of the Fund are required, and take any subsequent steps to seek necessary approvals to modify program documents and associated resource requests.

Recommendation #2:

It is recommended that the Director General Policy and Research share guidance with funding applicants/recipients regarding:

1. the development of measurable project outcomes; and
2. suggested information that would be beneficial to support progress towards achieving project outcomes.

The Policy and Research Division (PRD) has begun a review of the current Veteran and Family Well-Being Fund (VFWF) application form and is identifying options to incorporate an additional project management component in the upcoming 2023-24 funding cycle. To ensure applicants have sufficient project planning guidance to develop measurable project outcomes, VAC will include a more structured project planning section in the application as well as an enhanced application guide to support the application. The applicant guide will expand on project planning components including project objectives, activities, planned outcomes, and outcome measures. PRD will also provide VFWF staff with project support tools to ensure recipients are equipped to measure and achieve intended project outcomes.

Recommendation #3:

It is recommended that the Director General Policy and Research enhance Fund data tracking with the intention to maximize internal reporting efficiencies, and support performance measurement and program decision making.

Since its inception in 2018, the Veteran and Family Well-Being Fund has made continuous improvements to application assessment processes and project reporting requirements. As the Fund continues to mature, VAC will place additional focus on enhancing the Fund’s internal reporting capacity to ensure that the necessary information is available to support performance measurement, internal reporting requirements, and program decision making.

Scope and methodology

• Conducted in accordance with the directive and standards specified in Treasury Board of Canada’s 2016 Policy on Results and VAC’s 2022-23 to 2026-27 Departmental Evaluation plan.
• Covers the time frame from April 2018 to March 2022.
• Conducted between April 2022 and March 2023.
• Multiple lines of evidence were used including document reviews, data analysis, observations, file reviews, and key informant interviews.
• Purpose of the evaluation was to ensure the Fund is relevant and meeting needs, while determining if there are opportunities to improve effectiveness or efficiency. It is anticipated that evaluation results will support the Research Directorate in managing the Fund and future decision making.
• The evaluation placed less emphasis on the efficiency of the Fund, as staff continue to implement changes resulting from the 2021 VAC Audit.

Constraints and limitations

• The evaluation team was able to visit select Fund recipients in Nova Scotia and Prince Edward Island, and then conducted virtual interviews with other organizations, as scheduling logistics and air travel as well as COVID-19 continued to provide risks and challenges with travel.
• Reporting for the Fund is primarily manual in nature. Throughout the evaluation, the team noted a lack of documented statistical information. Where possible, the information was created by the program area. The evaluation team noted discrepancies in some instances, which were highlighted with the program area.
• There was an inconsistent level of performance information available to fully assess Fund success. There was a limited set of completed project results. Many projects are multi-year projects, with only one third of projects considered completed. The nature of the funding program (research and innovation) also presented challenges in defining and measuring ‘success’ (e.g., what is considered innovative/how to measure success of innovation). To mitigate this challenge, the evaluation looked at multiple information sources (e.g., file review, interviews with various departmental staff and funding recipients, and document review). Where available, the evaluation team noted any short-term, observable indications of success.
• Gender based analysis (GBA+) was incorporated into the evaluation based on overarching project theme.
• The Fund is still maturing and has seen many changes in funding levels, processes, and management since its inception in 2018. This was taken into consideration when evaluating the effectiveness and efficiency of the Fund to date.
• The VAC Research Funding Program was launched by the department in April. This new program is separate from the Veteran and Family Well-Being Fund and was not considered in scope for this evaluation. Some findings in this evaluation may be linked or impact the Research Funding Program, as there are some similarities among the two funding programs.