The Fund has a Performance Information Profile in place with outcomes identified. Information in support of the performance indicators and outputs is being reported to some extent. However, there are opportunities to enhance the Fund’s performance measurement.
A Performance Information Profile (PIP) is a document that identifies the planned performance information for each program in the department’s program inventory. Each PIP outlines program outputs and outcomes, as well as related performance indicators, targets, data sources and associated methodologies. The department’s Integrated Planning and Performance unit is responsible for monitoring and overseeing PIPs, while VAC program officials are responsible for establishing, implementing and maintaining PIP information in support of their programs. Status updates against individual PIPs are periodically presented to the VAC Performance Measurement and Evaluation Committee (PMEC).
The Veteran and Family Well-Being Fund shares a PIP with the Research Fund and the Centre of Excellence on Post Traumatic Stress Disorder and related Mental Health Conditions, under the Research and Innovation PIP. The most recent version of the Research and Innovation PIP is dated 2021. Additional outcomes and associated performance indicators were added in support of tracking Budget 2021 impacts. This information is targeted to be available for 2024.
Overall, there was a relatively small number of completed projects (34% of all funded projects, or 35/102) available for review during the evaluation scope. With a limited number of projects considered completed, most which are from the first two application rounds, it is difficult to make broad statements regarding the success of the Fund. The Fund has been successful in providing funding to numerous organizations targeting a variety of Veteran and well-being theme areas, such as homelessness, mental health, employment and Indigenous Veterans.
4.1 Achievement of expected outcomes
Results for the primary outcomes and indicators were documented for the period of the evaluation. Outcome results were published in the PIP; however, the evaluation team was unable to verify/locate supporting information or sources for some performance measures. During the evaluation scope period, there was significant changeover of program staff as well as ongoing changes to application and reporting templates, and processes. These changes in combination with inconsistent supporting documentation led to challenges for the evaluation team in confirming supporting performance measurement information. A review of the PIP as well as interviews with the program area identified that the outcomes and supporting performance indicators provide minimal information that aid in the management of the Fund. The success of the Fund is currently linked to the success of individual projects, which provides performance measurement challenges as well. This topic is further elaborated on in the next section of the report.
4.1.1 Program effectiveness
The overall objective of the Fund is to provide financial support to organizations undertaking innovative initiatives intended to improve the well-being of Veterans and their families.
The evaluation measured progress towards achieving the following stated outcomes of the Fund:
Immediate outcome #1 – “Increased engagement of organizations in projects to improve Veteran and family well-being”
The performance indicator identified to measure this outcome is the “percentage increase in new applications year over year”. This indicator measures the change in awareness of the Fund by measuring new applicants. There is no baseline measurement to know how many projects centered on Veteran and family well-being were occurring in Canada prior to launch of the Fund, making it difficult to measure how effective the program has been at increasing projects aimed at Veteran and family well-being. However VAC was aware of 80 plus inquiries to the department seeking funding over a three-year period (2012 to 2015). The number of applications received annually surpasses this three year total.
As seen in Table 6, between 42% and 54% of funding applicants per year are first-time applicants. This figure exceeds the original target of 10% stated in the PIPs. The ongoing engagement from new organizations indicates that the department has been effective in creating awareness of the Fund, and that a higher target would be appropriate.
Table 6: Number and percent of first-time applicant organizations
| 2018-19 | 2019-20 | 2020-21 | 2021-22 | 2022-23 | |
|---|---|---|---|---|---|
| Total annual applications received | 155 | 114 | 142 | 98 | 118 |
| Number of first-time applicants | N/A | 44% (50) | 54% (77) | 42% (42) | 43% (51) |
Note: Due to the manual nature of the methodology used to calculate the number of applications and first time applicants, the evaluation team noted discrepancies in reported numbers as methodological approach.
Another factor for consideration when measuring the effectiveness of engagement is the number of new applicant organizations awarded funding, which would provide additional information on the number of projects initiated in support of Veteran and family well-being.
The evaluation team notes that if annual budget levels do not allow for a reasonable percentage of approved projects, from diverse organizations, there is a risk that future application call levels may be impacted (i.e., organizations may not see the value in completing an application after multiple years of being unsuccessful in securing funding). The average approval rate for funded projects has been around 20%. The Fund’s budget will be further elaborated on in section 4.2, Program Efficiency and Economy.
Though not indicative of whether an organization submits an application, the visits to the Fund’s webpage has been steady with around nine to ten thousand visits per year. The webpage provides background information on the Fund as well as the link to the online application. As can be seen in Table 7, the first year of the Fund received significantly more traffic (30,000 plus visits), which is understandable as initial interest to learn more about the Fund would be expected. The evaluation finds that the VAC website has been an effective tool in raising awareness about the program.
Table 7: Views of the Fund’s web landing page
| Fiscal year | English | French | Total views |
|---|---|---|---|
| 2018-19 | 27,240 | 4,151 | 31,391 |
| 2019-20 | 8,044 | 1,105 | 9,149 |
| 2020-21 | 9,177 | 1,157 | 10,334 |
| 2021-22 | 8,194 | 1,142 | 9,336 |
During the evaluation scope period there were numerous posts on the department's Facebook page and Twitter regarding open application rounds for the Fund as well as posts on funded project events/announcements. There has been limited presence on VAC’s YouTube channel and the department’s Instagram account is dedicated to remembrance posts. There is an opportunity for the program to explore how new audiences might be reached more effectively and economically through social media as well as highlight funded projects to further enhance visibility. Currently, projects that received funding are often highlighted in media stories, which the Research Directorate began tracking in 2022. Additionally, VAC has an initiative to highlight Veteran stories through the departmental website as well as Salute!. Though not officially tracked, projects funded through the Veteran and Family Well-Being Fund and/or the organizations that have received funding have formed part of such personal stories, indicating that Veterans are indeed involved with these organizations. Unfortunately, due to the broad range of topics and organizations involved, the evaluation team was unable to attribute the Fund’s impact in personal Veteran stories and Salute! article reach. There is an opportunity to further consider how personal impact stories highlighting projects funded through the Veteran and Family Well-Being Fund might be tracked more officially.
Immediate outcome #2 – “Eligible organizations have access to the VFWB Fund”
The outcome relating to access of the Fund is measured by the amount of the annual core budget expended. This measure continues to be met each year, as the Fund has continually spent its full budget allotment and has received additional supplementary funds from 2019-20 to at least 2023-24 (end of Budget 2021 temporary funding). While monitoring the budget is a critical activity, this indicator does not provide much value to measuring the success of access to the Fund. In reality, based on the number of applications approved versus submitted annually, the documented performance indicator will continue to be achieved. A better measure may be the percentage of qualified applications that are approved for funding.
Approval of funding is primarily linked to the availability of the annual budget. The evaluation team notes that there was a spread in the length of funded projects. For example, in the initial year of funding (2018-19) projects were mainly funded for one year (52%), and in 2021-22 the majority of projects were funded for three years (67%), and no projects were approved for funding beyond three yearsFootnote8. Graph 1, highlights the variability in funded project length by fiscal year of approval.
Graph 1: Number of projects by project length (years funded) by fiscal year of approval
Graph 1: Number of projects by project length (years funded) by fiscal year of approval
| Number of projects | Project length (years funded) |
|---|---|
| 2018-19 | |
| 11 | 1 year |
| 2 | 2 years |
| 3 | 3 years |
| 1 | 4 years |
| 4 | 5 years |
| 2019-20 | |
| 9 | 1 year |
| 2 | 2 years |
| 6 | 3 years |
| 1 | 4 years |
| 4 | 5 years |
| 2020-21 | |
| 6 | 1 year |
| 3 | 2 years |
| 5 | 3 years |
| 2 | 4 years |
| 7 | 5 years |
| 2021-22 | |
| 4 | 1 year |
| 8 | 2 years |
| 24 | 3 years |
| n/a | 4 years |
| n/a | 5 years |
In terms of timely access to funds, the 2021 Audit of the Veteran and Family Well-Being Fund found payments to Fund recipients were made in a timely manner. Recipient organizations interviewed by the evaluation team agreed that payments were timely, once agreements were initiated, however the turnaround for funding decisions/announcements sometimes impacted their ability to initiate a project. There is a risk that recipient organizations may not receive the necessary funding in time to appropriately plan and ensure adequate staffing is in place to effectively deliver their planned project activities.
Immediate outcome – “Projects / initiatives support Veterans and their families well-being.”
The performance indicator linked to this outcome measurement is the percentage of funding recipients that have achieved their planned project outcomes with a target of 80%. Results from the PIP indicate that almost all projects achieved their planned outcomes (94% to 100% from 2019 to 2021). This seemingly indicates the Fund was effective in supporting projects aimed at Veteran and family well-being. However, the performance results as presented in the PIP were not linked to information sources or analysis documents. A review of documents and interviews with the Fund staff in the VAC Research Directorate indicate an inconsistent and informal approach to measuring the achievement of project outcomes. At the time of the evaluation, there was no specific strategy to measure and document the achievement of planned outcomes for projects.
Assessing achievement of planned project outcomes was challenging for the evaluation team. Generally, an outcome is defined as a change or consequence attributable to activities and outputsFootnote9 conducted by an initiative or program. Outcomes may be immediate/short-term, medium, or more long-term in nature. Projects funded through the Veteran and Family Well-Being Fund ranged from one year to five years, and had varying degrees of objectives and outcomes identified in project documentation.
Additionally, the evaluation team noted the following challenges when assessing the achievement of project outcomes:
- In some cases, the intended result for the project was a tangible output, which was a more straight forward measure, but is not necessarily an outcome (e.g., the development of a guide outlining services for Veterans is an output, but it doesn’t lead to an outcome unless the guide is used and Veteran or family well-being is improved).
- The end of a project funding agreement (when a project is considered completed by VAC) does not always mean that the organization’s project is complete, and the availability of quantifiable performance information may not be available yet.
- The interpretation of ‘success’ of projects can be subjective. Due to the nature of innovative projects (e.g., trialing new therapeutic services, conducting research), success is not always a linear measure, there may need to be a few iterations or changes, or even failures.
The evaluation’s review of project files found varying degrees of project outcomes identified in applications as well as varying supporting information available to measure the achievement and effectiveness of intended project outcomes. The evaluation team found that the majority (86%) of completed project files had some degree of information to support measuring the achievement of intended project outcomes. A few project files had no evidence available to the evaluation team, or the evidence was not measurable.
Outcome information is presented in an open-paragraph field in both the application and the final report templates. The specificity around what is required to substantiate the intended planned outcomes (i.e., degree of supporting information required) is not defined and therefore the level of information varied by project. The file review identified that project objectives and planned outcomes were not consistently presented at the beginning of the project. There are indications that applicant organizations sometimes have a different interpretation on ‘planned outcomes’. For example, the creation of an output (e.g., a reference guide) being identified as the planned outcome, versus identifying the expected outcome related to creating an output (e.g. the desired change in behavior or actions that would occur as a result of the reference guide being created).
Note: The 2022-23 Fund application form did not have a specific question requesting the planned outcomes of projects. The rationale for this change was related to the above indication (applicants were unfamiliar with the terminologies and the level of information submitted varied). The evaluation team notes this lack of information will most likely cause additional challenges measuring the success of funded projects and providing supporting information towards the PIP (unless what is being measured through the PIP is changed as a result of a potential review of Fund intent/objectives). As a mitigation strategy, outcome measures could be incorporated into project funding agreements.
One-on-one interviews with organization representatives provided the evaluation team an additional level of information that is not always easily captured in a textual way within a reporting template. The evaluation team acknowledges that this information may be known or more easily interpreted by Fund staff, who are more knowledgeable on individual projects and organization’s backgrounds, but this information was not always easily interpreted by the evaluation team based on the documentation available on the local network at the time of the evaluation.
There is a potential opportunity to capture and use more qualitative information, in conjunction with project self-reporting to inform performance measurement and program management reporting. Project reporting will be further elaborated on in section 4.1.4, Opportunities to improve effectiveness.
Additionally, there is an opportunity to consider how organizations collect feedback from Veteran participants in support of measuring the intermediate outcome. Based on the information reviewed by the evaluation team, there is not consistent tracking of input from Veterans by organizations and/or sharing this information with VAC, which would be helpful in measuring the impact on the well-being of Veterans and their families.
Enhanced outcome measurement guidance in the application package and/or increased engagement between Fund staff with recipients could help to:
- clarify expected project outcomes;
- define measurable indicators to be available within the project scope period;
- describe the level of supporting information expected in project reporting; and
- lead to improved project outcomes.
Based on the level of information available in project files to date, the evaluation is not able to comment on how effectively individual projects impacted the well-being of participants.
Ultimate outcome – “Veterans are physically and mentally well”
The performance indicator for the ultimate outcome is the percentage of Veterans who report that their health is very good or excellent. The data source for this indicator is those who self-report their health as excellent or very good in the Life After Service Survey. Since this survey has not occurred for a number of years, self-reported health status from the VAC National Client Survey (Veteran clients) and Canadian Community Health Survey (Canadian population) are also used to measure this outcome. Table 8 highlights the percentage of Veterans reporting their health as very good or excellent.
Table 8: Percentage of Veterans who report their health is very good/excellent
| Target | 2018-2019 | 2019-2020 | 2020-2021 | 2021-2022 |
|---|---|---|---|---|
| 50% | 46% | 39% | 39% | 21% |
Source: VAC National Client Survey
Measuring the impact on Veterans well-being as attributed by the Fund is difficult, as this is a long-term outcome, and the Fund is a relatively new program. In addition, recipients/participants of the funded programs or services are not required to be VAC clients.
Despite the above statements, in theory the Fund should contribute to Veteran well-being by funding organizations that are striving to support Veterans through direct programs and services, or through research and innovative initiatives that aim to progress knowledge and understanding on Veterans health and well-being issues. However, because the Fund does not directly serve Veterans, and organizations collect various levels of participation/satisfaction information to gauge the effectiveness of their initiatives, there is no direct source of performance information to measure this outcome.
4.1.1.1 Opportunities to improve effectiveness
The Fund continues to make adjustments to enhance the rigour of the application assessment process and monitoring of ongoing projects, as well as documenting supporting information.
The Fund has continued to grow and adjust since its inception in 2018. The approach to assessing applications being a good example. The process has evolved from:
- a review done almost entirely by the Fund staff and presented to a management review committee to determine recommendations, to
- the Fund seeking volunteers in the Policy and Research group to review applications, to
- engaging key subject matter experts based on topic area.
Each of these approaches have provided lessons learned and the Fund has continued to adjust processes and tools in response. As a result of these continual changes to the application form, the supporting information technology infrastructure is currently being rebuilt.
The VAC 2021 Audit of the Veteran and Family Well-Being Fund found that the assessment process was rushed, with little time for back and forth with applicants and consultation with subject matter experts. In the 2022-23 application process, internal subject matter experts related to the core topic areas were formally involved in assessing applicationsFootnote10. In total there were 118 applications and around 20 VAC resources involved in reviewing and assessing/providing feedback on applications. Of the 118 applications this round, 21 projects were recommended for approval. There is no established service standard for assessing applications and arriving at funding decisions.
There is an opportunity to enhance the reporting protocol for projects and the overall relationship with the funding recipient organizations.
In response to a recommendation in the 2021 Audit, the Fund has also enhanced project monitoring guidelines and developed a process for enhanced monitoring for projects considered higher risk. The main mitigation activities noted are media review, requesting additional information, and conducting recipient audits. Much of the required structure is in place to support project monitoring however there has been limited implementation in activities to date (note: COVID-19 provided continual impacts on the ability to conduct in-person recipient visits between March 2020 and into the fall of 2022).
As part of funding agreements, recipient organizations are expected to provide VAC with progress reports and a final report outlining details on achievement of project activities and outcomes. All completed projects were found to have a final project report on file. A complete report is also required by VAC in order to initiate a payment, whether that be a progress report for an annual installment, or a final report for final payment.
As mentioned, the level of information included in projects individual reporting varies. In some situations, recipient organizations shared actual participant attendance and/or participant feedback through project reporting, however this information is not consistently included, and not all projects would have participants as a measurement (e.g., research projects). In some cases, confidentiality concerns can impact the level of information shared with VAC (i.e., personal details/stories). Sharing the success or progress of a project may also be in a less quantifiable format, for example sharing general participant feedback or success stories verbally or observing the organizations physical space/programming. The evaluation team notes that meeting one-on-one with recipient organizations for the evaluation provided a valuable benefit in better understanding the essence of the individual projects and influence on the Veteran community. This level of personal impact is not always easily gleamed from reading a report. There is an opportunity for VFWF staff to incorporate more one-on-one discussions (in-person or virtually) with funding recipients to enhance the degree of qualitative information collected in support of measuring project success.
A number of representatives interviewed from recipient organizations stated that they had further information that could be shared with VAC about their projects, which from their perspective did not seem to fit into the pre-defined sections of the template. Interviews with Fund staff noted in some instances they do share feedback with organizations in order to clarify reporting information and/or to request additional information. There is an opportunity to enhance communication with funding recipients regarding the expected level of reporting information.
As identified in section 4.1.1, Program Effectiveness, the project report templates are mainly open paragraph style, which provides flexibility for recipients to share information. However, there are situations where more direction regarding targeted performance measurement information would be beneficial to support measuring how the individual projects contribute to the overall success of the Fund. Specifically requesting information such as: participation/attendance data (Veteran and families), quotes/personal impact stories or pictures, any engagement/partnerships with community stakeholders/organizations, and lessons learned. This information is not currently requested specifically, but may be included in project reporting.
Feedback from applicants and recipients on the application process and/or experience with the Fund has not been formally collected to date. As a new program, receiving user input would be a useful measure to gauge potential areas for improvement as the Fund continues to grow and mature.
The evaluation team determined that there is an opportunity to enhance the reporting protocol in order to maximize the collection of performance information, while enabling organizations the flexibility to share supporting evidence determined to be important from their perspective.
4.1.2 Gender Based Analysis plus considerations:
Gender-based analysis plus (GBA plus) is an analytical process used to assess how different groups of women, men and gender-diverse people experience policies, programs and services based on multiple factors including race, ethnicity, religion, age, and mental or physical disability. According to Treasury Board Secretariat (TBS), differential impacts on diverse groups of people are to be considered for policies and programsFootnote11. As of 2020, TBS requires GBA plus indicators be incorporated into PIPs.
With funding through Budget 2021, there were outcomes added to the Fund’s PIP including the percentage of annual budget expended on projects with a focus on women and LGBTQ2+. A target of 20% of funds going to these projects was established. In 2021-22 (when the bulk of the temporary funding was committed), the Fund reports that approximately 30% of funding awarded in that fiscal year went to projects focusing on women and/or LGBTQ2+ Veterans. Over the evaluation scope period, the spread of funding for women and LGBTQ2+ projects are slightly less at 14%. Based on reporting from the program area, the great majority of applications submitted that focus on women and/or LGBTQ2+ have been awarded funding.
Tracking projects that focus on Indigenous Veterans and their families is more challenging. Many projects cross-cut themes and may have multiple areas of focus (e.g. mental health and Indigenous Veterans). Between 2018-19 and 2021-22, over $625,000 was awarded to Indigenous organizations through the Fund, meeting 29% of the overall funding requested for applications focusing on Indigenous Veterans.
During the file review of completed projects, the evaluation team found that few projects completed to date were specifically directed to women (2), Indigenous (1) or LGBTQ2+ (0). Many project applications self-identified as targeting these populations, but upon closer review by the evaluation team, it was determined that the projects were not specifically directed towards these groupsFootnote12.
For the 2022-23 application round, the program area did track submitted applications by identified theme areas. Prior to 2022, tracking was done on a broader perspective. Continuing to track submitted applications and funded applications by theme, year over year, would provide additional information to help inform performance measurement and program decision making from a GBA+ lens.
During the 2021-22 fiscal year, the Fund application was updated to include questions asking how the proposed project could support equity seeking groups (such as women, LGBTQ2+, Indigenous, and homeless Veterans), and how the project aligns with GBA+ considerations. Project reporting templates were also updated so organizations can explain how their projects align with GBA+ considerations, as well as requests for supporting data for projects related to homelessness. The level of information collected depends on the level of information individual participants/clients are willing to share.
4.1.3 Unintended impacts
There is a risk that some organizations rely on VAC funding in order for their project/program to continue.
While interviewing organizational representatives from recipient organizations, the evaluation team noted that many demonstrated a passion for helping Veterans and expressed value in how their individual programs/services support Veterans and/or their families well-being.
A number of organizations have received funding multiple times and/or consistently throughout the evaluation scope period, with some organizations receiving continuous funding since 2018-19. While it is possible to receive funding multiple times for diverse projects, a number of funding recipients interviewed expressed reliance on the funding to continue to deliver their project or program.
The evaluation file review identified a number of completed projects (43% or 15/35) had potential for a risk of dependency in funding (e.g., funding was for a project that didn’t seem to be finite in nature or was already being provided, the same or similar project was funded in previous years, or funding was used for ongoing operational costs). As mentioned in Section 3.1, many of the organizations funded are not-for-profit organizations and thus there is a continual need for incoming funds to support programming and operations.
The Terms and Conditions for the Fund clearly stipulate that funding is to be used for activities/projects that are finite in nature, and that activities/projects that “represent or establish a requirement for ongoing funding or create a dependency” do not qualify. The challenge is some organizations have created well established programs that serve vulnerable populations and/or provide needed services and supports. Financial support through the Fund in many cases goes towards expanding geographical reach or funding increased participation for programs/services.
The Fund provides the Department with visibility and media exposure. Financially supporting a variety of Veteran groups/stakeholders offers a range of opportunities to highlight good news stories across the country.
In 2022-23, Fund staff began tracking media stories highlighting recipient organizations/projects. As of March 2023, there were 60 plus media stories captured regarding projects funded by VAC. Impact or breakdown of the messaging in media stories is not specifically captured, however Fund staff indicate that they use this information as a mechanism to monitor projects and identify potential risks. As mentioned in section 4.1.1, Program Effectiveness, there may be an opportunity to enhance tracking of Veteran success stories and Salute! articles highlighting projects/organizations that received funding through the Fund.
VAC continues to make strides to enhance stakeholder engagement and support Veteran organizations. The Fund provides a mechanism to financially support various endeavors. There is limited information available to state how much influence the Fund has, however individuals from recipient organizations interviewed indicated appreciation for the funding and VAC staff indicated the Fund provides positive exposure through a relatively small Veterans Affairs Canada program.
The ability to provide widespread and sufficient funding to organizations and Veteran projects in order to meet planned outcomes may be limited by funding capacity.
The Fund is limited in its ability to provide impact to multiple organizations. In 2024-25 the Fund will return to its base budget of $3 million. Based on the funds already committed to approved projects (as of April 2022), there will be approximately half of the annual budget available in 2024-25 for new projects. Previous funding levels and average project funding amounts indicate the Veteran and Family Well-Being Fund can only support three to four new projects next fiscal year.
Approximately 40% (41/102) of projects approved for funding received less money than requested in their application. Representatives from recipient organizations interviewed by the evaluation team noted that receiving less funding than requested impacted their ability to meet their initial outcomes or objectives. In many cases this meant scaling back on projects or tailoring their project’s scope.
Section 4.2, Efficiency Economy further elaborates on the challenges regarding the budget and funding capacity.
4.2 Program efficiency and economy
The Fund has made strides to enhance the governance and rigour of the application assessment process and progress reporting frequency. The added rigour as well as the increasing number of projects being monitored, requires a level of resourcing which cannot be maintained if the Fund returns to its core staff funding levels.
4.2.1 To what extent are Fund resources used efficiently and economically
As the Fund continues to mature, the program area has been making strides to improve governance of the Fund. As a result of the VAC 2021 Audit of the Veteran and Family Well-Being Fund, the Fund made a number of changes, such as: updates to the application formFootnote13, progress and reporting templates and the application scoring tool, creating an application guide, and creating processes and guidelines for potential site visits. In addition, more emphasis was placed on documenting supporting information for the application review and decision making.
The 2021 VAC audit found that the Fund lacked some controls in terms of reviewing contribution project expenditure claims. The Audit states that financial claims were always submitted; however, the department did little in the way of validating the expenditure claims, often taking information at face value. In response to the related audit recommendation, a monitoring process for high-risk projects was developed. Implementation of this process is new (February 2022), and the evaluation team did not assess the degree to which this process has been implemented. Expenditure claims are not required for grants. Grants have been the primary fund disbursement method since the audit was conducted, as Budget 2021 funding is entirely grants.
Each year VAC receives 100 plus applications seeking funding through the Veteran and Family Well-Being Fund. Each application is reviewed and ranked based on its merit and assessed against established criteria (which has evolved as the Fund matured, and as a result of the 2021 audit). The added rigour of the application assessment process adds additional administration costs to the Fund, including in-kind resources from redirecting subject matter experts from other areas of the department for a period of two to four weeks.
In the 2022-23 application round, 118 project proposals were reviewed by numerous VAC program/policy areas, as well as scored and ranked by Fund staff. Though the robust governance of the application process includes multiple reviewers, it enhances the objectivity of the assessment process. There is, however, significant time and effort spent assessing each individual project against numerous criteria and documenting supporting rationale. This time may be disproportionate to the amount of funds available for distribution and the number of projects which can be approved each year (if funding levels return to base level, this may only be a few projects in future years). Though the application assessment period is generally for a time specific period, significant resource effort is required to review, assess, and score applications as well as consider overall which projects to put forward as recommended for approval.
As the Fund continues to evolve and learn from past application rounds, there may be opportunities to find further areas to enhance the efficiency and economy of the application review process, while maintaining a solid governance structure.
As more projects are approved and funded for multiple years, there is a ripple effect on the monitoring of all ongoing projects and overall management of the Fund. In the 2021-22 funding round there were 36 new projects added to the project list. In the same fiscal year, 11 projects were completed. As of January 2023, the Fund had 67 projects underway. Based on current projects lengths and average funding lengths of projects to date (two to three years), the evaluation team estimates there will continue to be approximately 60 projects that need to be monitored at any given time for the next few years.
The program area currently has four full-time staff, consisting of two indeterminate analysts, one temporary analyst, and one program manager. The effort required to support each application round of 100 plus applications and adequately monitor 60 or more ongoing projects is most likely beyond the capacity of the current permanent staffing complement of two analysts.
Administrative costs for the Fund are not currently tracked as a distinct expense. When the Fund was rolled into the larger Research and Innovation PIP in 2020-21, administrative costs for the Fund were also included as part of this larger umbrella program. For 2018-19 and 2019-20, administrative costs totaled $430,539 and $460,703. Based on this change in administrative costing strategy, the evaluation team is only able to report on the overall actual budget and expenditures for the Fund. Total budget and expenditures for the Fund are highlighted in Table 9.
4.2.2 Budget capacity
The Fund has continually spent the full budget allotment each year, and committed funds to future budgets. How the Fund plans to address future demand is unclear.
Fund staff continually monitor commitment dollar amounts and payments throughout the year against individual projects. Discussions with VAC Finance representatives state that monitoring and management of the Fund budget occurs and had no concerns. Fund staff use multiple tracking documents to ensure that the appropriate amount of funds are distributed each year and that multi-year project commitments are appropriately accounted for in future years.
As previously mentioned, the Fund is allocated $3 million annually as core funding. The Fund has only operated one year within its core funding, and each year afterwards its budget has been temporarily supplemented with funds from within the department or through Budget 2021Footnote14. See Table 9 for Veteran and Family Well-Being Fund budget, funding available and expenditures. The full budget allotment for the Fund has been spent each year.
Table 9: Veteran and Family Well-Being Fund budget, funding available and expenditures as of March 31, 2022Footnote15
| 2018-19 | 2019-20 | 2020-21 | 2021-22 | 2022-23 | 2023-24 | 2024-25 | |
|---|---|---|---|---|---|---|---|
| Budget 2017 | $3,000,000 | $3,000,000 | $3,000,000 | $3,000,000 | $3,000,000 | $3,000,000 | $3,000,000 |
| Budget 2021 | $5,000,000 | $5,000,000 | $5,000,000 | ||||
| Transfer from Vote 1 | $1,830,000 | $4,000,000 | |||||
| Total Budget | $3,000,000 | $4,830,000 | $7,000,000 | $8,000,000 | $8,000,000 | $8,000,000 | $3,000,000 |
| Amount of budget already committed based on previous project approvals | n/a | $1,828,993 | $3,000,001 | $2,603,641 | $6,500,000 | $5,264,649 | $1,316,937 |
| Budget available for application rounds by fiscal year | $3,000,000 | $3,001,007 | $3,999,999 | $5,396,360 | $1,500,000 | $2,735,351 | $1,683,063 |
| Core funding available by fiscal year | $3,000,000 | $0 | $0 | $0 | $0 | $535,350 | $1,683,063 |
| Expenditures | $3,000,000 | $4,830,000 | $7,000,000 | $8,000,000 |
The $15 million of temporary funds was almost fully allocated in 2021-22 (the first year of the three-year allocation period), with $11.3 million committed to 36 projects in that fiscal year’s application round. Front loading the temporary funding enabled numerous projects to start immediately, with the potential to see impacts from the temporary funding sooner than expected. This approach impacted the amount of funding available to be awarded to projects in future years which can be seen in Graph 2, Availability of future budget dollars versus commitments.
Graph 2: Annual budget and commitments by fiscal year (as of March 31, 2022)
Graph 2: Annual budget and commitments by fiscal year (as of March 31, 2022)
| Year | Temporary funding budget | Core funding budget | Amount of budget committed ahead of each fiscal year |
|---|---|---|---|
| 2018-19 | $0 | $3,000,000 | $0 |
| 2019-20 | $1,830,000 | $3,000,000 | $1,828,992.68 |
| 2020-21 | $4,000,000 | $3,000,000 | $3,000,001 |
| 2021-22 | $5,000,000 | $3,000,000 | $2,603,640.50 |
| 2022-23 | $5,000,000 | $3,000,000 | $6,500,000 |
| 2023-24 (Estimate) | $5,000,000 | $3,000,000 | $5,264,649 |
| 2024-25 (Estimate) | $0 | $3,000,000 | $1,316,963.52 |
Based on the funds already committed to approved projects (as of April 2022), there will be approximately half of the budget available in 2024-25 for new projects when the Fund returns to core funding levels. If previous funding levels and average project funding amounts are an indication of future capacity, this would enable the Veteran and Family Well Being Fund to support approximately three to four projects in that annual application round. At the time of the evaluation, there were no specific plans in place to seek additional core funding.
The demand for funding from VAC has been high, as evidenced by the number of applications submitted each year, and the importance the department places on the Fund is evidenced by the continual temporary increases in funds. The temporary funding has allowed numerous additional projects to be supported for many years.
At this juncture, it would be timely for the department to consider if continuing with the Fund’s current intent and objectives are feasible based on the level of core funds available, or if any modifications to intent/objectives and/or core funding levels should occur.