4.0 Performance and Efficiency/Economy

The VEF Performance Information Profile (PIP) and logic model, which displays the outcomes for the VEF can be viewed in Appendix D.

4.1 Program Governance

The program area has demonstrated a commitment to program governance, in particular during its response to the unique needs that emerged during the Covid-19 pandemic. The program’s quality assurance function is a best practice that should be continued.

The evaluation finds that the program area has demonstrated a commitment to program governance. Weekly reporting and daily tracking of program dollars is ongoing, as is work by VAC’s Business Intelligence Unit to conduct and publish analysis of public opinion research that will provide a clearer picture of VEF recipients and their complex needs.

4.1.1 Quality Assurance

To further strengthen program governance, a quality assurance function was implemented by the program area. The evaluation finds that this function provides oversight and allows program management to identify and respond to issues in the field as they arise. As a result of issues/concerns identified during the course of the quality assurance function, the following actions have been taken by the program area:

• Weekly in-depth reports to Service Delivery Program Management, Finance, and Field Operations were developed (e.g., number of clients accessing the program per week, dollars disbursed etc.).
• A learning and sharing event was held in 2019 with field staff to discuss the program, share best practices, and provide further guidance. The event brought together Area Directors (ADs), Veteran Service Team Managers (VSTMs), Business Support Training Officers (BSTOs), Standard Training and Evaluation Officers (STEOs), Veteran Service Agents (VSAs) and Case Managers (CMs).
• The VEF application was modified to add space for the applicant to provide additional information about the emergency. Staff indicated they wanted to know more details prior to calling the applicant so they might provide referrals during the first contact (as mentioned previously, the VEF is the only VAC program where direct contact is made with the applicant prior to rendering a decision).
• The development of additional tools and reports to assist decision makers is ongoing based on observed need during the quality assurance process.

The VEF quality assurance function is a best practice that should be continued by the program area.

4.1.2 Covid-19 Interim Business Processes

The evaluation notes that program management quickly adopted new business processes at the onset of the Covid-19 pandemic to ensure that the VEF program was maintained in the face of closed Area Offices and staff working from home. Approximately one week after the unexpected closure of VAC offices, an interim business process was adopted to ensure all VSTMs received acquisition cards to provide payments to recipients where appropriate. In addition, by March 30, 2020, VSAs and CMs were granted expanded authority to approve VEF funding up to $10,000 per year per recipient under an interim business process. The interim process also gave VSAs and CMs the ability to provide Veterans and their families support for up to three months of requested expenses (subject to the $10,000 per year maximum).

The evaluation team reviewed VEF decisions made during the first three months of the new interim business process and observed that VSAs and CMs are exercising their new authority where appropriate and requesting supporting documentation when necessary (especially when requests exceed the standard threshold of $2,500). The evaluation finds that field staff are generally exercising due diligence in their decision making. Bank statements, invoices, and other supporting documentation are being requested to help determine need and to determine if the Veteran is in an emergency or potential emergency situation.

It should also be noted that the evaluation team was provided evidence to show that 50% of all approved applications and 100% of disallowed applications during the first three months of the Covid-19 pandemic were reviewed as part of the quality assurance function to ensure applications were being assessed correctly against the new procedures.

At the onset of the Covid-19 pandemic, a working group consisting of STEOs and head office program experts was established to discuss VEF decisions and questions posed by field staff about the interim business process. The evaluation finds that this forum provided an opportunity for an exchange of information and a means to bring information back to decision makers. It should also be noted that due to the short turnaround time for processing a VEF application, the program area has also made itself available to answer questions directly from field staff (when a STEO is not available) to ensure correct decisions are being made.

4.2 Achievement of Expected Outcomes

VEF requests are administratively burdensome for decision makers. However, VEF payments are generally being made within the 48 hour service standard.

Interviews with frontline staff and head office staff indicated that the VEF creates a large administrative burden (both for those directly serving Veterans and those managing the program). Of particular concern, 43% of frontline staff surveyed indicated it takes four or more hours to process a VEF decision. It was commented that decisions for other VAC programs with larger financial impacts are much easier to action and document. See Figure 8 for a breakdown of estimated time to process a VEF decision.

Figure 8 – Response to Survey Question “Estimate the Length of Time to Complete a VEF Decision”

Figure 8 – Response to Survey Question “Estimate the Length of Time to Complete a VEF Decision”

2 hours or less	17.4%
2 hours to less than 4 hours	40.5%
4 hours to less than 6 hours	16.5%
6 hours or more	25.6%

The use of CSDN to process decisions quickly in an emergency situation poses significant challenges. CSDN does not have clear workflows for VEF decision makers to follow and information has to be manually entered into several screens. The survey of decision makers revealed that 80.5% of respondents strongly agreed or somewhat agreed that the process for documentation a VEF decision in CSDN is complex. The structure and nature of the current system has required program management to develop a lengthy business process to support decision makers in completing the steps required to render an eligibility decision.^{Footnote 12}

Frontline staff currently document VEF decisions in “decision dockets” with accompanying client notes within VAC’s CSDN. A review of client files revealed that VEF decisions are generally well documented. However, CSDN is an older system that has significant limitations in data collection. Decision makers are required to manually enter information from the application into multiple screens of the decision docket. This same information then has to be manually added to a letter to the client and followed by a paper form to be signed, scanned, and then emailed to the departmental pay centre to issue payment.^{Footnote 13} A lack of integration with other VAC systems increases the administrative burden on decision makers. In addition, the system does not allow for easy data retrieval required to support program management.

Many decision makers (especially newer staff members) have little experience with developing decisions dockets in CSDN and must follow step-by-step instructions each time.^{Footnote 14} The CSDN system is unforgiving in that input errors often result in starting the process over again or require an intervention by IT staff to correct the situation. As VEF applications are comparatively rare (77% of survey respondents report dealing with VEF applications “infrequently”) it is difficult to build expertise in the system.

The administrative steps required to process a VEF claim in its current system infrastructure consume a significant amount of time for front line staff which impacts their ability to complete other duties. Additionally, accurate data retrieval from CSDN is impacting the program area’s (and the evaluator’s) ability to apply a fulsome GBA+ lens to the program. As a result, the evaluation makes the following recommendations:

Recommendation #1

It is recommended that the Director General, Service Delivery and Program Management implement measures to reduce the administrative burden on decision makers by exploring system changes to simplify the VEF process.

Recommendation #2

It is recommended that the Director General, Service Delivery and Program Management implement system changes to improve the capture of program performance data and to better capture GBA+ data/program recipient demographic data.

4.2.1 Turn Around Times

Though it does take a considerable amount of time to document VEF decisions, the evaluation team’s review of client files and interviews with staff revealed that payments are generally being made in a timely manner by the departmental pay centre once a decision has been made. Figures compiled by VAC’s Finance Division show that, on average, payments are being made within two days from the date of decision 79.5% of the time (the program service standard is 80%). See Table 11 for more details.

It should be noted that the current operating environment (e.g., staff working from home, office closures etc.) has had little impact on VAC’s ability to deliver VEF funds in a timely manner.

The departmental pay centre issues direct deposit payments when possible, either to the client or to a third party. However, as Table 12 shows priority cheques continue to make up a large portion of VEF payments (41% in 2018-19 and 40% in 2019-20). Cheques are often necessary when applicants do not have access to a bank account or when third parties cannot accept direct payment.

Multiple payment types are sometimes used when an applicant is approved for more than one type of assistance. For example, an applicant may be approved for emergency assistance with food costs and rent. The food portion of the payment may be made by direct deposit to the applicant’s account, while the rent portion may be paid directly to a landlord via a priority cheque. Flexibility with payment methods allows VAC to respond quickly in the most effective manner to the applicant’s needs. It also allows VAC staff to exercise discretion when distributing large sums of money to ensure the needs of vulnerable Veterans will be addressed.

4.2.2 Observation – VEF Eligibility Criteria on VAC Website

As indicated in Section 3.1.1, Veterans generally know about the VEF, though they may not know the eligibility criteria. Thus, Veterans sometimes apply to the fund with needs that are not considered an emergency (and are subsequently declined or voluntarily withdraw their application). More than 70% of survey respondents reported that they had encountered situations where there was confusion/misinterpretation of the program benefits or eligibility criteria. The evaluation team reviewed the VEF homepage on the VAC website and discovered it does not list the entire eligibility criteria for the fund; it lists what may be considered an emergency, but does not identify the types of issues that the policy deems ineligible. Given the administrative burden to render a VEF decision, the evaluation makes an observation that listing the full eligibility criteria on the VEF page may reduce the number of applicants applying with issues that are not deemed an emergency for the purposes of the program.

4.3 Operating Costs and Level of Administrative Effort

Operating funds were not allocated for the VEF program during initial program funding; since 2018, the program has been operating within existing reference levels. Operational impacts and costs have been absorbed by VAC’s Service Delivery Branch.

There are currently four staff partially assigned to the program at head office at the levels and intensity indicated in Table 13.

In total, there are only 1.5 full time equivalent employees assigned to the VEF at head office responsible for all program management duties such as quality assurance, program planning, reporting, training development, and continuous response to field requests (amongst other duties). In addition to head office staff, responsibility for program decision making is spread amongst the 700+ frontline decision makers. The level of effort, expense, and operational impacts caused by the additional VEF workload have been absorbed by Service Delivery Branch.

Due to the pandemic, the evaluation team was not able to directly observe the decision making and documentation process. However, given the number of steps in the business process, comments gathered through the survey of decision makers, and interviews with staff, evidence points to a decision making and documentation process that is excessively lengthy. Using a conservative 4 hrs as the time it takes a decision maker to administer a VEF application, it is estimated that the VEF has accounted for 12,000 hours of front line staff time since program inception (given the 3000+ decisions that have been made). This does not take into account training time (including the time STEOs spend on the program) or the time spent on the program by the departmental pay centre. The level of effort is particularly noteworthy given the scope of the program and respective funds disbursed.

4.4 Training and Guidance

Though VEF training has been developed and implemented, a requirement for continued and enhanced guidance from the program area was indicated by staff interviewed and surveyed as part of the evaluation. As stated previously, VEF eligibility is intentionally broad, meaning decision makers often encounter unique situations that their training has not necessarily covered. The survey of decision makers identified that staff find determining whether a situation meets the definition of an emergency and determining an applicant’s liquid assets as the top two most challenging aspects of making a VEF decision. These are potential areas for training focus.

4.5 Unintended impacts

Community aid organizations are referring Veterans to the VEF for assistance when they would normally provide the assistance.

Interviews with field staff indicated that community services may be referring Veterans to the VEF rather than addressing the Veteran’s needs. A survey of VAC decision makers confirmed that 35% of staff have encountered situations where Veterans were told to contact VAC/apply for the VEF by another organization that would normally provide the type of aid being sought. This has the potential to place strain on the VEF over the long term if more organizations take a “hands off” approach to providing services to Veterans.