2.1 Evaluation scope and questions
The evaluation was conducted according to the Treasury Board of Canada’s 2016 Policy on Results as well as with Section 42.1 of the Financial Administration Act which requires federal departments to review the relevance and effectiveness of ongoing programs for which they are responsible every five years.
The scope of the evaluation includes all the activities of the program between 2015-16 to 2019-20, with the exception of the Intermediate Care component of the VIP. The Intermediate Care component was included in the 2019 evaluation of the department’s Long-Term Care Program. The evaluation questions are included in Appendix C.
2.2 Multiple lines of evidence
The research methodology for the evaluation incorporated multiple lines of evidence to ensure reliability of collected information and reported results (Table 1).
| Methodology | Source |
| Departmental Documentation and Secondary Research Review | Departmental documentation and information were reviewed to understand the objectives of VIP and its intent, authorities, requirements, context and delivery. Documents included departmental planning documents, VIP policies, business processes, strategic documents, performance reports, research papers, a training module, and results from VAC’s National Client Survey (2020).Footnote 10 |
| Literature Review | Various non-departmental documents were reviewed, including peer-reviewed and non-academic research on the state of home care, in general, in Canada and the continuum of care. A comparative review was conducted that described similar programs to VIP in other jurisdictions.Footnote 11 |
| Interviews | In total, there were 25 interviews, across six groups, during which more than 60 VAC staff and external stakeholders were interviewed. The six groups were Senior Management, VAC VIP Managers and staff, other VAC Managers, Veterans Organizations, external Health Claims Processor staff, and external subject matter experts (researchers in home care, Veteran and gerontology. VAC staff were interviewed in the following areas: program and strategic policy, research, field operations, and service delivery program management. |
| Statistical Analysis | Statistical analysis included:
|
| Survey and Focus Groups with VIP Decision-makers | VAC staff (Veterans Service Agents, Team Managers and Case Managers) who make decisions about VIP applications were surveyed. Completed by 252 decision-makers, this survey had a response rate of 33%.
Further to the survey, two focus groups (one in English and one in French) were conducted to explore findings from the survey and gather qualitative feedback. |
| Survey of VIP Clients | A sample of VIP recipients were surveyed via VAC’s My VAC Account system, mail, and telephone for those who do not have a My VAC Account. It was completed by 269 VIP recipients representing a response rate of 34%. |
2.3 Considerations and limitations
The following limitations were identified and should be considered when reading the evaluation findings:
- Certain administrative and performance data (e.g., reasons for receipt of VIP, transactional data such as rate exceedances, disaggregation of program data by demographic characteristics) were not readily available so qualitative methodologies were used to assess some elements of effectiveness and efficiency.
- While the evaluation includes direct feedback from program recipients, the representativeness of the survey of clients conducted for this evaluation could not be assessed. Based on a predominantly online approach via My VAC Account and given the profile of these clients, the survey responses underrepresent older Veterans, survivors and PCGs.Footnote 12