The evaluation was conducted in accordance with VAC’s 2017-22 Risk-Based Audit and Evaluation Plan, and in compliance with the directive and standards specified in the Treasury Board of Canada’s 2016 Policy on Results.
In support of developing the scope for the evaluation, a risk/calibration assessment was completed as informed by preliminary interviews, a document review, and data analysis. Based on the risk assessment results, as well as the identified need by the program area, it was agreed that the evaluation would focus on the preferred admission beds initiative.
The preferred admission beds initiative has been the biggest change within the Program since it was last evaluated in 2013-14.
The time period covered by the evaluation was April 1, 2014 to September 30, 2018.
2.1 Multiple Lines of Evidence
The evaluation was formativeFootnote 10 in nature, utilizing a research methodology that incorporated multiple lines of evidence to ensure reliability of collected information and reported results. The lines of evidence used to evaluate the preferred admission beds initiative are shown in Table 3.
| Methodology | Source |
|---|---|
| Departmental Documentation and Secondary Research Review | The following Departmental documents/information were reviewed to understand the program objectives/intent, its authorities and requirements, complexity, context and any key issue areas: departmental planning documents, previous audits and evaluations, legislation, regulations, policies, business processes, records of decisions, strategic documents, performance reports, research papers, survey results, and correspondence. |
| Non-Departmental Document Reviews | Various non-departmental documents were reviewed such as: program literature from both federal and provincial government departments/agencies, Parliamentary reports, Budget Speeches, industry research, media documents, and Speeches from the Throne. |
| Interviews | Forty-nine interviews were conducted with: VAC senior management; VAC staff involved in the management and operations of the LTC Program and VIP-IC; subject matter experts; and administration/staff at select long-term care facilities. |
| Statistical Analysis | Financial and operational data collected by VAC for fiscal years 2014-15 to 2017-18 was analysed, where available. Client forecasts and demographic data was also analysed. |
| Site Visits and Observations | Site visits were conducted at five former Veterans hospitals and five VAC Area Offices to speak with staff and long-term care providers. Facility observations were also conducted. |
| File Review | A file review of 211 Veterans who accessed long-term care through preferred admission beds was conducted to develop a profile of recipients and to provide a secondary line of evidence to corroborate evaluation findings. |
Key evaluation issues and questions are contained in Appendix B.
2.2 Limitations and Considerations
The limitations and considerations noted below should be considered when reviewing the evaluation findings:
- The Program has evolved over the years to accommodate Veterans’ needs.
- Delivery of the Program is complex with numerous eligibilities, models of funding and delivery partners (e.g., provinces and health authorities; not-for-profit; and for profit facilities etc.).
- Regional, socio-economic, and demographic differences across the country result in diverse provincial long-term care systems and varying levels of demand for long-term care services.
- Limited program performance measurement information has been collected and monitored since the preferred admission beds initiative was launched in June 2016. The evaluation team conducted a file review and analysed available data to support the evaluations findings and conclusions.