Appendix A – Evaluation Matrix
| Issues / Questions | Indicators | Collection Methods | Data Sources |
|---|---|---|---|
| Relevance | |||
| 1. To what extent does the program continue to address a demonstrable need, now and in the future? | Number of applicants, favorable and unfavorable decisions. Key informant interviews and documentation review indicate ongoing need for program. Forecast of population indicates future demand for the program. |
Interviews/ Observations Survey GBA+ data analysis Document review |
VAC staff Staff survey Statistical and program data Legislation, departmental priorities/mandate letter, policies, processes, OVO reports Program documents |
| 2. To what extent does the program align with Government of Canada priorities and with federal roles and responsibilities? | Evidence of a Government of Canada commitment to Veterans in emergency situations (e.g. homelessness). Evidence of the program commitment included in departmental plans and priorities documentation. |
Interviews Survey Document review |
VAC staff Staff survey Legislation, policies, processes, VAC National Survey, OVO reports, budget documents, mandate letter Program documents |
| 3. To what extent is the Program responsive to the needs of recipients and target audiences? | Program eligibility enables Veterans who require the program to get access to it. Program funding is sufficient to respond to the needs of Veterans Prevalence of clients accessing the program on multiple occasions. Requests exceeding yearly maximums (requests for exceptions) Types of emergencies for which Veterans are requesting assistance Demographics of clients accessing program. |
Interviews Survey GBA+ data analysis File review Document review |
VAC staff Staff survey Program recipient feedback (program survey) Statistical and program data Veteran files Legislation, departmental priorities/mandate letter, policies, processes, OVO reports, financial reports Program documents |
| Performance | |||
| 4. Are processes currently in place to ensure the program is being administered as indicated in departmental guidance? | Presence of quality management activities. Program policy provides direction and definitions which build upon the legislation/regulations. Alignment of business processes with program policy. Appropriate governance structures in place to ensure accurate and consistent decision making Alignment of application/tools/assessments with program decision-making requirements. Adequate and appropriate training for decision makers. Alignment of payments to policy |
Interviews/ feedback from decision makers Survey File review Document review |
VAC staff Staff survey Veteran files Legislation, departmental priorities/mandate letter, policies, processes, guidelines Program documents |
| 5. To what extent is the program achieving its outcomes? | Program Information Profile in place and performance data is being collected, reviewed and assessed. | Interviews Document review File review |
VAC Staff Program recipient feedback Veterans files Statistical and program data |
|
Immediate – Veterans payments are made in a timely manner
|
Percentage of payments issued within 2 business days of approval | Data analysis | Statistical and program data Veteran files |
|
Immediate – Veterans have equitable access to VAC programs and services
|
Percentage breakdown of program clients by sex (M/F) compared to estimated Veteran population Percentage breakdown of program clients by sex (M/F) and age compared to estimated Veteran population Percentage breakdown of program clients by sex (M/F) and marital status Percentage breakdown of program clients by sex (M/F) and community (rural/urban) |
GBA+ data analysis GBA+ data analysis GBA+ data analysis GBA+ data analysis |
Statistical and program data Veteran files Statistical and program data Veteran files Statistical and program data Veteran files Statistical and program data Veteran files |
|
Immediate – Veterans and their families utilize the VEF
|
Number of Veterans and families who receive a VEF payment | GBA+ data analysis | Statistical and program data Program recipient feedback |
|
Intermediate – Veterans are referred to resources that assist them to manage their finances
|
Percent of Veterans who received referrals to other services | GBA+ data analysis Interviews Survey Document review File review |
Statistical and program data VAC staff Program recipient feedback (program survey), staff survey Program documents Veteran files |
|
Ultimate – Veterans are able to adapt, manage and cope within civilian life
|
Percentage of Veterans who report an easy adjustment to civilian life | GBA+ data analysis Interviews Document review Survey |
Statistical and Program Data VAC Staff Program Documents Program recipient feedback (program survey) LASS Survey |
|
Ultimate –Veterans and their families are financially secure
|
Percentage of Veterans who are satisfied with their financial situation | GBA+ data analysis Interviews Document review Survey |
Statistical and program data VAC staff Program documents Program recipient feedback (program survey) LASS Survey |
|
Ultimate – Veterans are satisfied with the services they receive
|
Percentage of clients who are satisfied with the quality of service delivery they receive from Veterans Affairs Canada | GBA+ Data Analysis Interviews Document review Survey |
Statistical and program data VAC Staff Program documents Program recipient feedback (program survey) VAC National Survey |
| Efficiency/Economy | |||
| 6. Are there opportunities to improve the efficiency and economy of the program? | Administrative costs ratio Processes allow for timely decision-making. Application process is timely and accessible. Opportunities for aligning with other departmental programs are identified and considered. Authority is delegated appropriately |
GBA+ Data Analysis Interviews Survey Document Review File Review Survey |
Statistical and Program data VAC staff Staff survey Program documents Veteran files Program recipient feedback (program survey) |
| 7. Are there any unintended impacts (positive or negative) that have occurred? | Positive and/or negative impacts realized from the program | Interviews Survey File Review GBA+ Data Analysis Survey |
VAC staff Staff survey Veteran files Statistical and program data Program recipient feedback (program survey) |