Services support Veterans with complex needs, including women, men and gender-diverse individuals with many identity factors (GBA+) in addressing their needs?

VAC’s Case Management Service is not a program, therefore, there is no requirement for a Program Information Profile^{Footnote 31} (Policy on Results). Without the requirement of a Program Information Profile, the associated outcomes for the service have not been readily tracked and monitored. The evaluation finds that to effectively monitor the performance of VAC’s Case Management Services, formal outcomes and associated targets must be established. This finding links to Recommendation #3.

In the absence of readily available outcomes measurement data, the evaluation team analyzed system data, undertook a manual file review, and reviewed public opinion research information to assess performance/outcomes associated with Case Management Services.

6.1 Case Plan Tool

In support of assessing outcomes associated with Veterans who were transitioned out of Case Management Services, the evaluation team first analysed data available through VAC’s Case Plan tool. The data provided information with respect to the volume of Veterans who transitioned out of case management into another level of support, and the number of goals/actions that were completed for these Veterans. The data showed that during the period from April 2014 to March 2018:

• 9,296^{Footnote 32} Veterans were transitioned out of Case Management Services.
  • The average length of time these Veterans had an open case plan was 978 days (2.7 years)
• 33,147 Desired Outcomes were recorded for these individuals, an average of 3.6 per Case Plan
  • 22,012 (66.4%) of the Desired Outcomes added were achieved, an average of 2.4 achieved per Case Plan.
• 19,615 Indicators for Success were recorded, an average of 2.1 per Case Plan.
• 59,328 Case Manager Action Steps were completed, an average of 6.4 Case Manager Actions Steps per Case Plan.
• 52,189 Veteran (Client) Action Steps were completed, an average of 5.6 Veterans (Client) Actions Steps per Case Plan.

Although the Case Plan Tool data identified overall volumes of activity, it could not provide sufficient information to identify whether Veterans’ complex needs had been addressed, or identify what domains Veterans were receiving support for (i.e., Health, Mental Health, Social, etc.). To gain further information in assessment of complex needs, the evaluation team; reviewed the results of the 2017 VAC National Survey, conducted an in-depth analysis of VAC’s case management tools completed at intake/transition out of case management (RRITs and CNCI), and completed a file review. The results of these individual activities are identified in the following three sections.

6.2 2017 VAC National Survey – Satisfaction with Case Management Services

The 2017 VAC National Survey^{Footnote 33} reported generally positive response results from case management recipients. The results showed that of the case management recipients surveyed (204):

• 83% agreed/strongly agreed that “My Case Manager and I have worked together to develop a plan to best meet my need.”
• 80% agreed/strongly agreed that “I have developed a productive working environment with my Case Manager.”
• 78% agreed /strongly agreed that “As a result of working with my Case Manager, I am better informed on how to access the services and supports I need.”
• 72% agreed/strongly agreed that “My case plan has helped me make progress towards reaching my goal.”
• 71% agreed/strongly agreed that “My Case Manager and I had regular discussions about my progress, my achievements and any problems that I was having.”

It is important to note that case management recipients surveyed were a random sample of overall clients in receipt of case management, and may have included clients who only recently started to receive case management support. Therefore, in these instances, results may be lower due to certain clients not having been in receipt long enough to have time to make progress towards reaching their goals and/or have discussions about progress/achievement/problems. In future surveys, it would be helpful for VAC to ensure the length of time in receipt of case management is included in the data for further cross-sectional analysis to be completed.

6.3 Analysis of CNCIs/RRIT-Rs completed at entry and transition out of Case Management

The evaluation team worked in collaboration with the Service Delivery and Program Management Division’s Business Intelligence Unit to assess results associated with RRIT-R and CNCI. In instances that these tools were completed at entry and when a Veteran was transitioned out of needing case management, key indicators/questions were analysed to determine if there was an improvement to the Veteran’s needs.

6.3.1 RRIT-R Analysis of Entry vs. Case Plan closure

The results associated with the RRIT-R analysis (entry to Case Management vs. Case Plan closure from Case Management) for the period from April 2014 to March 2018 are reflected in the following section.

In addition, an in-depth analysis of RRIT-R results was conducted to focus specifically on Veterans with poor self-rated physical health/mental health, and to review these results against key identity factors (GBA+). This analysis is reflected in Table 13 below.

Table 13: Analysis of Case Managers Impact on Veterans with Poor Self-Rated Physical Health

Key Demographical Indicator	# that reported poor self-rated physical health at entry	#/% with improved self-rated physical health at case plan closure
Overall Veterans	593	399 (67%)
Male	489	331 (68%)
Female	104	68 (65%)
Married/ Common-Law	397	262 (66%)
Single/Divorced	196	137 (70%)
Veterans under 40 years of age	56	42 (75%)
Veterans 40 to 50 years of age	212	148 (70%)
Veterans over 50 years of age	325	209 (64%)

As reflected in Table 13, improvements to self-rated physical health were generally consistent between gender and marital status. The biggest variance reflected was associated with the Veterans age group, and found that Veterans in younger age groups were more likely to report improvements to their physical health than Veterans in older age groups.

Table 14: Analysis of Case Managers Impact on Veterans with Poor Self-Rated Mental Health

Key Demographical Indicator	# that reported poor self-rated mental health at entry	#/% with improved self-rated mental health at case plan closure
Overall Veterans	682	528 (77%)
Male	568	438 (77%)
Female	114	90 (79%)
Married/ Common-Law	437	338 (77%)
Single/Divorced	245	190 (78%)
Veterans under 40 years of age	102	85 (83%)
Veterans 40 to 50 years of age	239	173 (72%)
Veterans over 50 years of age	370	294 (80%)

As reflected in Table 14, improvements to self-rated mental health were generally consistent between gender and marital status. The biggest variance reflected was associated with the Veterans age group, Veterans aged 40-50 were less likely to identify improvements in their mental health than Veterans that fell in younger or older age groups.

Overall, based on the analysis of the 1,853 RRIT-Rs completed for Veterans graduating/disengaging from case management, it was found that the differences were statistically significant.^{Footnote 34} It can be concluded that physical health, and mental health both improved when comparing Veterans self-rated results at the end of their case plan to their self-rated results at entry to case management.

6.3.2 CNCI Analysis of Entry vs. Case Plan Closure

An analysis of CNCI results was conducted to focus specifically on Veterans with considerable needs for improvement in each of the CNCI domains. The results, comparing needs at Case Management entry to Case Plan closure for the period of April 2014 to March 2018 are reflected in the following charts.

As identified above, in instances where Veterans had a “considerable” need for improvement within each domain, the majority of Veterans (76%-91%) had reduced levels of need at the end of their case plan.

Overall, based on the analysis of all 4,712 CNCIs completed for Veterans transitioning out of case management during the period of April 2014 to March 2018 (regardless of their level of need), it was found that improvements (differences) were statistically significant.^{Footnote 35} It can be concluded that all domains captured in the CNCI improved when comparing Veterans situations at the end of their case plan to their entry to case management.

6.4 File Review Results

During the period from July 2018 to September 2018, a review was conducted on 193 Veteran files^{Footnote 36} to assess needs associated with Veteran domains of well-being and associated changes during VAC’s Case Management Services. The sample was drawn from case plans:

• that were closed during the period of April 2016 to March 2018; or
• that have been open more than 2 years.

Each of the 193 files examined were reviewed to determine if the Veteran had an initial need in the following domains^{Footnote 37} :

1. Health - Physical;
2. Health - Mental:
3. Addiction related Mental Health;
4. Employment;
5. Finances;
6. Social Integration;
7. Housing; and
8. Life Skills.

If a need was identified in any of the above domains, the file reviewer recorded: the initial level of improvement required (considerable, moderate, or minimum), what the current level of improvement for the domain was (considerable, moderate, minimum, or none), and/or the level of improvement at the time of the case plan closure (considerable, moderate, minimum, or none).

The results associated with each domain, based on the Veterans initial situation vs. the Veterans current situation/situation when case plan was closed identified in the following chart.

Chart 2: Assessment of Veterans’ Needs- Initial Entry to Case Management vs. Current Situation/Situation at Case Plan Closure

*This data information can be provided in alternate formats by contacting vac.communications.acc@canada.ca.

As reflected, the majority of case managed Veterans (ranging from 63% to 85%) with needs in each domain are having their needs reduced during the receipt of Case Management Services. It is important to note, that the results associated with a Veterans change in level of need for employment and financial domains were found to be significantly different in instances where a Veteran was deemed to have Diminished Earnings Capacity (DEC)^{Footnote 38}.

In instances that a Veteran received an eligible decision for DEC, needs associated with employment decreased for 97% of the Veterans assessed. In instances that a Veteran did not have an eligible DEC decision, 50% of Veteran files reviewed had decreased levels of employment needs.

In instances that a Veteran received an eligible decision for DEC, financial needs decreased for 100% of the Veterans assessed. In instances that a Veteran did not have an eligible DEC decision, 75% of Veteran files reviewed had decreased levels of financial needs.

Charts identifying the assessment results for case managed Veterans initial needs vs. current needs/needs at the end of a case plan can be found in Appendix A.

In further analyzing the results of each domain, the evaluation team focused on the situations which identified Veterans that were most in need (had considerable need for improvement). As identified in Chart 3 below, through receiving Case Management Services, needs are being reduced for these Veterans.

Chart 3: Assessment of Veterans’ with “Considerable Need for Improvement”- Initial Entry to Case Management vs. Current Situation/Situation at Case Plan Closure

*This data information can be provided in alternate formats by contacting vac.communications.acc@canada.ca.

Changes in Complex Needs Based on Key Demographic Indicators (GBA+)

File reviewers were asked to provide an assessment of the Veterans’ overall complex needs for each file they reviewed. The results based on key demographic indicators for the 192 files are reflected in Table 15 below.

Table 15: File Review Results by Key Demographic Indicator

Demographic Indicator	Percentage with Reduction in Complex Needs	Percentage with maintained Complex Needs	Percentage with Increased Complex Needs
Gender
Male	65%	29%	6%
Female	58%	36%	7%
Marital Status
Married/Common Law	66%	31%	3%
Female	60%	30%	10%
Age
< 40	70%	14%	16%
40-50	58%	43%	0%
50-59	58%	35%	7%
>/=60	69%	28%	3%
Location
Rural	63%	28%	9%
Urban	64%	31%	6%

*May not add due to rounding.

As identified, complex needs are being reduced across all key demographical indicators available.