2.1 Rationale
Disability benefits is being audited because:
- they are a gateway to access other VAC benefits and services;
- they are the highest dollar value benefits delivered by VAC (total expenditures of $2,031.1 million for the fiscal 2016-2017, representing 60% of total program spending by VAC that year);
- there is a processing backlog of applications;
- service standards are not being met;
- over 90% of individuals being served by the Department are receiving disability benefits; and
- they have not been subject to an internal audit in more than five years.
Data integrity and performance data was identified as a key area of focus as prior studies pointed to a need for improvements in this regard, interviews with key stakeholders identified the availability of performance data as a risk, and observations by the audit team pointed to some data integrity issues. It is essential to have reliable data to manage programs and make improvements in performance.
2.2 Audit Scope and Objectives
The scope of the audit includes operational performance data and data integrity related to first applications for disability pensions and disability awards. The audit included a review of CSDNFootnote 2, My VAC AccountFootnote 3 and the VAC Enterprise Data Report PortalFootnote 4 to ensure integrity of data and appropriateness of performance data. Data gathered at the intake, claims preparation and adjudication steps were included in the audit. The audit covered data integrity and performance data for the 2017-18 fiscal year. The audit excluded the payment processing due to favourable results over that process from previous quality assurance exercises.
The audit objectives include:
- To provide assurance on the integrity of the data related to the Disability Benefits
- To assess the relevance of operational performance data related to the Disability Benefits
The audit criteria are provided in Appendix A.
2.3 Methodology
The audit findings and conclusions contained in this report are based on sufficient and appropriate audit evidence gathered in accordance with procedures that meet the Institute of Internal Auditors’ International Standards for the Professional Practice of Internal Auditing as supported by the results of the quality assurance and improvement program. The opinions expressed in this report are based on conditions as they existed at the time of the audit and apply only to the entity examined.
| Methodology | Purpose |
|---|---|
| Direct Observation | Direct observation was conducted at all regional Disability Benefits Units (DBU) and VAC Head Office. Observation was done on how reports are generated, how data is entered and how information flows throughout the system. Direct observation was also used to test system controls. |
| File Review | A file review using a sequential sampling methodFootnote 5 was completed on a randomly selected sample of 73 first applications to assess the accuracy and completeness of data; the reasons for errors; the impact of incomplete/inaccurate data. |
| Data Analysis | Data from CSDN system was analysed to test system controls, identify any data integrity and performance data weaknesses. |
| Documentation Review | A review of business documents and performance reports was undertaken to determine the relevance of performance data and data integrity. |
| Interviews | Interviews were completed with 29 staff and managers responsible for disability benefits and information technology specialists to determine data integrity issues, the rationale for errors and if current performance data meets the needs of identified users. |
| Questionnaire | A questionnaire was sent to managers to determine if the current performance data meets their needs and where data gaps exist. |