3.1 Internal controls for the calculation of benefits
Policies and procedures - calculation of benefits
Why it’s important
Policies and procedures are key to ensuring proper internal controls. They help ensure consistency, accuracy and efficiency in the work to be performed. Monitoring the implementation of policies and procedures through a quality control process helps ensure quality decisions by VAC employees.
It would be expected that policies and procedures for the IRB calculation of benefits would be documented with an adequate level of detail to provide guidance for both first applications and recalculations. In addition, it would be expected that senior management would determine what level of approval is required for functional direction (See appendix D for definition), and ensure those approvals are obtained. Due to the rapidly changing environment that IRB decision makers are working in, regularly scheduled reviews of procedures to determine if updates are necessary would be beneficial. In addition, it would be expected that there would be a quality control process in place to ensure decisions are being made in accordance with policy and procedures.
What we found
An IRB policy guides IRB calculation decisions, as well as multiple business processes and other functional direction documents.
The IRB policy provides a formula for IRB calculations, which is the same for both first applications and recalculations, as well as definitions for the variables to be used in the formula. VAC’s “How to Process Income Replacement Benefit Applications” business process does not fully support the policy in that it does not provide the steps for VAC employees to follow when calculating the benefit.
In addition to the IRB policy and business processes, there are other functional direction documents to guide IRB calculation decisions. This includes a manual, emails, checklists, etc. There are different formats used and they are not stored in central location. The IRB manual for first applications is not an approved document, but it does get updated by BPOs based on other functional direction the BPOs receive. Thus, there is a document in regular circulation and used by BPOs which may not be meeting management’s expectations for how to conduct the work. The audit team was unable to determine who was responsible for approving that document, as there was no guidance available and interviewees did not know. There was no IRB manual for recalculations. Not all of the functional direction documents are available in French.
There are minimal quality control processes in place for either first applications or recalculations to ensure calculation decisions are being made correctly and using proper supporting documentation (in accordance with approved policy and business processes).
To help test the internal controls that were in place for the calculation of benefits, a sample of 264 first application decisions was reviewed. Our audit testing began with:
- ensuring an eligibility decision was made for IRB;
- ensuring that the applicant signed for the truthfulness of information provided; and
- ensuring that the applicant was only being paid for one health condition.Footnote7
The test then went on to:
- re-calculate the monthly IRB payment for the first application, ensuring documentation was obtained to support the decision, and
- determine whether follow-up requests were sent for missing information.
An IRB eligibility decision had been made for all files in our review. With the exception of one IRB application during our review, all applicants had signed their application, providing a statement as to the truthfulness of the information provided.
In addition, for all files, Veterans received payment for only one health condition. The application form for the IRB did not request all information required to determine the MMS at time of release or completion of service to be used in the IRB calculation. The audit team was informed that this was because any information submitted by the Veteran requires verification by the BPO. It is easier for the BPO to gather the verification on behalf of the Veteran. However, there is no business process or functional direction for how to obtain or verify the MMS, and the IRB policy indicates that the Veteran is required to submit this information with their application.
Despite these gaps, IRB decision makers are diligent in obtaining MMS verification, and our audit work concluded that they are using the accurate MMS as of the time that the decision is made to calculate IRB first application payments.
The second component to an IRB calculation decision is income offsets. Our file review noted 11 first application decisions (4.2 per cent) that were missing documentation to support some income offsets. As described later in this report, the lack of clarity between the policy and the business process made it difficult to precisely determine through a file review whether supports for income offsets aligned with Departmental expectations.
It was determined through interviews, a review of documentation and the first application file review, that the recommendations from this report will help improve internal controls for both first applications and recalculations, so it was not considered necessary to review a sample of recalculation decisions.
What is the effect/impact?
The weaknesses described above may lead to a risk of incorrect or inconsistent IRB calculation decisions. There is also a risk of inefficiencies as VAC employees must sort through many sources of functional direction, or consult with peers, to obtain guidance for their work. A lack of documentation to support income offsets will create problems if errors arise and/or questions are raised. For example, if an applicant has questions about their benefit but supporting documentation is missing in VAC’s files, it will be difficult for VAC employees to respond to questions.
Training, tools, and information
Why it’s important
Training, tools, and information prepare VAC employees to make decisions in accordance with policies and procedures and guide them when questions or difficulties arise. Together, they help ensure consistency, accuracy, and efficiency in the work to be performed. It would be expected that a comprehensive training plan would be in place and kept up to date with changes in processes. After training, it would be expected that the proper tools and information technology systems would be available to support decision makers in doing their job.
What we found
Through interviews, BPOs felt they were adequately trained. Training, as described by BPOs, was mainly peer-to-peer job shadowing, with some trainees receiving ad hoc documents prepared by peers to use as reference documents.
Training modules exist, however they cover only basic applications and calculations, are short, and not updated regularly to reflect changes in processes. There is no documented training plan. Some tools exist to supplement the training, such as checklists and quick reference charts.
The tools available to guide decision making were discussed in the previous section on business processes and other functional direction. As a brief recap, functional direction needs improvements to ensure it is complete, accurate, approved, consistently presented, stored in a central location and available in both official languages.
VAC implemented GCcase, a new information technology system for managing client applications, in April 2019. GCcase was intended to process IRB applications, however BPOs are required to use both the old IT system (Client Service Delivery Network (CSDN)) and GCcase due to the limited functionality developed with GCcase. In addition, GCdocs, a Government of Canada repository for electronic information, was being used to store supporting documents. This was also due to GCcase’s limited functionality. It should be noted that this practice of storing supporting documents in GCdocs was discontinued while the audit was ongoing. However, BPOs are still using both CSDN and GCcase to process IRB applications.
Interviews with staff and management revealed a common theme: working in multiple systems is difficult and inefficient, and GCcase needs better functionality to better support staff to do their work.
What is the effect/impact?
Relying mainly on peer-to-peer training and job shadowing may help BPOs feel comfortable with their job responsibilities. However, it could result in inconsistencies in terms of how applications are processed and/or create bad practices during training (i.e., bad habits being passed on). In addition, it may be more efficient to have a training plan in place for new employees. Working in multiple systems makes it difficult for employees to do their work in an efficient and accurate matter.
There is potential for decisions not to be made as quickly or accurately if VAC employees are not properly trained and equipped with the proper tools to perform their duties.
Roles and responsibilities
Why it’s important
VAC employees need to understand expectations to perform their job to the best of their abilities. Clearly documented roles and responsibilities specific to the job being performed provide employees with an understanding of expectations. It also helps them determine priorities when workload management conflicts arise. The audit team expected to see roles and responsibilities, key deliverables, and timelines that were clearly defined for BPOs calculating IRB payments.
What we found
The audit team was not able to identify a specific document that clarified the roles and responsibilities of employees. However, this seemed to have minimal impact on BPOs’ ability to do their job.
BPOs generally felt their roles and responsibilities for calculations of benefits were clear and well communicated, however documents provided to the audit team were general in nature, and not specific to the IRB. The audit team was informed that a team in one location processes mostly first applications, while a team in a different location processes mostly recalculations. The intention is to have everyone cross-trained on both types of applications. There is no documentation that distinguishes roles and responsibilities between these two roles.
When roles are not clearly defined, there is the potential for unsatisfactory job performance and/or job satisfaction due to lack of clarity or the inability to properly evaluate performance.
Recommendations
- Essential: The Director General, Service Delivery and Program Management, should work in collaboration with the Director General, Centralized Operations Division, to update the business processes “How to process IRB applications for Veterans” and ensure the IRB manual and other functional direction documents are formalized, approved, stored in a central location and available in both official languages. The documents and their location should be shared with employees. A schedule should be implemented for reviewing and making updates, if necessary, to these documents.
Management response
The Director General, Service Delivery and Program Management in collaboration with the Director General, Centralized Operations Division, will review and update functional direction materials. The updated materials and existing “How to process IRB applications for Veterans” business process will be posted to a central location and available in both official languages. An email will be sent to all employees advising of the changes and location of the IRB guidance documents. Functional direction documentation will be reviewed every three years and on an as needed basis.
Target completion date: March 2024
- Essential: It is recommended that:
A) The Director General, Service Delivery and Program Management, should work in collaboration with the Director General, Centralized Operations Division, to develop a quality control process for IRB decisions; and
B) The Director General, Centralized Operations Division, should work in collaboration with the Director General, Service Delivery and Program Management, to implement the quality control process for IRB decisions.
Management response
The Director General, Service Delivery and Program Management and Director General, Centralized Operations Division, will collaborate to develop and implement a quality control process for IRB decisions. Quality control requirements and parameters for the framework will be determined. A sample of completed IRB decisions will be reviewed using the new framework; results from the review will be analyzed to identify trends. The findings, and any associated recommendations, will be shared with management, staff, and key stakeholders.
Target completion date: November 2024
- Essential: The Director General, Service Delivery and Program Management, should work in collaboration with the Director General, Centralized Operations Division, to develop and implement a training plan for IRB decision makers that includes updated training materials for both first applications and recalculations.
Management response
The Director General, Service Delivery and Program Management, in collaboration with the Director General, Centralized Operations Division, will develop a curriculum and update the current training materials based on IRB decision makers’ feedback and implement new training materials for IRB recalculations and first awards. An IRB training schedule (or map) will be available online in both official languages. A communication will be sent to IRB decision makers promoting the new tools and training products.
Target completion date: December 2024
- Essential: The Director General, Information Technology, Information Management & Administration (ITIM & Admin), in collaboration with the Director General, Service Delivery and Program Management, should implement system enhancements to eliminate the need to work in multiple systems for the processing of IRB claims (including supporting documentation).
Management response
The Director General, Information Technology, Information Management, Administration and Privacy Division in collaboration with the Director General, Service Delivery and Program Management, will collaborate to document the system requirements required to eliminate the need to work in multiple systems to process IRB claims. The priority of this effort will be reviewed in relation to other Departmental priorities to ensure alignment and prioritization of initiatives. A roadmap will be created that will identify the steps required to get to the point where employees are processing IRB claims from a single user interface. Once all the steps are complete, the required system enhancements will be implemented.
Target completion date: March 2028
3.2 Risk management of income offsets
Income offsets risk assessment
Why it’s important
Income offsets represent a significant component of the calculation of IRB payments and are sometimes a complex figure to determine. It can also be difficult to obtain supporting documentation for income offsets. The audit team expected to see a documented approach to risk management for IRB – including a risk assessment for income offsets – at the planning stage for the program, and VAC’s risk tolerance toward income offsets.
The audit team also expected that income offset risks were being mitigated and monitored and that the risk assessment was being updated as the program evolved, and as new risks were identified. Risk management discussions and decisions should be documented on a regular basis, should consider the controls that are currently in place, and the resulting impact and likelihood of the occurrence of fraud or error.
What we found
The audit team could not determine who was accountable for risk management at the program level, which would include income offsets risk. The areas responsible for program delivery identify risks, which is one step in risk management. However, they indicated during interviews with the audit team that they are not tasked with the responsibility of a formal risk management process as it relates to IRB. We were unable to find documented responsibility for risk management at the program level, and therefor unable to find documented responsibility for risk management of income offsets.
The audit team wanted to determine whether VAC had completed a risk assessment for income offsets when planning and developing the IRB program. It was found that VAC identified risks for Pension for Life, however it was a wide ranging risk identification for a broader initiative that did not include the detailed levels to the individual program level of IRB, let alone for the income offsets component of the program.
Service Delivery and Program Management (SDPM) informed the audit team that during the audit scope period, an informal risk assessment was completed for income offsets. SDPM held meetings and discussions but no minutes or documentation of meetings were produced. There was no documented process for identifying the likelihood and/or impact of risks. SDPM did identify and document risks as part of a Treasury Board submission in 2022.
Overall, the audit team found little evidence to show that program risks, including income offset risks, are being managed. Had responsibilities for risk management been identified and documented, and a process put in place at the outset of the IRB program, it would be expected that better controls would be in place for income offsets (see the next section of audit report for more information).
The identification of risks by SDPM is a good starting point, but it lacks the level of depth expected for effective risk management. Identifying risks as part of a Treasury Board submission does not lend itself to regular monitoring and updating of risks. It would be preferable to have clearly identified who is responsible for risk management, how it should be undertaken and documented, and how the monitoring and follow up should happen.
Some risk management factors that should be considered include, but are not limited to:
- Consideration for the controls that are currently in place;
- The resulting impact and likelihood of occurrence of fraud or error under the current program
- For example, what are the chances of over or under payments because of this risk?
- What is the potential dollar impact of over or under payments?
- Consideration of how this aligns with VAC’s overall risk tolerance.
What is the effect/impact?
When there is no clear line of accountability for the risk management process at the program level, and no documented and clear process in place for risk management, it is difficult to manage the risks to the program. It is also difficult for management to determine whether risks are being mitigated in accordance with expectations and risk tolerance. In particular, income offsets was identified to the audit team as an area of risk, but it does not have formal risk management processes in place.
If risk discussions are not documented, the transfer of knowledge will be more difficult when there is employee turnover. In addition, there could be a duplication of effort regarding challenges and issues that have already been identified and assessed.
There may be limited resources available to address competing priorities. Lack of documented risk tolerance and assessment of risks will lead to less informed decision making concerning where resources should be prioritized.
Response to income offsets risk
Why it’s important
Proper internal controls regarding income offsets would help ensure consistent and efficient decision making, while decreasing the likelihood of an overpayment or underpayment of IRB. It would also ensure compliance with the requirements per the Regulations, while ensuring good financial stewardship.
The audit team expected to see that VAC had implemented clear requirements for BPOs to obtain supporting documentation for income offsets and that these requirements aligned with the Veteran Well-being Regulations. We expected that these requirements would lead to BPOs obtaining supporting documentation for income offsets that was complete and appropriate.
What we found
IRB applicants and recipients are required to report their income offsets to VAC. They are also required by the Veterans Well-being Regulations to submit an annual (at any time during the year) statement of income to VAC.Footnote8 The Department relies on the Veteran to identify, report and submit supporting documentation for their income offsets. If all income is not reported or there are changes to income that are not reported, it could result in IRB over or under payments. Relying on the applicant to submit supporting documentation for their income offsets could also lead to an accuracy risk, if the documentation submitted is inadequate.
It would be expected that VAC would implement internal controls to reduce these risks to an acceptable level. The controls identified included requesting a statement of income upon initial application and if the application was over six months old at the time of calculation of the IRB first application. As the information to support income offsets often is not provided with the application or is insufficient, VAC program employees work with several external stakeholders, such as Manulife, the Canadian Armed Forces (CAF), and/or the Veterans and their family to obtain the required permissions and documentation to support an IRB payment. There are short training modules and a checklist provided to BPOs on the sources of income offsets and acceptable forms of verification. In addition, there is a business process that BPOs can follow for suspension and cancellation of IRB due to a lack of financial information.
The above noted processes show that an effort is being made to obtain support for income offsets, however, it is inadequate from an internal controls perspective. The requirements of the Veterans Well-being Regulations, which indicate that IRB recipients are to submit an annual statement of income, are not being met by the current processes in place. There is no business process that directs staff to request or obtain an annual statement of income. Currently, BPO’s request statements of income upon initial application, and on an ad hoc basis after initial application.
Secondly, the processes in place to reach out to stakeholders to obtain required information are not documented in any business process, and do not ensure the income reported is complete. There should be a business process in place that provides guidance for employees on income offsets. Finally, there is no evidence of approval for the business process “Suspension, Reinstatement and Cancellation of the IRB-Due to Lack of Financial Information.” The business process can’t be easily accessed by employees on VAC’s internal website, and it is not available in French. The business process should be approved in both official languages and be easily accessible for employees.
To further expand upon these issues, the business process “Suspension, Reinstatement and Cancellation of the IRB-Due to Lack of Financial Information” indicates that if the Veteran does not provide support for income offsets, their IRB may be suspended or cancelled. The requirements for BPOs to obtain the supporting documents are not clear, as the policy indicates that the Veteran “must” provide information for the purposes of the calculation, yet the business process indicates the BPO “may” suspend or cancel IRB. This leaves a gap for what actions a BPO is required to take to obtain supporting documentation. If they do not obtain it, they do not have to suspend or cancel based on the business process. BPOs should have very clear requirements for when they are to suspend IRB.
Figure 3 below notes the variations in functional direction for providing/obtaining documentation.
Figure 3
Comparison of Approved Functional Direction for Income Offsets
Source: AED Analysis based on review of documentation
Due to the lack of clear processes and policies, it was difficult to precisely determine through a file review whether supports for income offsets aligned with expectations of the Department. It was evident through interviews and our file review procedures that that Department’s intentions were to obtain support for all income offsets prior to the first application for IRB being calculated. As mentioned earlier in this report, 96 per cent of files reviewed contained support for first application income offsets.
Generally, BPOs are not following the business process to suspend or cancel due to lack of supporting information. In many cases, BPOs issue letters to Veterans to obtain the documents to support both the IRB first application, as well as subsequent recalculations. There are several different letters being issued to Veterans which differ from those listed in the business process, and the timelines (as written in the business process) are generally not followed. There is a reluctance to suspend the IRB, with Veterans being given longer than the timelines suggest prior to suspension or cancellation.
What is the effect/impact?
These issues pose many risks to the organization. Some of them include:
- Compliance risk: There is a risk that VAC will continue in its non-compliance with government regulations.
- Financial risk: VAC is at risk of making under or overpayments. Further, there is a risk that overpayments may go unidentified or may not be collected on a timely basis.
- Without a process in place to verify the completeness of the income offset information submitted, VAC may not be maintaining a system of internal control to provide assurance that public resources are used prudently and in an economical manner as required by Treasury Board Policy on Financial Management.Footnote9
- Reputational risk: Once identified, the under or over payment has the potential to cause the Veteran unnecessary stress and impact their health and well-being if it is determined that they have received incorrect payments.
- In addition, introducing a new component (i.e., annual income verification) to a program that has been in effect since 2019 poses a reputational risk, as Veterans and stakeholders may ask what has changed to require this additional effort by the Veteran (when in reality, it was required since the program began).
- Reputational risk: Another reputational risk involves equitable treatment.
- If there is a lack of clarity for when a BPO must suspend the IRB, there is a risk that applicants are not being given an equal amount of time to submit required information for income offsets before being suspended.
- Operational risk:Operational documentation that isn’t properly approved leads to a risk of following direction that is not the intent of management.
Recommendations
- Critical: The Assistant Deputy Minister, Service Delivery, in collaboration with the Director General, Strategic Planning, Results & Cabinet Business (SPRCB), should define and communicate responsibilities for risk management at the program level. A risk management process should be developed for the IRB that includes documented identification of risks, assessment of risks, risk response and regular updating and monitoring of the risks.
Management response
The Director General, SDPM, has been identified as the position responsible for program level risk management. Using resources available through SPRCB, the Director General, SDPM, will identify and document the top risks facing the IRB program. The risks will be assessed and mitigation strategies, where necessary, will be developed and implemented. Going forward, the program’s risk profile will be reviewed annually taking into consideration current risks facing the program.
Target completion date: December 2024
- Critical: The Director General, Service Delivery and Program Management, in consultation with the Director General, Centralized Operations Division, should improve the verification of income offsets to include:
- Requirements of the Veterans Well-being Regulations;
- A completeness check to ensure Veterans have reported all income offsets for the year;
- Clear direction for BPOs on processes and requirements for income offsets to aid in decision making; and
- Impacts to the applicants’ IRB if they do not provide an annual statement of income, information required for the completeness check on income offsets, and/or sufficient support for income offsets.
Management response
The Memorandum of Understanding in place with the Canada Revenue Agency has been updated to include the provision of IRB data that will be used to validate income offsets. The Director General, SDPM, in consultation with the Director General, Central Operations Division, will ensure the annual income verification process encompasses the Veterans Well-being Regulation requirements and create a streamlined process for the submission and processing of income offsets. To aid in the efficiency of the process, system updates will include
- The automated distribution of the IRB Annual Verification Form;
- The ability for Veterans to update income offset information through My VAC Account; and
- Auto populating updated income offset information within CSDN.
Target completion date: December 2024
3.3 Audit opinion
The audit opinion is provided based on the scope of the audit, which was identified as the internal controls, legislation, policies, business processes, decisions and other documentation that relate to an entitled Veteran’s calculation of the IRB and the support for income offsets (between 1 April 2020 and 31 March 2022).
Controls in place over the calculations of the IRB require improvement. There is a need to improve the clarity and authority for key functional direction. There is a need to implement a quality control process, additional, consistent training for all decision makers, and system enhancements to eliminate the need to work in multiple systems.
Risk management practices at the program level require improvement. Although risks related to income offsets have been identified, there is a lack of clarity over roles and responsibilities for risk management over income offsets and at the program level. There are no documented IRB risk management practices in place. Internal controls for income offsets do not address all requirements of the Veterans Well-being Regulations, are ambiguous in terms of the requirement to obtain verification for income offsets, and do not ensure the completeness of offsets reported for IRB.