3.1 Agile at VAC
Why it's important
Under the Policy on Service and Digital and Government of Canada Digital Standards departments are encouraged to innovate and experiment in how they design and deliver their services, and to adopt agile, iterative and user-centred methods where possible. Effective system development practices can help deliver faster and better tools to support service delivery.
What we found
VAC’s first major adoption of agile methodologies was in 2018 with the Pension for Life project to develop and rollout a new Veterans benefit program employing a case management IT application called GCCaseFootnote 2. The ambitious project had a tight timeline to launch the new system which lended itself to an agile approach for rapid development. Employing this new approach required changes to how the Department delivered projects. New cross functional scrum teams were formed, agile training was provided to staff, and new tools and processes were put in place to document the system development process. In 2019 the Department rolled out the first working version of the new system, referred to as a minimum viable product, and since then has been working on ongoing improvements to the GCCase system.
Scrum: an agile system development methodology which organizes development teams into collaborative groups to iteratively develop digital products.
Following the Pension for Life project the Department opted to employ the same scrum approach for all future IT projects and system development in the Department. To support this approach, the IM/IT group formalized new team roles, such as a product owner and scrum master, and put in place a process to develop systems in an iterative fashion with multiple small phases of work called sprints. This approach is highly aligned with the Government of Canada Digital Standards.
3.2 Frameworks and guidance
Why it's important
Clear and comprehensive guidance facilitates the adoption of new ways of working, improves efficiency and standardization, and helps the growth and learning of staff working to develop digital services. Under the Policy on Service and Digital departments are responsible to support innovation and experimentation in service and IT, including the adoption of agile development practices. Under the Directive on the Management of Projects and Programmes, departments are responsible to establish, document, and maintain a department-wide project and programme management framework.
What we found
In 2018 VAC launched the Project Management (PM) Renewal Project to revise VAC’s project governance framework and update departmental project management templates. In 2019 VAC’s Enterprise Project Management Office (EPMO) updated the Department’s project governance framework to align with Treasury Board requirements. The framework functions to articulate the gating requirements for departmental projects, but does not reference agile development and is not a tool designed to provide guidance on how to employ project management to agile development.
Benefits realization: project management methodology to track if a project or initiative is achieving its intended outcomes and delivering value.
VAC does not currently have a benefits realization framework or the ability to track project benefits. Frequently, the benefits of IT enabled projects are only realized after a project is completed and the business or program group can realize gains in service or efficiency. The updated project management framework requires projects to complete specific documents to evaluate whether projects achieved their intended outcomes; however, the Department has not put in the capacity to measure or report on this area and project teams have not been completing these documents. This impacts the ability for the Department to measure the return on investment and value achieved from its IT enabled projects.
In conjunction with a project governance framework, IT development activities are typically guided by a complementary System Development Life Cycle (SDLC) framework which provides a standardized methodology for the progression of software development, including essential documentation, process requirements, and approvals. Currently, VAC lacks a clear and comprehensive SDLC framework. The absence of a well-defined SDLC framework creates a lack of clear outlines for the documents and oversight required for significant releases and development milestones, and obstructs the continual improvement of the agile system development process.
Although VAC has provided training to IT and Service Delivery staff on agile practices and articulated the high-level process for the Department’s use of agile methodologies, there was limited guidance or templates in place to support project teams in implementing agile processes, including functional roles, standardized documents, and application of agile to VAC governance and project management requirements. At the time of audit, the Department was in the process of developing additional playbooks and guidance to enhance agile practices.
Recommendation 1
It is recommended that the Director General, Information Technology, Information Management, Administration and Privacy Division:
- develop and implement a benefits realization framework and a System Development Life Cycle (SDLC) framework consistent with VAC agile practices and align these frameworks with the existing project management framework, and
- develop additional guidance on the application of agile in the Department and publish the guidance on the Department’s intranet.
Management agrees with this recommendation.
Many of the development activities currently employed within our agile methodology follow a structured life cycle from conception to realization. The Department will document the existing SDLC activities, while addressing any gaps, and document the standard agile SDLC to ensure consistent application throughout the organization. A benefits realization framework would apply more towards project management frameworks than our agile processes which take into account value and targeted in its execution. Therefore the Department will develop a benefits realization framework within our project management process. The Department will update the information currently available on VAC@Work with additional details pertaining to the adoption, and use of agile development methodology.
Target date: March 2025
3.3 Organizational change management and workforce digital literacy
Why it’s important
Enhanced workforce awareness, capacity and capability result in better service experiences, improved program outcomes and operations. Under the Directive on Service and Digital departments are responsible to collaborate on digitally enabled business transformation with the business owner and other stakeholders. Increases in expectations should be gradual to ensure that employees understand the changes and can contribute to their success. Fostering a common understanding across the Department of the digital goals and agile approaches for system development improves buy-in and participation from VAC staff and enhances the Department’s ability to be responsive to change.
What we found
Although the Department has made efforts to improve its capacity for organizational change, the Department did not have a mature organizational change function and the effectiveness of change management activities varied by project.
Organizational change management refers to the guided process for managing the effect of new business process and technological change to ensure that the people involved in the change are ready, equipped, have the necessary skills, and are supported throughout the process.
Beginning in 2019, as part of the PM Renewal project, change management training was delivered to 60 individuals across the Department, including project managers, communications staff, and project sponsors. Approved projects at VAC also require change management plans developed at key gates throughout the project lifecycle. In 2021 VAC also created a small group dedicated to supporting targeted development and change for individual services on a case-by-case basis, referred as an Innovation Hub.
Product owner: formal role in a scrum development team who develops a backlog of the features and capabilities which will be developed each sprint.
While key change management practices are in place for projects, as the Department increasingly adopts an iterative agile approach, significant development activity is conducted on an ongoing basis independent of the project management framework. VAC’s methodology for guiding organizational change in the agile sprint process had yet to be defined. A central concept for the success of agile development teams is designing with users and subject matter experts involved in the process. VAC has created scrum teams that include representatives from the service delivery group responsible for designing systems in coordination with users. A key role on these teams is the product owner who is responsible for communicating with business stakeholders and aligning system development to the client needs.
According to discussions with the employees involved in the development process, there were situations where subject matter experts and business users did not have enough time to properly incorporate or adapt to changes, especially when these changes affected business. Additional improvements could be made to develop a more systematic approach by scrum teams to support users in the training and deployment of new system features. This includes better definition of the role of the product owner as it relates to supporting organizational change, and creating clear expectations for communications and change management in the development cycle.
VAC did not have a comprehensive digital literacy strategy to adapt its workforce to support digital transformation and create a common departmental understanding of the Department’s digital goals and agile methodology. Digital literacy refers to the technical skills, knowledge, and abilities required by a workforce to use digital technologies and operate responsively in an environment with frequent digital innovation and change. Increased digital literacy helps the Department’s workforce operate effectively using agile practices. In 2022 VAC conducted an internal survey and review on its digital literacy and noted opportunities for improvement across the Department in strengthening awareness and understanding of digital terminology, increased engagement of employees in digital initiatives, and digital training. Based on interviews with VAC staff it was noted that conceptions of agile development varied significantly, and the benefits of change were not always recognized.
Recommendation 2
It is recommended that the Director General, Information Technology, Information Management, Administration and Privacy Division, in collaboration with the Director General, Service Delivery and Program Management:
- develop a more systematic approach for communication to users and subject matter experts as part of the agile process,
- strengthen the organization’s change management capacity and methodology,
- provide greater definition of the product owner role, and
- develop a digital literacy strategy aligned to the Department’s digital strategy.
Management agrees with this recommendation.
The Information Technology, Information Management, Administration, and Privacy Division, in collaboration with the Service Delivery and Program Management Division, will execute the following. First, we will work with the VAC Communication team to develop a communication plan for Agile processes and applicable information. We will also work to increase capacity to support and create additional automation in software change and release management. This however will be dependent on a business case to acquire the additional capacity required to perform the work. Going further, VAC will offer additional training in Agile for both its development teams and external stakeholders. This will support a more consistent application of agile development methodologies and educate external stakeholders in the practices being employed.
With respect to organizational change management, VAC will implement additional change management processes and capacities in project management to insure a more consistent change management approach in the development of software. As with the previous point, this will be dependent on a business case to obtain additional resources to perform these functions. VAC will also create additional communication materials to clarify the currently defined role of product owners to a broader audience. Finally, VAC will ensure the strategy regarding digital and data literacy is included in the departmental digital strategy as a key element.
Target date: March 2025
3.4 Tools to enable agile system development
Why it’s important
The ability to make frequent updates to IT systems enables the Department to be more responsive to changing demands, and to accelerate the pace of development. Under the GoC Digital Standards: Playbook departments are encouraged to iterate and improve frequently, and create processes to empower teams to deliver value to services as often as possible.
What we found
The Department’s IT and service delivery groups have adopted software development tools enabling development teams to make frequent changes and deploy those changes into operations at a faster pace. This process is aligned to the iterative sprint phases of VAC’s agile process. VAC relies on Shared Services Canada (SSC) to manage its IT infrastructure. As part of the SSC role to manage IT infrastructure, VAC needs to submit requests for changes of its systems to SSC, and SSC requires long lead times to implement those changes into production.
It was noted that this technical change process with SSC was inefficient to support agile development. The long lead times frequently required wasted effort and was incompatible with the sprint process employed by the VAC development team. The VAC IT development group noted that a preferred arrangement would be for SSC to delegate greater change management control to the VAC team through an integrated process which enables VAC to make changes directly. This would require development of new tools and automated processes by the VAC IT group, as well as agreement with SSC on modification of the standard operating processes currently in place. Although some preliminary discussions have taken place, VAC has not yet developed a clear plan to achieve this.
Further automation of VAC’s system development processes would be compatible with the adoption of more cloud-based solutions in the future. At present, the majority of VAC’s IT infrastructure is delivered through SSC and hosted on physical premises. As the Department continues to modernize it may opt to pursue storage and computing using cloud services, which would typically leverage automated development tools and processes discussed above.
Recommendation 3
It is recommended that the Director General, Information Technology, Information Management, Administration and Privacy Division, coordinate with Shared Services Canada and develop and implement a plan to enable additional capacity for software development automation to align with the Department’s new agile process.
Management agrees with this recommendation.
VAC will work with Shared Services Canada to enhance the ability of the department to engage in more automated release to production environments.
Target date: May 2024
3.5 Digital governance and prioritization
Why it’s important
Under the Policy on Service and Digital departments are responsible for establishing governance to ensure the integrated management of service, information, data, IT, and cyber security within their department. Integrated governance ensures that perspectives from all of the relevant functional areas are considered proactively in the development of government initiatives. This allows officials to draw connections between different functional areas and make decisions strategically in support of a more efficient, high-quality, and well thought-through suite of programs and services. It also ensures activities in each area of management are aligned with clear business outcomes.
What we found
The audit team found that VAC could strengthen its digital governance to improve alignment across governance committees, create a clear understanding of the prioritization process for IT activities, and ensure program areas were adequately involved in prioritization of development activities.
Senior management direction on corporate and departmental priorities is provided by the Senior Management Committee (SMC) chaired by the Deputy Minister. Although periodically briefed on all aspects of planning and operations across the Department, SMC sets high level priorities but is not directly involved in digital operations or prioritization. At present, SMC has not communicated detailed digital priorities across the Department.
In 2021, the Department established the Veteran Systems Priorities Committee (VSPC) to align digital decision making and priority setting between the service delivery branch and the IT/IM group. This Director General-level committee sets priorities which determine staff resources allocated from the various scrum teams to projects and system development initiatives. The committee employs a kanban board tool to document the priorities and the rolling plan.
Kanban board: agile project management tool that visualizes work in progress at various stages of a process.
VSPC is supported by the Product Owner-level Coordination and Integration Committee (CIC), which has regular engagement at the manager and director levels. While VSPC does not have the authority to allocate funding or approve project changes, it serves as the primary committee in the Department for digital priority setting. The specific mandate of the committee is still evolving and the terms of reference for the committee are being revised.
VSPC prioritization covers client-facing services and IT systems but does not include development of back-end systems that support internal operations . Many back-end and front facing systems are supported by the same development teams. Consequently, there is a need for separate governance mechanisms to address back-end system development prioritization. This separate governance has yet to be defined and the linkages across back-end and client-facing system prioritization is not formally aligned. In 2023 the Department began a project titled IT Modernization for the purpose of updating and replacing many of the back-end systems and architecture which undergirds the Departments case management and programs services. This project is highly aligned to the Department’s digital goals, however, the impact of this project on team capacity to simultaneously improve front-end systems has not been fully assessed, and the current governance structures do not provide a clear methodology to ensure this alignment.
Minimum Viable Product (MVP): agile terminology for an initial working version of an IT system which doesn't contain all the features or functionality of the final product.
Based on interviews with stakeholders across the Department, there was a common lack of understanding outside of the IT group and the Business Integration Directorate in the Service Delivery Branch on the prioritization methodology employed by VSPC in determining development priorities. There was also frequent dissatisfaction with the level of ongoing system development and feature improvement for new IT systems once they have been deployed into operations. In an agile methodology, development teams often release an initial working version of an IT system that doesn’t contain all the desired features of a final product. This initial release is referred to as a minimum viable product and is intended to be one version along the way to developing a final product through future releases. In the past VAC has deployed software systems using this philosophy into operations, such as the GCcase system. Although these initial releases are a perfectly acceptable approach to agile system development, releases such as these created dissatisfaction amongst users of the new systems who needed to employ manual workarounds for system features they expected to see. Multiple interviewees noted that once IT systems have been deployed, ongoing development of IT systems to complete the features desired from the business is often lacking, and program teams are forced to use sub-par systems for an extended period. This has resulted in employee dissatisfaction on the benefits and execution of an agile development approach.
One downside of VAC’s application of an agile methodology has been the inability to clearly communicate a product roadmap to program areas outlining the plan for when new features, functionality and improvements will be delivered. Decisions made on the prioritization of development activities, including both development of new systems and ongoing improvements of existing systems, are made at the Veterans Service Priorities Committee. Representatives from program areas participate in VSPC; however, many noted that they did not fully understand the methodology being employed in the prioritization decisions, and development of new systems frequently took higher priority at the expense of improvements to existing systems. This has created a disconnect between the expectations of program areas and the IT and service delivery group’s ability to meet business expectations. VSPC is currently in the process of redesigning its business intake and prioritization criteria.
Recommendation 4
It is recommended that the Director General, Information Technology, Information Management, Administration and Privacy Division, and the Director General of Service Delivery and Program Management:
- finalize the terms of reference and mandate of the Veterans Systems Priority Committee (VSPC),
- communicate additional details to program areas on the product roadmap for key systems,
- update the VSPC prioritization criteria and consult with program area representatives on the committee to ensure the criteria and prioritization process is clear and satisfactory.
Management agrees with this recommendation.
VAC will work to finalize the updated Terms of Reference for VSPC and will share the document with committee members. We will also develop an internal communication approach and share product roadmaps more widely within the department. Going further we will implement and finalize new criteria and share with committee members.
Target date: March 2024
3.6 Audit opinion
Based on the findings, the Department has adopted effective project management and system development practices to support digital modernization. VAC has embraced an agile methodology for delivering digital technologies and services which is highly aligned to the Government of Canada’s Digital Standards and direction for digital delivery. This way of working is relatively new to the Department, and work remains to be done to communicate the benefits and develop the digital literacy across the organization. Improvements can be made to strengthen the departmental frameworks and guidance on system development and ensure program areas are adequately involved in all areas of digital modernization.