Introduction
We are accustomed to running projects at Globant using Agile methodologies. Most projects start with a few teams but then quickly grow in size and before we know it, there are several PODs (Agile teams) building one or more projects for a customer. In the best scenario, we have a few PODs per project, but usually, we are building complex solutions for large companies with multiple teams that have to synchronize their work to release a high-quality, integrated product.
In my experience, scaling from Scrum or other similar frameworks is challenging but not impossible. Big projects for larger companies often present the greatest challenge, regardless of industry, due to their mature culture, strong structures, heavy processes, rules, and complex infrastructures. In these environments, it is often assumed that Agile initiatives will be addressed solely by the development team. However, engagement and adoption from groups outside of the dev and IT teams, like business and upper management, are crucial to support cultural change and embrace a new way of collaborative work.
Within this context, you can imagine how difficult it can be to simultaneously scale while continuing to develop and deliver software and maintain high levels of quality. Defining both a test strategy and testing processes in Agile at scale represents new challenges for quality and test managers, test architects, and other test specialists, together with the entire organization.
Two years ago I joined an ongoing project, a huge program for a large company, that was implementing SAFe (Scaled Agile Framework for Lean Enterprises). They were trying to adjust the testing processes and even though I had experience with Agile, I had no experience in implementations with SAFe.
The first time I saw the SAFe 4.5 diagram, it was great to see a huge structure to drive the organization to adopt and scale Agile, but I was also surprised that there were no explicit references to testing. I’m glad to see that SAFe 5.0 has more detailed explanations and more guidance on testing related topics.
This article is based on my experience managing testing and quality in large projects when scaling Agile, and my thoughts on how quality can be embedded at every level. Doing this enables the scaling of multiple self-organizing, cross-functional teams to build quality and collaboration on a single release.
Scaling in Agile
Organizations that are adopting Agile methodologies plan to have iterative and incremental delivery and a short time-to-market. There are popular Agile frameworks like Scrum, Kanban, and XP, as well as hybrid models that combine Agile methodologies with the more traditional, iterative methodologies that aim to deliver customer value in a continuous flow.
As previously mentioned, one of the common challenges when implementing Agile is getting organizations to understand that Agile is not just for development teams. Business agility and responsiveness are key factors to real adoption, especially for larger organizations.
For Agile at scale, most of the larger programs and structured organizations embrace SAFe (Scaled Agile Framework for Lean Enterprises) because they find an organic structure, along with clear steps to transition from a traditional development process to a business Agile process.
When I started working with SAFe Portfolio configuration, I noticed how important it is to have a clear team organization and structure. I like the structure defined in levels (team level, program level, portfolio level) to distribute the effort and responsibilities from the individual development teams to business and product management.
Another interesting and helpful aspect in SAFe is how the development planning is organized in Program Increments (PIs). Each PI is a time-boxed interval that aims for planning, building, and validating a system increment. The PI is typically 8 – 12 weeks long, and the most common pattern is five development Sprints, followed by one Innovation and Planning (IP) Sprint.
The process starts with PI planning where the team is guided by the product objectives to plan the work for the 5 sprints, as well as be able to identify dependencies and risks in advance to properly address them within the Product Increment. Each PI execution is then a sequence of Agile ceremonies that most of the development teams are familiar with, like Sprint Planning, Backlog Refinement, Daily Stand Up, Sprint Review, and System Demo. These activities keep the teams on track to deliver each system increment aligned to the objectives set and the business expectations.
Quality culture in Agile
Based on my experience, most teams still view testing as the final phase of work. Changing this mindset and building a culture of quality within teams is a major challenge. Quality is a responsibility of the whole team, every developer, architect, and designer, can contribute to the quality of the work, building the code following best practices for code development.
Putting a fire out is more difficult and risky than preventing the fire, and in the same way, preventing issues by building quality into the product development from the beginning is much more efficient and effective than fixing them later. Build-in quality is therefore a key objective for the team. A well-defined architecture and design determine the system’s testability and how well a system can handle current and future requirements, and the ease and speed of adding new capabilities depends on the reliability and simplicity of the code developed.
My advice is to go beyond simply assigning testers to the Agile teams and consider testers as quality ambassadors through each step of the process, opening discussions from a quality lens from the requirements definitions through its development and release. The main role of testers and any other quality engineers in the program should be helping the team change their mindset and ensure that quality is built-in to the process from beginning to end.
Defining the quality strategy
In a scaled Agile implementation, it is very important to define an effective quality strategy that includes what is needed in order to have an integrated and tested solution delivered to customers. Even though you can rely on each Agile team to apply effective Agile testing practices, in order to scale, you need predictability and a comprehensive quality strategy defined for the whole organization.
Once you understand business and product needs, you have to start defining the quality strategy. It may sound basic, but this should always be the first step for any type of project and framework. A clearly defined quality strategy will ensure there is a strategic definition of the different testing types throughout every level. This strategic definition will help deliver a product with a clear understanding of what is critical, not only from a technology point of view, but also from business and operational perspectives.
In larger programs and organizations, it is very easy to fall into a waterfall cycle for product testing. This is why it is so imperative to shift-left testing as early as possible in the Software Development Life Cycle (SDLC). Instead of thinking in phases where different levels of testing are executed sequentially, testing for both functional and non-functional requirements has to be executed in parallel and incrementally at various layers within a continuous integration (CI) and continuous delivery (CD) approach.
We also do not want to forget that in a scaled environment, teams are working on a shared code base. This implies that every autonomous team must ensure that the changes they introduce don’t break the existing functionalities or negatively impact the customer journey. This may trigger regular and automated regression and integration testing.
In my experience, it was very useful combining various approaches to help select the appropriate testing types for each product being tested while simultaneously defining a comprehensive testing strategy that helps to ensure quality aligned to the business needs.
Considering both the complex application we were delivering and the client’s existing process in my most recent project, we started defining an onion layer structure with the traditional levels of testing (unit/component testing, integration testing, system testing, and acceptance testing). We also included in the analysis of the types of testing required, the Agile testing quadrants, that provide a helpful taxonomy to identify and plan the testing needed. Finally, based on the test automation pyramid, we completed the test automation strategy.
Automating your tests is key to changing the mindset to a continuous testing approach that evolves and extends test automation to address the increased complexity of the application. Continuous testing is defined as a software testing type that involves a process of testing early, testing often, testing everywhere, and automating. This means that you have to define a process to execute automated tests as part of the software delivery pipeline in order to obtain feedback on the business risks associated with a software release candidate as early and often as possible.
A good starting point is to incorporate the main flows in the Build Verification Test suite (BVT) that is automatically executed with every commit to the main branch and becomes the first quality gate. The remaining automated tests would be part of the Automated Regression Suite that is executed with a regular cadence (for example, nightly), to provide early and continuous feedback of the new changes implemented in the code.
The diagram below shows a high level approach which can help you drive the testing effort needed at different levels:
Continuous improvement process
Even though an Agile framework is selected to provide both guidance on good practices and an implementation roadmap, each organization has to define their own processes, practices, and guidelines aligned to their needs.
There are no magic recipes to define the right ones for your project. To get maturity on processes, you must go incrementally with baby steps, define, try, and learn. Every situation is a good opportunity to learn and identify what to keep and what to improve, as well as what to avoid and change.
Nothing is written in stone and all processes are alive, so do not hesitate to experiment with different practices until you find what works for the product, the organization, and the team. Spikes, proof of concepts, and pilots are alternatives that you can consider implementing. Every team member can contribute great ideas and be part of the solution. You can also create “Communities of Practice” (CoP) that can help provide focus on specific areas for a continuous improvement process.
Retrospectives, lessons learned, and inspect & adapt events, are very helpful to have a regular, scheduled time to reflect, apply problem-solving techniques, and take on improvement actions needed to increase the velocity, quality, and reliability of the next Program Increment.
In part 2 of this blog, I will go through testing at the different levels and how quality can be embedded at every organizational level while scaling.
