In today’s digital world, even though most of the projects are following the Agile methodology, often testers might not get enough time to quantify the problem scope and test the product effectively. Even if a sprint lasts for two weeks, the QA team would get the complete functionality for testing, only two or three days before the sprint completion. Eventually, the QA team would have to rush the testing, struggle for test completion and even end up with improper test coverage and bugs being leaked into production. So the testing phase is often considered as a bottleneck for the release by the management.
Studies done by analysts suggest that the maximum number of defects occur during the requirement and design phase of the software development life cycle. More than half of the defects occur during the requirement and design phase of the SDLC, i.e 56% of the total defects. Out of this 56%, 23% occurs during the design phase, 7% in the development phase and 10% defects emerge during the other phases. 2019 witnessed test automation going mainstream with 44% of IT organizations automating more than 50% of all testing and these figures are expected to go up in the upcoming years. Thus it becomes highly necessary to step up the testing game and ensure that it is done quite efficiently and this is where Shift Left Testing comes into play. Detecting defects early in the software development cycle can prove to be very crucial in regards to cost and efficiency.
This whitepaper discusses how shift left testing could help you reimagine the entire QA testing process.
Generally, during the conventional software development process, QA teams encounter several problems. Since testing is not performed at every stage, the bugs are revealed only once the product development phase is completed. If critical bugs are revealed in the end, more time and effort is required to fix them and this causes a potential delay in the release of the product to the customers. Also, testers are not involved in the most important stages of the product life cycle and the focus is more on ‘testing the software to detect the bugs’. There is also very little collaboration between the developers and testers.
Thus to solve these problems and to improve the testing efficiency, Shift Left Testing was introduced as a solution. Shift left testing is a process where testing is performed earlier in the development process rather than as a trailing activity before release. Shifting the ‘Testing Phase’ to the left from right facilitates speed, quality, and cost-effectiveness. Once the team becomes confident with the roles and responsibilities, the model could be gradually implemented and the concept of Shift Left Testing could be followed in our regular projects. Thus the team would be able to conquer the limitations of the conventional method. As testing is performed from the earlier stages itself and regularly, it becomes easier to detect the bugs at every stage in the development process. The early defect detection accelerates the process of taking corrective actions, thereby reducing the cost and time for fixing the issues. By testing often, the team and the client always remain aware of the current state of the code and are able to make planned decisions throughout the project. Right from the project initiation, the test team is engaged in the system and they also play a significant role in meeting the business goals of the organization. The graph given below depicts the difference between the Shift Left Model and the Typical Quality Model.
Shift Left Testing can be performed in four different manners and they are as follows:
1) Shift Left Traditional Testing: In the traditional V-Model approach, it could be noticed that the majority of the focus was on the System and Acceptance level testing. But with this shift-left testing approach, the focus and emphasis will shift more towards the lower level tests like unit and integration level testing. For this, the testing team can make use of tools and testing types like API, Database testing and the like that doesn’t rely completely on the GUI.
2) Shift Left Incremental Testing: To mitigate the failure risk of V-Model in large complex projects, the trend is to breakdown the master V-Model into different increments of Shift-Left V-Model. With this approach, we can ensure that testing is not being pushed towards the last of the master V-Model, but is included as a critical element of each incremental release. Incremental shift-left testing makes testing easier since it is performed on smaller tasks made by breaking down the project tasks.
3) Shift Left in Agile/DevOps: In this model, the time period for the minimum viable product will be reduced as compared to earlier models, i.e. this type of shift left testing is performed in multiple sprints. The benefits of shift left can be reaped in agile-based models as well. Agile/DevOps shift left testing is mainly used for developmental testing and not for operational testing. Organizations are using this method based on project requirements.
4) Model-Based Shift Left: The concept of shift left testing is to identify bugs at the earliest. But the main problem is that the majority of the defects are introduced during the requirements gathering and design phase. In this model, testing is started as early as possible by evaluating the requirements, architecture, and reviews so that the bugs can be reported before the software development cycle begins.
Just as every aspect of the software development cycle requires, shift left testing too should be performed systematically and meticulously. It requires careful planning and consistent follow-ups and the process is as follows:
1) Planning the Test Lifecycle
The organizations must define the testing lifecycle in such a way that the most important testing activities take place at the initial stages of the software lifecycle. Also, they are supposed to possess good knowledge about the whole lifecycle before starting the development.
2) Collaborate Development and Testing Activities
Let’s assume that we need to understand the scope of testing in various stages, so it is advisable to integrate the development and operation processes with testing activities. It not only helps to identify the required time and effort but also helps prevent unwanted duplication.
3) Regular Feedback Mechanism for Early Bug Detection
Generally, testing takes place after completing the development process. In such cases, the product is more prone to errors. In shift left testing, we follow a regular feedback mechanism where the testers intimate their feedback to the developers promptly which helps to resolve the defects as early as possible. This would also help management to make calculated decisions on the project.
4) Coding Exposure to Testers
Let’s assume that the testers have good coding knowledge. In such a case, it would be easy for them to understand more about the defects and they would be capable of resolving trivial errors without seeking the help of developers.
5) Reducing Risks by Static Testing
Early error detection can be achieved by doing static testing like inspection, reviews and this reduces the risk of major defects during the later phases. It helps to ensure that the product is defect-free and stable enough to move to the next stages.
Shift Left Testing has been an aspect of QA testing that has been gaining steam recently and the reason for the same is its countless benefits. Let us list down a few advantages that we gain from adopting this model.
1) Enhanced Product Quality
Let us consider that there’s constant communication between different teams in a project. Then, the developers and testers will work together and the testers will provide feedback to the developers which will lead to quick bug fixes. Also, automation being a major part of shift left testing will help us in performing testing a few times a day which will gradually improve the code and product quality.
2) Declined Cost in Development, Testing and Bug Fixing
Suppose a bug is discovered at the initial stages of development, then the cost of fixing it will be very less compared to the bug found in later stages. It is evident that the cost of bug fixing substantially increases with the time it is discovered. In shift left testing, since testing takes place at all the stages, bugs are found and resolved earlier and easily; thus the cost of development, testing and bug fixing is declined. The graph given below depicts the relative cost of fixing defects.
3) Decreased Software and Business Risk
By analyzing processes it is easy to identify areas where software flaws could end in serious issues in production. Using early testing we can achieve business growth through continuous quality improvement and can also reduce business risk caused due to outages. We can explain this through an example. Suppose a defect is introduced in requirement or any early phase in SDLC and the tester finds that out in a later stage of SDLC then the rework cost is high and it will ultimately lead to business risks.
4) Improved Test Coverage and Better Software Quality
Let’s assume that the testing gets started from the beginning of the development process. Then, the application will be evaluated for all the features from the very beginning itself. In addition, the testing team would have a better understanding of the business and functional requirements due to improved collaboration. Each of the functionality and performance will be tested. Thus improved test coverage helps in achieving better software quality.
5) Efficient Resolution of Bugs through Developer-Tester Collaboration
In the shift left method, the developers and testers work together. So, testing is initiated from the very beginning of the development cycle and along with it bug fixes and verification are also done.
6) Increased Developer and QA Staff Productivity
Defects found in the later stage of SDLC lead to rework and thus it will hinder the productivity of the developers and the QA staff. So, it is evident that by early testing we can minimize the number of defects in an application and through this, we can increase the developer and QA staff productivity. The following graph compares the test effort for early testing and shift left testing.
7) Improved Application Security and Overall Code Quality
Code quality could be improved considerably if an application has minimal bugs. Early testing plays a very important role in decreasing the number of defects. Also, testers will get enough time for the requirement analysis, documentation preparation, test case preparation, etc. and thus it becomes easy to understand the scope and analyze it.
How is it possible to perform testing even when the application is not available? There are two types of testing techniques that can be used for this.
1. Static Testing
2. Dynamic Testing
Static testing is also called ‘Non-Executable Testing’ and Dynamic Testing as ‘Executable Testing’ and in this whitepaper, we are only discussing the former. The types that come under static testing are:
Inspection is a formal type of review and is typically led by a trained moderator who isn’t the author. It is a checklist based review and it makes use of the entry and exit criteria. The ultimate aim of inspection isn’t only to spot defects but also to provide guidance for process improvement and also help improve the quality of the document under review.
The review process is a systematic examination of a document by one or more people with the aim of finding and removing errors in the early stages. The review process can improve the development team productivity and can either be formal or informal. Early detection of defects is possible through this method.
The Walkthrough is not a formal process. It is the best way to teach what the application is about and all the functionality flows in the application. It is conducted by the authors / Subject matter expert.
Test activities in the early cycle can be explained through the below given diagram.
As shown in the Figure above, Guided Inspection and Architecture Trade off Analysis Method (ATAM) are two test activities that are part of the early cycle.
Guided Inspection: In the early stages of the software development cycle, one doesn’t have the application or the code to test but we do have different models such as requirement models, analysis models, architecture models and the like. Guided Inspection is a type of inspection technique in which the reviewer will examine the model using the guidance of selected scenarios to check whether certain defects exist. By tracing the contents of the model, the reviewer will define how the model would handle the scenario.
ATAM: ATAM is a risk identification method that has several implications. ATAM can be done during the early phase of the software development cycle. It is less time consuming and cost-effective as it assesses only the architectural design artifacts. Using ATAM we cannot predict the accurate quality attribute behavior as predicting the accurate quality is impossible in the early stages of SDLC. Instead of checking the accurate quality, we can perform risk identification activity using ATAM, i.e., learning where an attribute of interest is affected by the architectural design decision. Also, using ATAM we can improve the level of architectural documentation and through this method, we can analyze the risks, sensitive points, and the trade-off points.
The cost of fixing bugs increases 10 to 100 times in the stages after product development and hence early testing becomes mandatory to get rid of this. We can explain this through an example. Suppose a tester is going through a requirement document and there exists a requirement as given below.
Requirement: Validation error should be displayed when a user enters an invalid username or password.
Error message to be displayed: “Registration/Signup error”
But the above-mentioned error message is not correct when the user enters an invalid username and password. The actual error message to be displayed for invalid login is.
“Invalid username or password”
So there comes an error in requirement. The tester can add that requirement immediately to the document so that there arises no issues in the development phase. Otherwise, if early testing had not been done, then the developer would not have done the coding for this requirement. So the tester would have reported that defect in the testing/later phase. This leads to rework by the development team and thus the cost to fix this issue increases.
From the figure, it is clear that testing is not an activity that should start only in the testing phase. In this diagram, testing is present during Planning, Analysis & Design and Code & Build. Test planning should start from the early stage of the project and once after all the requirements are base lined, the test plan and test cases should be prepared. This will prove to be helpful in uncovering the gaps in requirements. Employing shift left testing helped us transform a leading financial services company. The client followed manual testing processes and this resulted in longer test execution time which in turn delayed the release to the market.Implementing early bug detection and fixing them in the early stages enabled developers to spend more time on feature development rather than debugging and fixing them all the time. The developers were able to identify and fix the bugs in the early stages of the project using automated unit tests and static analysis.Also, the development team was able to identify the integration problems and the code merging conflicts early in the SDLC by implementing CI. This eventually simplified integrations and enabled rapid software releases. The fact that the bugs were detected and fixed early increased the build quality and helped the client to save both time and money.
Keeping in mind that the shift left model is a different type of strategy, testers need to make some changes in their already existing workflow. Here are some points that they need to keep in mind while adopting the shift left test model.
The Shift Left Testing approach could be a lifesaver for testing teams, who have been struggling with deadlines towards sprint closure and companies who suffer from huge defect fixing cost in production and poor software quality. By using the shift left method, we can obtain software that is stable and functionally reliable. The shift left method helps to find the bugs earlier thereby reducing the cost by fixing it in the initial stages itself. It also provides effective means to perform testing in parallel to the development processes. This increases the quality of the applications and allows enterprises to be competitive and productive. It is worth a shot to try implementing the Shift Left model in our regular projects and once the benefits are noticed by management, it would be automatically incorporated into other projects as well.