Exact Process And Plan To Execute Test Cases With Real Examples.
Today, in our Software Testing mini training course, we are progressing into the last stage of the STLC, which is the Test Execution.
You can check out the list of all tutorials posted in this free QA training series on this page: End to End software testing training on a live project.
Test Execution is, without doubt, the most important and ‘happening’ phase in the STLC and also the entire development lifecycle. The reason is – every team/team member’s contribution and work gets validated here:
Has the Business Analyst interpreted the requirements correctly?
- Has the development team translated the business requirements to functional requirements and eventually to code correctly?
- Has the data architect and DBAs designed the right back-end systems?
Well, test execution is where all the answers to these questions would be found. That does make us, QAs the heroes of the entire software building process, doesn’t it? 🙂
Test Execution is also the “Test” part of the SDLC.
Once the test cases are written, shared with the BAs and Dev team, reviewed by them, changes are notified to the QA team (if any), the QA team makes necessary amends- Test design phase is complete. Now getting the Test cases ready does not mean we can initiate the test run. We need to have the application ready as well among other things.
Table of Contents: [Show]
Test Execution Guidelines
Let us now make a list of all things that are important to understanding the Test Execution phase:
#1) The build (the code that is written by the dev team is packaged into what is referred to a build- this is nothing but an installable piece of software (AUT), ready to be deployed to QA environment.) being deployed (in other words, installed and made available) to the QA environment is one of the most important aspects that needs to happen for the Test Execution to start.
#2) Test Execution happens in the QA environment. To make sure that the dev team’s work on the code is not in the same place, where the QA team is testing, the general practice is to have a dedicated Dev and QA environment. (There is also a production environment to host the live application).
This is basically to preserve the integrity of the application at various stages in the SDLC lifecycle. Otherwise, ideally, all 3 environments are identical in nature.
#3) Test team size is not constant from the beginning of the project. When the Test Plan is initiated the team might just have a Team lead. During the test design phase, a few testers come on board. Test Execution is the phase when the team is at its maximum size.
#4) Test Execution also happens in at least 2 cycles (3 in some projects). Typically in each cycle, all the test cases (the entire test suite) will be executed. The objective of the first cycle is to identify any blocking, critical defects, and most of the high defects.
The objective of the second cycle is to identify remaining high and medium defects, correct gaps in the scripts and obtain results.
#5) Test Execution phase consists of- Executing the Test scripts + Test script maintenance (correct gaps in the scripts) + Reporting (defects, status, metrics, etc.) Therefore, when planning this phase schedules and efforts should be estimated taking into consideration all these aspects and not just the script execution.
#6) After the Test script being done and the AUT is deployed – and before the Test execution begins, there is an intermediary step. This is called the “Test Readiness Review (TRR)”. This is a sort of transitional step that will end the test designing phase and ease us into the test execution.
For information on this step and a sample “Test Readiness Review checklist”, check out this link: Software testing Checklist
#7) In addition to the TRR, there are few more additional checks before we ensure that we can go ahead with accepting the current build that is deployed in the QA environment for test execution.
Those are the Smoke and Sanity tests. Detailed information on what these are is at: What is Smoke and Sanity Test?
#8) Upon successful completion of TRR, Smoke and Sanity tests, the test cycle officially begins.
#9) Exploratory Testing would be carried out once the build is ready for testing. The purpose of this test is to make sure critical defects are removed before the next levels of testing can start. This exploratory testing is carried out in the application without any test scripts and documentation. It also helps in getting familiar with the AUT.
#10) Just like the other phases of the STLC, the work is divided among team members in the Test Execution phase also. The division might be based on module wise or test case count wise or anything else that might make sense.
#11) The primary outcome of the test execution phase is in the form of reports primarily i.e. Defect Report and Test Execution Status report. The detailed process for reporting can be found at Test Executions reports.
New Columns In Test Cases Document
The Test Case document now gets to be expanded with the following two columns – Status and Actual result.
(Note: For live project Test Execution, we have added and updated these columns with test execution results in the test cases spreadsheet provided for download below)
#1) Status Column
Test Execution is nothing but, using the test steps on the AUT, supplying the test data(as identified in the test case document) and observing the behavior of the AUT to see if it satisfies the expected result or not.
If the expected result is not met, it can be construed as a defect. And the status of the test case becomes “Fail” and if the expected result is met, the status is “Pass”. If the test case cannot be executed because of any reasons (an existing defect or environment not supporting) the status would be “Blocked”.
The status of a test case that is yet to be run can be set to No run/unexecuted or can be left empty.
- For a test case with multiple steps, if a certain step’s (in the middle of the test case steps) expected result is not met, the test case status can be set to “Fail” right there and the next steps need not be executed.
- The status “Fail” can be indicated in red color, if you would like to draw attention to it immediately.
#2) Actual Result Column
Here we analyzers can keep what the deviation in the normal outcome is. At the point when the normal outcome is met (or an experiment whose status is “Pass”) this field can be left vacant. Since, in the event that the normal outcome is met it implies the genuine result=expected result, and that implies changing it in the genuine outcome section will be a reiteration and overt repetitiveness.
A screenshot of the deviation can be attached to this column for enhanced clarity of what the problem is.
Test Execution Results For OrangeHRM Live Project
Let us now get OrangeHRM and carry out the test execution based on the above guidelines listed.
Here are a few points to note:
- The lengthy experiment layout.
- Exploratory testing as shown is to be completed without test scripts. So kindly go ahead and test the application in lined up as you see fit.
- Because of the restrictions that we have in introducing the live undertaking as lucid substance just a restricted measure of experiments/usefulness of the OrangeHRM application is displayed in the example Test Execution layout. Once more, kindly feel to chip away at something else for the most viable experience.
- The Mental stability and Smoke test suites are additionally added to the report, to give you a thought regarding what sort of experiments are considered for these stages.
- Abandons are not logged at this point, despite the fact that the situation with some experiments is set to “Fall flat”. This is on the grounds that logging the deformities is the following generally significant/ordinarily dealt with a part of our life as analyzers. In this way, we need to manage it exhaustively in the following article.
Test Cases with Execution Results:
It Contains – Test cases execution result, Smoke tests, Sanity tests, Exploratory test – spreadsheets
Lastly, if a Test Management tool was used for creating and maintaining the test case, the same can be used for test execution as well. The use of a tool makes reporting easier, but otherwise, the process of running the test cases is the same. Please check out this article to get an idea of how to use HP ALM for Test Case Execution.
(Click on the image for an enlarged view)
This brings us to the end of another interesting segment of the testing process. In the next and last article of this free online Software Testing QA training mini-course, we will look into defects in detail; wrap up topics like “when to stop testing”, metrics and QA sign off.
YOU MAY BE INTERESTED IN
The Art of Software Testing: Beyond the Basics
Automation testing course in Pune