What is smoke test?

The Smoke Test is a set of non-exhaustive tests that verify end-to-end that the main functionalities of the product are working correctly. It runs programmatically to detect potentially serious bugs or "fires". If “smoke signals” are detected, the QA (Quality Assurance) must investigate further the functionality that generated it. It is from these analogies of “fires” and “smoke signals” that this methodology was called the Smoke Test.

How to validate smoke test?

The customer must be strongly supported in defining and validating the Smoke Test, as they are the one who defines the main test cases that cannot fail in any way. It is necessary to test the main flow of the system, and the customer must participate in its definition.

Why do a smoke test?

In the initial phase of a project, many build cycles (is the term used to identify a compiled version of a program) are released. As the project and team are new, changes released to the system may break some important functionality of the product. Any failure indicates the need to create a task for development time. Whenever there is a change in the system, we will carry out a smoke test to ensure the stability of the system.

How to do the smoke test

Smoke testing is usually done manually, as testing approaches vary from one company to the next. It is performed to ensure that navigation through critical paths is as expected and does not disrupt system functionality. As soon as a new build is released to the test environment, test cases for the high priority features should be run to see if they find any defects. If the tests pass, we move on to functional testing. If a test fails, a task must be created with evidence of the bug so that the development team can correct it. When a new version is generated, QA starts the smoke test again.

Some advantages of the smoke test

• Easy tests to perform
• Defects will be identified in the early stages
• Improves system quality
• Saves effort and testing time
• Easy to detect critical errors and correct errors
• It is a test that runs quickly on the system
• Minimizes integration risks

Smoke test document example

The smoke test document is composed of two parts: the first is the summary, where it is possible to observe the global status of the test execution and the percentage of each status. Possible statuses are:

• Passed: when the test scenario is done step by step and it works correctly.
• Failed: when the test scenario is done step by step and a bug occurs.
• In progress: test scenarios being executed by the tester.
• Not applicable: when there is a test scenario that cannot be tested due to lack of data in the test environment.
• Blocked: when the test scenario is blocked by some system bug that prevents testing this scenario.

The second part contains the listing of test scenarios, which contains the following information:

• Number: Identifier of the test scenario
• System: It is the system being tested. Many projects have different systems, for example: Admin, Mobile App, Web App.
• Module: Which system module that is part of the test scenario that will be tested, for example: authentication.
• Test scenario: This is where a summary of what will be tested is located, for example: entering a valid email and an incorrect password, password recovery.
• Status: It's where you fill in if the test passed or not and from this column here the % of the summary is calculated.
• Comments/Bug id: It's where you fill in comments about that test scenario and if it doesn't pass the test, here's the bug tag that was created.
• Priority: In this column is placed the severity of the bug which in this documentation is numbered from grade 0 to 4, I will briefly describe the criteria I use to number the severity of a bug.
• Grade 0: bugs are numbered with this grade when it blocks a main flow of the system / bugs reported by the customer that are occurring in the production environment (grade 0 is the highest priority to be solved).
• Grade 1: bugs are numbered with this grade when the bugs affect the functionality of the main stream or block alternative streams.
• Grade 2: bugs with this degree are numbered when bugs that severely affect the usability of the system, but do not interrupt any flow.
• Grade 3: bugs are numbered with this grade when the bugs do not affect usability or block any system flows.
• Grade 4: bugs are numbered with this grade when they are layout bugs (grade 4 is the lowest priority bug).

The second part of this smoke test document we call the main sheet. In it, we describe the step-by-step that needs to be done to test each test scenario. The step-by-step should be very detailed, so that even a person who doesn't know the system would be able to use it following the description. Finally, we describe the result expected by the system, and, if a bug is detected during the test. describe what is happening in the current result and the status (passed, failed or blocked) as well as the id of the bug found doing the test case.

Conclusion

Smoke testing helps determine the stability of the deployed build. These tests should be quick to run and their purpose is to provide assurance that key system functionality is working as expected. Thus, this form of testing verifies that the QA team can proceed with further testing of the software product. That is, the smoke test is a minimal set of tests run on each build.

References

https://cynoteck.com/pt/blog-post/what-is-smoke-testing/

https://www.linkedin.com/pulse/smoke-test-robson-agapito-correa/?originalSubdomain=pt