Understanding Software Testing: Why testing is so important

Understanding Software Testing: Why is testing so important?

Thanks to the “challenges” of launching the Heathcare.gov website, there has been a lot of discussion recently revolving around the importance of testing. In this blog post, we will try to provide you with a very high level over view of the 4 major levels of testing: Unit, Integration, System and Acceptance Testing.

DISCLAIMER: Before we get started, we have a brief disclaimer for all of you testers and programmers out there….we realize that there are many different types of tests that are used regularly. We are not claiming that these are the only tests out there or that this is an exhaustive list. We are simply trying to give the less technical people in our audience an introduction to testing.

Testing should always play a major role in the development lifecycle. Investing a little extra time and money during the development cycle will pay dividends in the long run--the assumption being untested code is code that is going to fail. 

The goal of testing is to identify any defects or weaknesses and to ensure the software meets all of the user requirements before it is released. To elevate testing practices to a higher level, it is crucial to first understand the stages and various methods of testing. As we mentioned, there are 4 main stages of software testing: unit testing, integration testing, system testing, and finally user acceptance testing.

Unit Testing

This first stage is unit testing. Unit testing focuses on validating the functionality of internal structures and methods. It involves isolating and testing small portions of source code through developer written unit tests. Unit testing is often performed by developers early on during the development stage. This allows for the early discovery of defects and is a huge advantage as defects are the least costly to correct when they are caught early on.

There are several aspects of a unit test that maximize its effectiveness. One important characteristic of an effective unit test is that it should only test a small portion of code. This greatly reduces the difficulty in correcting a defect by minimizing the amount of code a developer needs to search through to find the problem. Unit tests should be kept separate from the code being tested. This allows for the application to be deployed without having to run its unit tests each time. Also, a unit test should be completely independent of other unit tests. They should be executable in any order. Finally, a unit test suite should test both the expected behavior and exception handling for each method. This will often require each method to have multiple unit tests.

Integration Testing

The second stage of software testing is integration testing. Integration testing involves combining individual units that have already passed unit testing and testing their functionality as a group. Performing this testing after successful unit testing will reveal errors relating to the interfaces between units. There are four main strategies when approaching integration testing: top-down testing, bottom-up testing, big-bang testing, and sandwich testing.

The Top-down approach requires that the highest level modules be tested first. Lower level modules are progressively tested after the high level modules. There are several advantages and disadvantages to this method. One advantage is that testing the highest level modules first allows for major flaws in design to be caught early. The need for drivers is minimized and a demonstration of an early prototype is possible. However, there are of course some disadvantages. This method will rely heavily on stubs which can complicate the testing process and introduce errors. Stubs are dummy modules that simulate low level modules. This is needed in top-down integration since we are testing the high level modules separately before the low level modules.

The Bottom-up approach integrates and tests the lowest-level units first. High level modules are tested later in the process. An advantage here is that the need for stubs is minimized. It will instead rely heavily on drivers. Drivers are dummy modules that simulate high level modules by calling the low level methods we want to test. Since the high level modules are tested last in bottom-up integration, drivers are needed to act as their temporary replacements so we can test the low level methods first. A disadvantage with this method is that the high level logic will be tested late. Testing the high level logic late prevents the possibility of releasing an early prototype with limited functionality.

In Big Bang Integration Testing, all of the modules are integrated immediately and the system is tested as a whole. An advantage to this is that stubs and drivers are rarely required. This method is generally only recommended for small systems. Otherwise its disadvantages often outweigh its advantages. Big bang integration requires that all of the modules are ready before any testing can be done. Debugging and fault localization can become very difficult. It is usually much easier with top-down or bottom-up integration.

Sandwich integration combines top-down and bottom-up testing. The system is divided into 3 layers: top, target, and bottom. The target layer is identified first. It is usually somewhere in the middle. The components above the target layer are tested by the top-down strategy. The components below the target layer are tested by the bottom-up strategy. Testing converges at the target layer. A big advantage here is that both top-down and bottom-up testing can occur simultaneously. The need for drivers and stubs is reduced but both will still be required.

System Testing
Once integrated, the next step is system testing. System testing is the testing of a fully integrated system to verify its functionality and ensure that the user requirements have been met. System testing is a form of black-box testing. Black-box testing is any testing that does not require internal knowledge of code. Instead it verifies the functionality of the system by its specified requirements. There are many different types of testing that should be included in system testing. Some critical tests includes: load, security and regression testing.

The term load testing can be used interchangeably with performance and stress testing. The main objective in this type of system testing is to confirm that the system can operate efficiently under both normal and stressed load conditions. The system is purposely put under high stress to determine the maximum capacity the system can handle and identify any bottlenecks in the system.

Security testing is performed to ensure that the system is able to protect against any attempts of unauthorized access. Proper security testing should verify the following aspects are secure: authentication, authorization, confidentiality, integrity and non-repudiation. Penetration tests are often utilized in this testing. This test simulates a potential attack a hacker might execute and evaluates the system’s response to the attack.

Regression testing involves retesting the system after a change has been made to the software. This is to ensure that the change has not introduced any new defects to other parts of the system. It generally does not require new test cases to be written. It is often performed by rerunning previous test cases from earlier testing. In particular, it can often involve rerunning unit tests.

User Acceptance Testing

The final stage is user acceptance testing. User acceptance testing tests the system for acceptability. Its purpose is to verify that the system meets the business requirements by testing the system in its “real world” environment. This is done by involving end-users and real data in the testing process.

The testing is done by the system’s intended users or business representatives. There are often misunderstandings or miscommunications between developers and clients during the requirements gathering process. Inviting clients into the testing process provides an excellent opportunity to identify and resolve any such misunderstandings. The data used is often real data supplied by the client rather than the developer written test data. This is an important factor as it can reveal instances in which the system fails because the real data exercises the system differently than the test data.

Once user acceptance testing is complete, the system is ready to be deployed. If all of these testing practices mentioned have been enforced diligently, the final product will have undergone necessary and extensive improvements that will certainly satisfy the client.

Now that we have explained the four basic stages of testing, it must be noted that all of the testing in the world cannot fix a bad design. Proper care needs to be taken up front to create a solid project plan that meets the needs and business requirements of the client. Testing should always be a part of that design.

Remember: THINK…DESIGN…BUILD

By: Sam Gonell, CEO & Co-Founder eTag Technologies and Cynthia Murillo, Application Developer & Tester eTag Technologies