Testing Thread Applications
In this chapter, we will learn about testing of thread applications. We will also learn the importance of testing.
Why to Test?
Before we dive into the discussion about the importance of testing, we need to know what is testing. In general terms, testing is a technique of finding out how well something is working. On the other hand, specifically if we talk about computer programs or software then testing is the technique of accessing the functionality of a software program.
In this section, we will discuss the importance of software testing. In software development, there must be double-checking before the releasing of software to the client. That is why it is very important to test the software by experienced testing team. Consider the following points to understand the importance of software testing −
Improvement of software quality
Certainly, no company wants to deliver low quality software and no client wants to buy low quality software. Testing improves the quality of software by finding and fixing the bugs in that.
Satisfaction of customers
The most important part of any business is the satisfaction of their customers. By providing bug free and good quality software, the companies can achieve customer satisfaction.
Lessen the impact of new features
Suppose we have made a software system of 10000 lines and we need to add a new feature then the development team would have the concern about the impact of this new feature on whole software. Here, also, testing plays a vital role because if the testing team has made a good suite of tests then it can save us from any potential catastrophic breaks.
User experience
Another most important part of any business is the experience of the users of that product. Only testing can assure that the end user finds it simple and easy to use the product.
Cutting down the expenses
Testing can cut down the total cost of software by finding and fixing the bugs in testing phase of its development rather than fixing it after delivery. If there is a major bug after the delivery of the software then it would increase its tangible cost say in terms of expenses and intangible cost say in terms of customer dissatisfaction, company’s negative reputation etc.
What to Test?
It is always recommended to have appropriate knowledge of what is to be tested. In this section, we will first understand be the prime motive of tester while testing any software. Code coverage, i.e., how many lines of code our test suite hits, while testing, should be avoided. It is because, while testing, focusing only on the number of lines of codes adds no real value to our system. There may remain some bugs, which reflect later at a later stage even after deployment.
Consider the following important points related to what to test −
-
We need to focus on testing the functionality of the code rather than the code coverage.
-
We need to test the most important parts of the code first and then move towards the less important parts of the code. It will definitely save time.
-
The tester must have multitude different tests that can push the software up to its limits.
Approaches for testing concurrent software programs
Due to the capability of utilizing the true capability of multi-core architecture, concurrent software systems are replacing sequential systems. In recent times, concurrent system programs are being used in everything from mobile phones to washing machines, from cars to airplanes, etc. We need to be more careful about testing the concurrent software programs because if we have added multiple threads to single thread application having already a bug, then we would end up with multiple bugs.
Testing techniques for concurrent software programs are extensively focusing on selecting interleaving that expose potentially harmful patterns like race conditions, deadlocks and violation of atomicity. Following are two approaches for testing concurrent software programs −
Systematic exploration
This approach aims to explore the space of the interleavings as broadly as possible. Such approaches can adopt a brute-force technique and others adopt partial order reduction technique or heuristic technique to explore the space of interleavings.
Property-driven
Property-driven approaches rely on the observation that concurrency faults are more likely to occur under interleavings that expose specific properties such as suspicious memory access pattern. Different property-driven approaches target different faults like race conditions, deadlocks and violation of atomicity, which further depends on one or other specific properties.
Testing Strategies
Test Strategy is also known as test approach. The strategy defines how testing would be carried out. Test approach has two techniques −
Proactive
An approach in which the test design process is initiated as early as possible in order to find and fix the defects before the build is created.
Reactive
An approach in which the testing does not start until the completion of the development process.
Before applying any test strategy or approach on python program, we must have a basic idea about the kind of errors a software program may have. The errors are as follows −
Syntactical errors
During program development, there can be many small errors. The errors are mostly due to typing mistakes. For example, missing colon or a wrong spelling of a keyword, etc. Such errors are due to the mistake in program syntax and not in logic. Hence, these errors are called syntactical errors.
Semantic errors
The semantic errors are also called logical errors. If there is a logical or semantic error in software program then the statement will compile and run correctly but it will not give the desired output because the logic is not correct.
Unit Testing
This is one of the most used testing strategies for testing python programs. This strategy is used for testing units or components of the code. By units or components, we mean classes or functions of the code. Unit testing simplifies the testing of large programming systems by testing “small” units. With the help of the above concept, unit testing may be defined as a method where individual units of source code are tested to determine if they return the desired output.
In our subsequent sections, we will learn about the different Python modules for unit testing.
unittest module
The very first module for unit testing is the unittest module. It is inspired by JUnit and by default included in Python3.6. It supports test automation, sharing of setup and shutdown code for tests, aggregation of tests into collections, and independence of the tests from the reporting framework.
Following are a few important concepts supported by the unittest module
Text fixture
It is used to set up a test so that it can be run before starting the test and tear down after the finish of test. It may involve creation of temporary database, directories, etc. needed before starting the test.
Test case
The test case checks whether a required response is coming from the specific set of inputs or not. The unittest module includes a base class named TestCase which can be used to create new test cases. It includes two by default methods −
-
setUp() − a hook method for setting up the test fixture before exercising it. This is called before calling the implemented test methods.
-
tearDown( − a hook method for deconstructing the class fixture after running all tests in the class.
Test suite
It is a collection of test suites, test cases or both.
Test runner
It controls the running of the test cases or suits and provides the outcome to the user. It may use GUI or simple text interface for providing the outcome.
Example
The following Python program uses the unittest module to test a module named Fibonacci. The program helps in calculating the Fibonacci series of a number. In this example, we have created a class named Fibo_test, to define the test cases by using different methods. These methods are inherited from unittest.TestCase. We are using two by default methods – setUp() and tearDown(). We also define the testfibocal method. The name of the test must be started with the letter test. In the final block, unittest.main() provides a command-line interface to the test script.
import unittest def fibonacci(n): a, b = 0, 1 for i in range(n): a, b = b, a + b return a class Fibo_Test(unittest.TestCase): def setUp(self): print("This is run before our tests would be executed") def tearDown(self): print("This is run after the completion of execution of our tests") def testfibocal(self): self.assertEqual(fib(0), 0) self.assertEqual(fib(1), 1) self.assertEqual(fib(5), 5) self.assertEqual(fib(10), 55) self.assertEqual(fib(20), 6765) if __name__ == "__main__": unittest.main()
When run from the command line, the above script produces an output that looks like this −
Output
>This runs before our tests would be executed. This runs after the completion of execution of our tests. . ---------------------------------------------------------------------- Ran 1 test in 0.006s OK
Now, to make it clearer, we are changing our code which helped in defining the Fibonacci module.
Consider the following code block as an example −
def fibonacci(n): a, b = 0, 1 for i in range(n): a, b = b, a + b return a
A few changes to the code block are made as shown below −
def fibonacci(n): a, b = 1, 1 for i in range(n): a, b = b, a + b return a
Now, after running the script with the changed code, we will get the following output −
>This runs before our tests would be executed. This runs after the completion of execution of our tests. F ====================================================================== FAIL: testCalculation (__main__.Fibo_Test) ---------------------------------------------------------------------- Traceback (most recent call last): File "unitg.py", line 15, in testCalculation self.assertEqual(fib(0), 0) AssertionError: 1 != 0 ---------------------------------------------------------------------- Ran 1 test in 0.007s FAILED (failures = 1)
The above output shows that the module has failed to give the desired output.
Docktest module
The docktest module also helps in unit testing. It also comes prepackaged with python. It is easier to use than the unittest module. The unittest module is more suitable for complex tests. For using the doctest module, we need to import it. The docstring of the corresponding function must have interactive python session along with their outputs.
If everything is fine in our code then there will be no output from the docktest module; otherwise, it will provide the output.
Example
The following Python example uses the docktest module to test a module named Fibonacci , which helps in calculating the Fibonacci series of a number.
import doctest def fibonacci(n): """ Calculates the Fibonacci number >>> fibonacci(0) 0 >>> fibonacci(1) 1 >>> fibonacci(10) 55 >>> fibonacci(20) 6765 >>> """ a, b = 1, 1 for i in range(n): a, b = b, a + b return a if __name__ == "__main__": doctest.testmod()
We can see that the docstring of the corresponding function named fib had interactive python session along with the outputs. If our code is fine then there would be no output from the doctest module. But to see how it works we can run it with the –v option.
(base) D:\ProgramData>python dock_test.py -v Trying: fibonacci(0) Expecting: 0 ok Trying: fibonacci(1) Expecting: 1 ok Trying: fibonacci(10) Expecting: 55 ok Trying: fibonacci(20) Expecting: 6765 ok 1 items had no tests: __main__ 1 items passed all tests: 4 tests in __main__.fibonacci 4 tests in 2 items. 4 passed and 0 failed. Test passed.
Now, we will change the code that helped in defining the Fibonacci module
Consider the following code block as an example −
def fibonacci(n): a, b = 0, 1 for i in range(n): a, b = b, a + b return a
The following code block helps with the changes −
def fibonacci(n): a, b = 1, 1 for i in range(n): a, b = b, a + b return a
After running the script even without the –v option, with the changed code, we will get the output as shown below.
Output
>(base) D:\ProgramData>python dock_test.py ********************************************************************** File "unitg.py", line 6, in __main__.fibonacci Failed example: fibonacci(0) Expected: 0 Got: 1 ********************************************************************** File "unitg.py", line 10, in __main__.fibonacci Failed example: fibonacci(10) Expected: 55 Got: 89 ********************************************************************** File "unitg.py", line 12, in __main__.fibonacci Failed example: fibonacci(20) Expected: 6765 Got: 10946 ********************************************************************** 1 items had failures: 3 of 4 in __main__.fibonacci ***Test Failed*** 3 failures.
We can see in the above output that three tests have failed.