Software Testing

General Topics

###3. Software Testing

Supposing that you have acquired some familiarity with C++ and Software Engineering, things are starting to become serious. You have learned about some software design patterns and which coding practices consist a merit and which not. But above all you are trying to adopt new programming styles and a new cooperation culture: the agile development process.

Agile practices are best suited for people that are eager to adapt quickly but yet are stable. People who do not follow a austere to-do list but are structured. People who feel confident about their choices but are willing to change them if needed. Testing will help someone boost their confidence, learn to work in a structured way and define a benchmark upon which one can measure his choices. Software testing is all about writing pieces of code that serve and validate the correctness of your development code. So actually are programs that test other programs. Someone might say: but who, then, tests the tests? The latter meaningful question makes us understand that some rules are needed in order to test correctly and making the matter of testing the tests unnecessary.

Virtues of a test

1. It is simple: Ideally it is like a huge validation table. If f: D(f) -> A, then you check as many values of D(f) needed to ensure that you will get the correct response in f(D(f)).
2. Checks for odd inputs: How will your structure behave to something large? to something out of domain's range or something at domain's borders? to misbehavior of other components? is it safe? (stress testing)
3. Tests only one responsibility at a time: Testing a component means that you trust the functionality of its sub-components. Otherwise there is no need to make modular our design. Modular -> Easily testable.
4. Has no logic and flow control: Something that controls flow needs to be tested! So ifs and switches and other means of flow control are prohibited.
5. Uses safe APIs and libraries: Usage of working things are permitted. So, for example, boost's smart pointers are to be used as they are and not to be tested by some procedure.
6. Is designed as an object oriented program: program after all ... sorry i got away :p

But how do we test software?

Hopefully there are libraries (that don't need to be tested of course) that allow us to do so in a object oriented way, thus organizing our tests to structures. For C++ there is Google's gtest library, which in conjunction with gmock, provides a complete testing infrastructure. Mocks are objects that implement the interface of a tested object and are used to test the sequential logic of an object, its event succession and overall how program flows through it. Mocks are supported with functions that count and test how many times their methods are triggered and sometimes react to their calls. Fake objects, instead, are just stubs of the interface that react to their calls. At this point, i would recommend reading gtest's and gmock's primer tutorials and trying to make a simple test.

What do we test?

Everything.

What? not satisfied??? OK let's organize a little bit the testing process. The third virtue of a test implies that there should be some structure and sequence in our testing, otherwise we cannot be sure of our testing results. There are two methodologies top-down and bottom-up testing. In bottom-up testing, one tries to test first all independent modules and then recursively all modules that their submodule have already been tested successfully. Regardless the method, all modules are approached as unique units, something that is enabled because of the object oriented approach. Unit testing means that one will test method-by-method of a module for (ideally) all module's states. In unit testing, we do white-box or gray-box tests. That is because we have access to the module's states and we can manipulate our module in each test suite to have a certain state. Getting one level above unit testing, when we have completed testing the topmost module too, then we proceed with functional testing our program. Functional testing is trying to prove that our program does what is developed to do, meets the minimum requirements, is robust and bug free. Is what every developer does when he finishes a program. Runs the program from the start with a set of inputs, expecting the right output. This is called black-box testing. Gray-box testing is utilized too. Functional testing, of course, is of course again done by a program-tester.
Sometimes a software project is composed by many programs that collaborate with each other in a greater scale architecture in order to make a certain system running. Then further testing is needed. In this scale we want to test the interfaces between the components as well as their integration into larger components, up to the point that they consist the whole system. This scale of testing is called integration testing in general. We try to make this part of testing run by a program too, usually by considering it as a bigger functional test (whose difficulty is depended on the interface framework that is used between the components). Other techniques are also used to ensure the stability and health of the system by testing the interfaces between the components. Interface testing means we ensure that a system's components are communicating satisfyingly.
Lastly, as a system scale test, we can run user-driven or AI-driven simulations in a appropriately developed environment that is able to give a system correctly its inputs and handle correctly its outputs. Usually during a simulation, we want to see in real-time how the system is behaving.

Modern development methods emphasize on automating the testing process. Agile development needs testing in order to keep its stability while adapting and changing continuously. Testing needs to be continuous too and try to keep up with the pace of the development. For this purpose a continuous integration (CI) project scheme can be made in order to automate the whole testing process, from the existence of program-testers to the automatically triggering of the test suites written. In project management level of organization, the techniques that involve how the main program along with its tests are to be developed are described by the practices of test-driven development (TDD). Combining agile development with test-driven development correctly you can achieve Extreme Programming (XP).

Next chapter: Tools