Software Testing

May 12th, 2009

WaterFall Model is the most common and classic of life cycle models, also referred to as a linear-sequential life cycle model. WaterFall Model is very simple to understand and use.  In a waterfall model, each phase must be completed in its entirety before the next phase can begin.  At the end of each phase, a review takes place to determine if the project is on the right path and whether or not to continue or discard the project.  Unlike what I mentioned in the general model, phases do not overlap in a waterfall model.

Requirements: Defines needed information, function, behavior, performance and interfaces.

Design: Data Structures, software architecture, interface representations, algorithmic details.

Implementation: Source code, database, user documentation, testing.

Advantages:

• Simple and easy to use.
• Easy to manage due to the rigidity of the model – each phase has specific deliverables and a review process.
• Phases are processed and completed one at a time.

Works well for smaller projects where requirements are very well understood.

Disadvantages:

• Adjusting scope during the life cycle can kill a project
• No working software is produced until late during the life cycle.
• High amounts of risk and uncertainty.
• Poor model for complex and object-oriented projects.
• Poor model for long and ongoing projects.
• Poor model where requirements are at a moderate to high risk of changing.

When to Use Water Fall Model:

• Requirements are very well known.
• Product definition is stable.
• Technology is understood.
• New version of an existing product.
• Porting an existing product to a new platform.

SDLC:Software Development Lifecycle Model

Requirements

Business requirements are gathered in this phase.  Who is going to use the system?  How will they use the system?  What data should be input into the system?  What data should be output by the system?  These are general questions that get answered during a requirements gathering phase. This phase is the main focus of the project managers and stake holders.  Meetings with managers, stake holders and users are held in order to determine the requirements.  This produces a nice big list of functionality that the system should provide, which describes functions the system should perform, business logic that processes data, what data is stored and used by the system, and how the user interface should work.  The overall result is the system as a whole and how it performs, not how it is actually going to do it.

Design

The software system design is produced from the results of the requirements phase.  Architects have the ball in their court during this phase and this is the phase in which their focus lies.  This is where the details on how the system will work is produced.  Architecture, including hardware and software, communication, software design (UML is produced here) are all part of the deliverables of a design phase.

Implementation

Code is produced from the deliverables of the design phase during implementation, and this is the longest phase of the software development life cycle.  For a developer, this is the main focus of the life cycle because this is where the code is produced.  Implementation my overlap with both the design and testing phases.  Many tools exists (CASE tools) to actually automate the production of code using information gathered and produced during the design phase.

Testing

During testing, the implementation is tested against the requirements to make sure that the product is actually solving the needs addressed and gathered during the requirements phase.  Unit tests and system/acceptance tests are done during this phase.  Unit tests act on a specific component of the system, while system tests act on the system as a whole.

So in a nutshell, that is a very basic overview of the general software development life cycle model.  Now lets delve into some of the traditional and widely used variations.

Packages and Object Diagrams

To simplify complex class diagrams, you can group classes into packages. A package is a collection of logically related UML elements. The diagram below is a business model in which the classes are grouped into packages.

Packages appear as rectangles with small tabs at the top. The package name is on the tab or inside the rectangle. The dotted arrows are dependencies. One package depends on another if changes in the other could possibly force changes in the first.

Object diagrams show instances instead of classes. They are useful for explaining small pieces with complicated relationships, especially recursive relationships.

This small class diagram shows that a university Department can contain lots of other Departments.

object diagram below instantiates the class diagram, replacing it by a concrete example.

Each rectangle in the object diagram corresponds to a single instance. Instance names are underlined in UML diagrams. Class or instance names may be omitted from object diagrams as long as the diagram meaning is still clear.

Class Diagram:

A Class diagram gives an overview of a system by showing its classes and the relationships among them. Class diagrams are static — they display what interacts but not what happens when they do interact.

The class diagrams below models a customer order from a retail catalog. The central class is the Order. Associated with it are the Customer making the purchase and the Payment. A Payment is one of three kinds: Cash, Check, or Credit. The order contains OrderDetails (line items), each with its associated Item.

UML class notation is a rectangle divided into three parts: class name, attributes, and operations. Names of abstract classes, such as Payment, are in italics. Relationships between classes are the connecting links.

Our class diagram has three kinds of relationships.

• Association — a relationship between instances of the two classes. There is an association between two classes if an instance of one class must know about the other in order to perform its work. In a diagram, an association is a link connecting two classes.
• Aggregation — an association in which one class belongs to a collection. An aggregation has a diamond end pointing to the part containing the whole. In our diagram, Order has a collection of OrderDetails.
• Generalization — an inheritance link indicating one class is a superclass of the other. A generalization has a triangle pointing to the superclass. Payment is a superclass of Cash, Check, and Credit.

An association has two ends. An end may have a role name to clarify the nature of the association. For example, an OrderDetail is a line item of each Order.

A navigability arrow on an association shows which direction the association can be traversed or queried. An OrderDetail can be queried about its Item, but not the other way around. The arrow also lets you know who “owns” the association’s implementation; in this case, OrderDetail has an Item. Associations with no navigability arrows are bi-directional.

The multiplicity of an association end is the number of possible instances of the class associated with a single instance of the other end. Multiplicities are single numbers or ranges of numbers. In our example, there can be only one Customer for each Order, but a Customer can have any number of Orders.

This table gives the most common multiplicities.

Every class diagram has classes, associations, and multiplicities. Navigability and roles are optional items placed in a diagram to provide clarity.

Use Case diagram:

Use case diagrams describe what a system does from the standpoint of an external observer. The emphasis is on what a system does rather than how

Use case diagrams are closely connected to scenarios. A scenario is an example of what happens when someone interacts with the system

Let’s take an example of a medical clinic:

“”A patient calls the clinic to make an appointment for a yearly checkup. The receptionist finds the nearest empty time slot in the appointment book and schedules the appointment for that time slot. “”

A use case is a summary of scenarios for a single task or goal. An actor is who or what initiates the events involved in that task. Actors are simply roles that people or objects play. The picture below is a Make Appointment use case for the medical clinic. The actor is a Patient. The connection between actor and use case is a communication association (or communication for short).

Actors are stick figures. Use cases are ovals. Communications are lines that link actors to use cases.

A use case diagram is a collection of actors, use cases, and their communications. We’ve put Make Appointment as part of a diagram with four actors and four use cases. Notice that a single use case can have multiple actors.

Use case diagrams are helpful in three areas.

• Determining features (requirements). New use cases often generate new requirements as the system is analyzed and the design takes shape.
• Communicating with clients. Their notational simplicity makes use case diagrams a good way for developers to communicate with clients.
• Generating test cases. The collection of scenarios for a use case may suggest a suite of test cases for those scenarios.

What Is UML?

The OMG specification states:

“The Unified Modeling Language (UML) is a graphical language for visualizing, specifying, constructing, and documenting the artifacts of a software-intensive system. The UML offers a standard way to write a system’s blueprints, including conceptual things such as business processes and system functions as well as concrete things such as programming language statements, database schemas, and reusable software components.”

In simple words:

UML (Unified Modeling Language):

• Is a graphical modeling language used to express designs.
• Is a standardized language used to specify artifacts and components of a software system.
• It describes notation not a process.
• It does not put forth a single method or process of design, but rather is a standardized tool that can be used in a design process.

There are nine kinds of UML diagrams. Let’s discuss them one by one with an example.

Black box testing:

Black box testing also known as “Functional Testing”. A type of software testing where internal workings of item being tested is unknown to the tester. Say for example, in Black box testing when applied to software engineering the tester has only the knowlegde of “legal” inputs not the expected output and how program actually arrives to those outputs.

Black box testing is what most of us practice and is used most widely. Black Box testing which is very close to customer experience.

Black box testing methods include equivalence partitioning, boundary value analysis, all-pairs testing, fuzz testing, model-based testing, traceability matrix, exploratory testing and specification-based testing.

For Black Box testing, test groups are often used, “Test groups are aka “professional idiots”…people those who are good at designing incorrect data. Also, do to the nature of black box testing; the test planning can begin as soon as the specifications are written.

As a test engineer if you are performing black box test cases, one thing that you need to make sure is that you do not make any assumptions about the system based on your knowledge. Assumption created in our mind because of the system knowledge could harm testing effort and increase the chances of missing critical test cases.

The only input can be for test engineer in the Black Box testing is the requirement document and functionality of system which can be get by working with the system.

Purpose of black box testing is to

1. Make sure that system is working in accordance with the system requirement.

2. Make sure that system is meeting user expectation.

For example, suppose we are testing a function that uses the quadratic formula to determine the two roots of a second-degree polynomial ax²+bx+c. For simplicity, assume that we are going to work only with real numbers, and print an error message if it turns out that the two roots are complex numbers (numbers involving the square root of a negative number).

We can come up with test data for each of the four cases, based on values of the polynomial’s discriminant (b²-4ac):

Easy data (discriminant is a perfect square):

Typical data (discriminant is positive):

Boundary / extreme data (discriminant is zero):

Bogus data (discriminant is negative, or a is zero):

As with glass-box testing, you should test your code with each set of test data. If the answers match, then your code passes the black-box test.

Process of making sure that older programming still works with the new changes is called Regression testing.

Any time you modify an implementation within a program, Regression software testing should be performed. This can be done by rerunning the existing test cases against the modified code to check whether the code change breaks the existing functionality. Regression testing is a normal part of the program development process and, in larger companies, is done by code testing specialists. Test department coders develop code test scenarios and exercises that will test new units of code after they have been written. One should try to spend as little time as possible doing regression testing without reducing the probability that you will detect new failures in old, already tested code.

Some strategies and factors to consider during this Regression Testing process include the following:

• Make changes (small and large) to data and find any resulting corruption.
• Identify tests that the program consistently passes and archive them.
• Watch for side effects of fixes. The bug itself might be fixed but the fix might create other bugs.
• Test fixed bugs promptly. The programmer might have handled the symptoms but not have gotten to the underlying cause.
• Focus on functional issues, not those related to design.
• Trace the effects of the changes on program memory.
• Write a regression test for each bug fixed.
• If two or more tests are similar, determine which is less effective and get rid of it.

Process of performing end to end test cases under anticipated production load is called Load Testing.

Objective here to determine the response of various time critical transactions and business processes and to ensure that they are with in documented expectations (or Service Level Agreements – SLAs). Load testing can also be done in the field to obtain a qualitative idea of how well a system functions in the “real world.”

Load testing is a part of a more general process known as performance testing. Examples of load testing include:

• Subjecting a server to a large amount of e-mail traffic.
• Downloading a series of large files from the Internet.
• Assigning many jobs to a printer in a queue.
• Running multiple applications on a computer or server simultaneously.
• Writing and reading data to and from a hard disk continuously.

Purpose of Load testing:

The purpose of any load test should be clearly understood and documented. A load test usually fits into one of the following categories:

• Quantification of risk- Determine, through formal testing, the likelihood that system performance will meet the formal stated performance expectations of stakeholders, such as response time requirements under given levels of load. This is a traditional Quality Assurance (QA) type test. Note that load testing does not mitigate risk directly, but through identification and quantification of risk, presents tuning opportunities and an impetus for remediation that will mitigate risk.
• Determination of minimum configuration- Determine, through formal testing, the minimum configuration that will allow the system to meet the formal stated performance expectations of stakeholders – so that extraneous hardware, software testing and the associated cost of ownership can be minimized. This is a Business Technology Optimization (BTO) type test.

Logical extension of unit testing is called Integration Testing.

Objective here is to make sure that the interaction of two or more components produces results that satisfy functional requirement. In its simplest form, two units that have already been tested are combined into a component and the interface between them is tested. In Integration Testing the test cases are developed with the purpose of exercising the interface between the components. Integration Testing can also be treated as testing assumption of fellow programmer. Integration testing identifies problems that occur when units are combined.

By using a test plan that requires you to test each unit and ensure the viability of each before combining units, you know that any errors discovered when combining units are likely related to the interface between units. This method reduces the number of possibilities to a far simpler level of analysis.

Integration Testing can be performed in various ways but here are the following three strategies:

• Top-down approach: Here Integration Testing requires the highest-level modules be test and integrated first. This allows high-level logic and data flow to be tested early in the process and it tends to minimize the need for drivers. However, the need for stubs complicates test management and low-level utilities are tested relatively late in the development cycle. Another disadvantage of top-down integration testing is its poor support for early release of limited functionality.
• Bottom-up approach: Here Integration Testing requires the lowest-level units be tested and integrated first. These units are frequently referred to as utility modules. By using this approach, utility modules are tested early in the development process and the need for stubs is minimized. The downside, however, is that the need for drivers complicates test management and high-level logic and data flow are tested late. Like the top-down approach, the bottom-up approach also provides poor support for early release of limited functionality.
• Third approach: Sometimes referred to as the umbrella approach, requires testing along functional data and control-flow paths. First, the inputs for functions are integrated in the bottom-up pattern discussed above. The outputs for each function are then integrated in the top-down manner. The primary advantage of this approach is the degree of support for early release of limited functionality. Integration Testing also helps minimize the need for stubs and drivers. The potential weaknesses of this approach are significant, however, in that it can be less systematic than the other two approaches, leading to the need for more regression testing.

Here is the above picture first the lowest level modules “B” and “C” are tested with drives then next higher module “A” is testing with a drive integrated with module “B” and “C”. Finally the integration will be continued till the top-level module is tested. The total number of tests required by this technique is the sum of the cyclomatic complexities of all modules being integrated.