Testing throughout the Software Life Cycle: CTFL Tutorial
2.1 Testing throughout the Software Life Cycle
Hello and welcome to the Certified Tester Foundation Level (CTFL®) course offered by Simplilearn. This is the second lesson of the course. In this lesson, we are going to discuss ‘Testing throughout the Software Life Cycle.’ Let us look at the course map in the next screen.
2.2 Course Map
Lesson two is divided into four topics. They are: Software Development Models, Test Levels, Test Types, and Maintenance Testing. Let us begin with the objectives in the next screen.
After completing this lesson, you will be able to: Explain the importance of software development models, Identify the different levels of testing, Explain the important types of testing, and Distinguish between maintenance testing and new application testing. In the next screen we will begin with the first topic, ‘Software Development Models.’
2.4 Software Development Models
This topic deals with the different software development models used in the industry. The next screen introduces us to the software development models.
2.5 Software Development Models—Introduction
Software development process is divided into several logical stages. It allows a software development company to organize its work efficiently. This, in turn, helps in building a software product of the required functionality, within a specific time frame and budget. Many development life cycles, ranging from lightweight to fast methodologies, are established to achieve different required objectives. The development motive is the time to market, which is achieved through fully controlled and documented methodologies, where quality and reliability are the key factors. Each of these methodologies has its place in modern software development. Each model has its own phases and order in which they are carried out. The selection of model will have a huge impact on testing as testing does not exist in isolation, and its activities depend on that of software development. Testing is planned and executed depending on the Software Development Life Cycle or SDLC. In the next screen, we will understand the concept of product lifecycle management.
2.6 Product Lifecycle Management
Product Lifecycle Management or PLM is a process of managing the entire lifecycle of a product from requirements, through design and manufacture, to service and disposal. PLM describes the engineering aspect of a product, from managing descriptions and properties of a product through its development and utility. PLM speeds up the product development process by systematically arranging product requirements, and product data management. This reduces the engineering costs. Let us focus on the product lifecycle, in the next screen.
2.7 Product Lifecycle
The four product lifecycle phases are introduction, growth, maturity, and decline. Introduction phase is where the product is developed and launched into the market. This phase requires more investment without any returns. Hence, they are no profits during this phase. After the product has been successfully launched and promoted in the market, it enters the Growth phase, where the product is in use and has new customers, bringing profits to the company. When the product reaches Maturity, this phase brings the highest profit. In Decline phase, profits drop gradually as customers move to more advanced products, either by a competitor or by the same vendor. For example, both Office 2003 and Office 2007 are products of Microsoft. We will continue our discussion on product lifecycle in the next screen.
2.8 Product Lifecycle (contd.)
Product lifecycle involves a series of projects to plan, develop, launch, enhance, and maintain the product. The lifecycle duration could vary from weeks to years, depending on the nature and need of the product, in the market. In the introduction phase of the product, it is creates the market for itself and slowly inches towards growth. During the growth phase, the product increases its market share very rapidly. This is the phase in which the product registers the steepest growth in revenue. Organizations try to maximize this stage by continuously adding new features or spending more on marketing. In the long run however, each product passes through the growth phase into the maturity phase. In this phase, though the market share is still growing, the growth becomes slower than before. After a period, the growth becomes negative and the market share is eroded by other products in the market. At this point, the product enters the decline stage. While the product still has significant market share, it is no longer growing and might eventually be phased out. The management should keep the introduction phase the shortest, to maximize the amount of time the product stays in the growth and maturity phase. In the next screen, we will discuss an example of product lifecycle.
2.9 Product Lifecycle—Example
One of the first products that made music portable with the user was the Radio. This was soon transitioned by the Walkman, which also had an inbuilt radio. The Walkman was replaced with the IPod, which was soon replaced with the Smart Phone. Each product, irrespective of the degree of innovativeness, has its cycle of Introduction, Growth, Maturity and Decline. Some products have a longer Maturity phase than the others, however, at the end, each product enters the Decline phase. In this case, the Radio had probably the longest maturity phase and the pace of innovation in those days was very less. Let us look at a project lifecycle, in the next screen.
2.10 Project Lifecycle
A project life cycle is a logical sequence of activities to accomplish the objectives of the project. Regardless of complexity of the project, all projects go through a series of activities from inception to closure. The first phase of a project is Initiation, in which, critical success factors or core objectives for the project are defined. The key deliverable from this phase is the project charter, which authorizes the project start, defines the objectives and high-level budgets, and sponsors commitments for the project. The second phase is Planning, in which, the project approach is determined and the project is divided into smaller tasks with interim deliverables. The Project Manager plans for all support requirements, identifies risks, and defines roles and responsibilities for the project. This phase also includes determining milestones for each of these deliverables. The third phase is the Execution, in which the project execution takes place, and the project deliverables are created and implemented. The last phase is Closing, where the project is formally closed or handed over to the next team. For example, once a product is developed and launched, it is handed over to the production support team that maintains and enhances the product. We will continue our discussion on the project lifecycle, in the next screen.
2.11 Project Lifecycle (contd.)
Project activities must be grouped into phases so the Project Manager and the core team can plan and organize resources for each activity easily and effectively. While these phases are sequential in the first occurrence, they often become iterative during the course of the project as constant re-planning, and re-execution is required to manage the risks and change requests for the project. The curve on the graph indicates the level of cost and effort involved in the project at each of the stages. The effort is considerably low during the initiation and planning phases, as there are only few people working on the project planning and strategizing the project. However the effort peaks during the execution phase and finally starts to taper off as the project nears completion. During the closure phase, most of the resources are let go and the cost of running the project becomes significantly low. Let us focus on software project lifecycle, in the next screen.
2.12 Software Project Lifecycle
A software project lifecycle consists of all the phases of a regular project which includes initiation, planning, execution, and closure. In addition, software development lifecycle includes requirement gathering, design, coding, testing, and project launch. For small projects, all the phases may be conducted as a part of one project. For large projects, each phase may be a separate project. For example, there may be one project to gather the requirements from the users. This project, when closed, will transition to the design project, where the requirements will be designed, reviewed, and so on. In the following screen, we will look at an example of project lifecycle.
2.13 Project Lifecycle—Example
IPod Mini was released on February 20, 2004. Its revised version was declared on February 23, 2005, and the product was released later in the same year. During production, due to its popularity, it was often unavailable to the customers. On September 7, 2005 it was replaced by the iPod Nano. The move from iPod’s touch-sensitive scroll wheel to the switches beneath the wheel or click wheel can be considered as an individual project with phases of initiation, planning, execution and closure to get the click wheel implemented. During the manufacture of click wheel, the software division of Apple worked on writing the software behind its execution. This software followed the SDLC including Requirements Capture, Design, Testing, and Coding. We will focus on software development models, in the next screen.
2.14 Software Development Models
As already discussed, software lifecycle models describe the phases of the software cycle and the order in which those phases are executed. The common SDLC models are Waterfall Model, V-Model, Iterative or Incremental Development Model, and Testing within a Life Cycle Model. Let us discuss the waterfall model in the following screen.
2.15 Waterfall Model
Waterfall is the most common one of all SDLC models. As seen in the image on the screen, in this model, each phase must be completed before starting the next phase. At the end of each phase, a review takes place to determine if the project is on the right path, and whether to continue it. In summary, the waterfall model is meant to function in a systematic way that takes the production of the software from the basic step to detailing, resembling a waterfall which begins at the top of a cliff and flows down. Let us continue our discussion of the waterfall model in the following screen.
2.16 Waterfall Model—Phases
Waterfall model is a sequential design process in which progress is seen flowing steadily down through conception, initiation, analysis, system design, construction, testing, production or implementation, and maintenance phases. Let us understand each phase. In System Conceptualization or conception phase the overall vision and the system requirements are identified; and the high level objectives are determined. System Conceptualization refers to the consideration of all aspects of the business, with the goal of determining their interrelationship and priority in the software development. The purpose of the Initiation phase is to conduct a high-level investigation of the business requirement and give a solution recommendation. Here, for each phase, the sponsorship commitments are secured; budgets and timelines are finalized; and objectives are developed and signed off. The core team for the project including the Project Manager is also identified. Analysis refers to the collection of system requirements, with the goal of determining how they will be accommodated into the system. Extensive communication between the customer and the technical team is essential to understand the possibility of implementing the requirements into the software. Designing is about identifying how the system will be developed to meet the user requirements. More specifically, the system design phase focuses on different aspects like, the information to be processed in the system, software construction, and appearance of the system, and standards to be followed. Let us continue our discussion on waterfall model in the following screen.
2.17 Waterfall Model—Phases (contd.)
Other phases of Waterfall model are: Construction, programming, or development phase involves the creation of system software. Requirements and systems specifications from the design phase are translated into machine readable computer code in this phase. Testing is performed to ensure the software created and added to the developing system is working correctly and efficiently. Testing generally focuses on two areas; they are: internal and external efficiency. The goal of internal efficiency testing is to ensure the computer code is efficient and standardized. The goal of external efficiency testing is to verify that the software is performing all the necessary functions or sub-functions according to system design. . After successfully completing the testing phase, the software will be made available for real-time use post its implementation on the production site. In the Implementation phase the users use the system. Maintenance and management is the last phase and is needed to ensure the system will continue to perform as desired. We will focus on V-model in the next screen.
V-model as also known as Sequential Development Model follows an ordered path of execution of phases. Each phase must be completed before the next phase begins. As seen in the image on the screen, the four key levels of testing in this model are: Component test, Integration test, System test, and Acceptance test. This model is appropriate for test activities as they start in parallel with each phase of requirements, design, and coding. The testing procedures are developed early in the life cycle before any coding is done, giving testers enough opportunity to understand the requirements, and raise questions and issues on the requirements. The first phase in this life cycle model is requirements gathering. Requirement gathering deals with capturing the needs of the user from the system. As requirements are finalized, testing team gets involved to review the requirements and build test plans for Acceptance testing. As the system design gets finalized, testing team gets involved in review of the design and development of the integration test plan that will be used to test these design elements. In the low-level design phase, the actual software components are designed, and unit tests are created. The testing team also starts developing systems, and integrating test cases and test procedures in this phase. Coding is the phase, where all coding takes place. After coding, the path of execution continues, where the test plans developed earlier are used. Let us discuss iterative-incremental model in the following screen.
2.19 Iterative-Incremental Model
Iterative-Incremental model is appropriate for complex projects where short development cycles are needed. As seen in the image on the screen, in this model, each short development cycle has its own requirement, designing, building, and testing phases. Each phase addresses a couple of requirements which will go through the usual software development cycle. The next iteration addresses another set of requirements using the same development cycle. These additional requirements are added to the existing, already developed system. The Iterative-Incremental model is appropriate where a working system is essential to validate requirements and get user confidence. A few examples of Iterative-Incremental model are: prototyping, Rapid Application Development or RAD, Rational Unified Process or RUP and Agile Development models. These models are used, if time to market is a Key factor. In the next screen, we will discuss testing within a life cycle model.
2.20 Testing within a Life Cycle Model
Testing should be built within a life cycle of any development model, which is most cost effective as it gives a chance to develop system and identify defects introduced in a phase immediately unlike traditional model, where defects are undetected till the testing phase. The right test approach selection should be based on a variety of factors. As seen on the screen, few factors to be considered are: nature, software development model, risk associated, and complexity of the project. We will continue our discussion on testing within a life cycle model, in the next screen.
2.21 Testing within a Life Cycle Model (contd.)
Every SDLC model has several testing characteristics. Every development activity should have a corresponding testing activity. This will ensure the developed product, is tested at the same level of granularity. Each test level has objectives specific to it. Testers should be involved in reviewing documents as soon as drafts are available in the development life cycle. This ensures the defects are caught as early as possible. Type of test and test strategy depends on the nature or complexity of the project. There is no standard strategy which meets all the project requirements. In the next screen we will begin with the second topic, ‘Test Levels.’
2.22 Test Levels
This topic deals with the different levels of testing. There are four main levels of testing. They are: component testing, integration testing, system testing, and acceptance testing. These levels are built for each level of development activity. Let us introduce these briefly, in the next screen.
2.23 Levels of Testing
The test levels to be performed in a sequential order are as follows: The first level, component testing, tests separately testable software products. The second level, integration testing, tests interfaces between components, and interactions with different parts of the system. The third level, system testing, investigates both functional and non-functional requirements. The last level, acceptance testing, is where the system readiness to move to production and end user usage have to be assessed. Let us focus on the levels of testing beginning with component testing, in the next screen.
2.24 Component Testing—Introduction
Component may be a class or a cluster of classes. There are several situations in which an individual class should be tested regardless of their size or complexity. Components can be broadly categorized as: reusable, domain, and commercial components. Reusable components should be tested over a wide range of values than a component intended for a single focused use. Domain components represent significant domain concepts and should be tested for accuracy. Commercial components are sold as individual products and should be tested as it is a potential source of liability. Component testing is an important aspect of the test plan for a project. This is the first level testing of executable code and therefore a large percentage of errors can be found at this level. In this level, each component is individually tested using white box testing techniques, usually by developer. For example, an application has 10 functional components, and each is tested in isolation before all are integrated into one. Component testing is strongly recommended invariant from the type of development model selected. We will understand the concept of test-driven development, in the next screen.
2.25 Component Testing—Test-Driven Development
Test Driven Development is a unique way of conducting component testing. The idea of test driven development is to write tests before the code. When the code is written, confirm whether a new chunk of code is completely functional. In most cases, for a single test, if the test fails, you write code to pass the test. Then, you repeat the test while building and integrating the next piece of code. Test-Driven development ensures all the code present in the system is tested and there is no redundant code. However, it is time consuming and needs a disciplined approach to build this code. In the next screen, we will discuss the inputs, test objects, and outputs of component testing.
2.26 Component Testing—Inputs, Test Objects, and Outputs
Inputs, test objects, and outputs of component testing are listed below. Requirements, detailed design, and code are the inputs for component testing. Developers should study this test basis to identify how to test each component. The items to be tested are called Test objects, which can be components, different programs, data conversions or migrations programs, or database modules. Unit tested code is the output of component testing. This code should not have any component level defects. Let us look at integration testing, in the next screen.
2.27 Integration Testing—Introduction
Once components have been tested individually, these are then integrated and tested. This is called Integration testing, where software components, hardware components, or both are combined and tested to evaluate their interactions. Testers perform integration testing with formal defect management mechanisms. At this stage, the testers concentrate only on integration. For example, if they integrate Component A and B, they are interested in testing the communication between A and B and not the functionality of the individual modules. Both functional and structural approaches can be used to conduct integration testing. In the next screen, we will understand the strategies to perform integration testing.
2.28 Integration Testing—Strategies
Integration testing can be performed in different ways. The three common strategies to perform this testing are: Top-Down, Bottom-Up, and Big Bang approach. The Top-Down approach to integration testing requires the highest-level modules to be tested and integrated first. This allows early testing of high-level logic and data flow in the process and tends to minimize the need for drivers. However, the need for stubs complicates test management, and low-level utilities are tested in the development cycle later. Another disadvantage is its poor support for early release of limited functionality. The Bottom-Up approach requires the lowest-level units or utility modules to be tested and integrated first. Through this approach, utility modules are tested early in the development process and the need for stubs is minimized. However, the need for drivers complicates test management. The high-level logic and the data flow are tested later. Similar to the top-down approach, this approach also provides poor support for early release of limited functionality. The third approach Big Bang, or umbrella approach, requires testing functional data and control flow paths. First, functions inputs are integrated in the bottom-up pattern. Each function outputs are then integrated in the top-down manner. The primary advantage of this approach is the degree of support for early release of limited functionality. It also helps in minimizing the need for stubs and drivers. Let us understand the difference between component and system integration testing, in the next screen.
2.29 Component vs. System Integration Testing
Logically, Component Integration Testing is an extension of Component testing, where, two already tested components or units are combined into a component and the interface between them is tested. Integration testing identifies problems that occur when units are combined. This is typically carried out after Component Testing. System integration testing tests the interactions between different systems or between the hardware and the system. This form of testing tests the software system or application as a whole. This is typically carried out after System Testing. In the next screen, we will discuss the inputs, test objects, and outputs of integration testing.
2.30 Integration Testing—Inputs, Test Objects, and Outputs
Inputs for integration testing includes use cases, workflows, architecture, system design, and software. Integration testing mainly tests objects like subsystems, database implementation, infrastructure, interfaces, and system configuration and configuration data. Outputs of integration testing include integration test results, defect logs and the integrated tested code without defects. Let us look at system testing, in the next screen.
2.31 System Testing—Introduction
Once the integration testing has been performed, the system is subjected to end-to-end or system testing. System testing is done on application software to evaluate its overall compliance with the business, functional, or end-user requirements. System testing mainly uses Black Box software testing techniques, hence the knowledge of internal design, structure, or code is not required. It is usually carried out by independent testing team and aims to detect defects or bugs both within the interfaces, and the software. It follows formal defect management process as well. During system testing, the focus is on the software design, behavior, and expectations of the customer. System testing can be described as a sequence of creation of system test plan, creation of system test cases, selection and/or creation of test data for system testing, execution of test cases, bug fixing, regression testing, and repeating the software test cycle activities. System testing will include both functional and non-functional requirements such as performance, usability, and security. The test environment for system testing should correspond as closely as possible to production environment, to minimize the risk of not finding environment specific failures in testing. In the next screen, we will discuss the inputs, test objects, and outputs of system testing.
2.32 System Testing—Inputs, Test Objects, and Outputs
The inputs for system testing are use cases, functional specifications, system design, and risk analysis report. In system testing, the main test objects are the system, the associated user and operational manuals, and the system configuration and configuration data. The main test outputs include test results, defect logs, and the integrated tested code. Let us focus on acceptance testing, in the next screen.
2.33 Acceptance Testing—Introduction
Once the application is ready to be released, the crucial step is acceptance testing before the software is released commercially. The goal of the acceptance testing is to instill confidence in the system to users. This is carried out by a third party user or the customer. To ensure effective acceptance testing, test cases are created which validates end-to-end business process, system transactions, and user access. The test cases ensure proper coverage of all the scenarios during testing. Here, the main focus is on the exact real world usage of the application and testing is done in a similar environment that simulates the production environment. The next screen deals with the types of acceptance testing.
2.34 Acceptance Testing—Types
Typical forms of Acceptance testing includes User Acceptance testing which is also known as UAT, Operational Acceptance testing, Contract or Regular User Acceptance testing, and Alpha and Beta testing also known as Field testing. The user acceptance testing tests if the system is fit to be operated by the user. The main goal of operational acceptance testing is to ensure the system backup and recovery is 100%, and that the security data load and migration tasks are functioning according to business requirements. Contract or Regular UAT tests ensure the contractual, government and/or regulatory body regulations, and agreed upon terms are met. Alpha testing is done at the software development site. This differs from system testing and is tested in a simulated production environment. However, Beta testing is done at customer’s place in a simulated production environment, which provides opportunity of getting direct end user feedback and improving before the system launch. In the next screen, we will discuss the inputs, test objects, and outputs of acceptance testing.
2.35 Acceptance Testing—Inputs, Tools and Techniques, and Outputs
The inputs for acceptance testing are system requirements, use cases, user requirements, business process, and risk analysis report to understand the system. The main test objects for system testing are business processes on fully integrated system, configuration data, operational and maintenance processes, user procedures, forms, and reports. The main outputs are test results, defect logs, and the acceptance tested code without defects. In the next screen, let us understand the test levels with an example.
2.36 Test Levels—Example
Let us consider the Android Operating System as the Software under test. In the component testing stage, the developers are focused on independently testing each component under development. For example, Contacts, Messaging, and Internet Browser. In the Integration Phase, they incorporate these components and test it. For example, check whether the messages being sent correctly to the Contacts. During the System Testing Phase the OS is tested for End to End flows, which execute multiple components sequentially. For example, searching a contact, calling the contact, disconnecting the phone in between, sending a message to the contact, and copying the contact information to internet browser. During this stage, the OS is handed to live users to test and report. In the next screen, we will begin with the third topic, ‘Test Types.’
2.37 Test Types
This topic deals with the different test types. Let us begin discussing the types of testing, in the next screen.
2.38 Types of Testing
Testing types are distinguished based on the objectives of testing. The four major testing types are: Functional, Non-functional, Structural, and Re-testing and Regression testing. Let us focus on Functional Testing, in the next screen.
2.39 Functional Testing
Functional testing can be defined as ‘Testing of all features and functionalities of software to ensure all the requirements and specifications provided in Business Requirement Documents or BRD are met.’ Functional testing is considered as Black Box testing as the software is tested based on functional requirements. Here, the internal structure of the software is not considered. Functional testers normally do not need to understand the coding of software, however, they should understand the functional requirements expected to be met by the tested system. Here, functions or modules are tested by giving different input data and the output is compared against the expected results of business requirements. These tests verify whether the system is behaving as intended. Functional testing mainly involves identifying the functionalities to be tested, preparing the test data, documenting the expected result, executing the planned test case, comparing the actual result with expected result, and reporting the deviation, if any. Functional testing can be done at any level like unit, integration, and system testing. Functional test cases can also be automated to execute them constantly with different data sets. Let us look at the different types of functional testing, in the next screen.
2.40 Functional Testing—Types
There are different types of Functional testing like Unit Testing, Smoke or Sanity Testing, Integration Testing, Interface and Usability Testing, Security Testing, System Testing, Regression Testing, Pre-user Acceptance Testing, User Acceptance Testing, White Box and Black Box Testing, and Globalization and Localization Testing. In unit testing, developer will test their developed code against the functional requirements. Smoke or Sanity testing is the first type of testing done once the code is deployed into the test environment. These tests are done to ensure the software does not have any show stopper defects and are available for tester. Here, tester tests the functionality and compares it with the functional requirement document. Integration Testing includes testing the requirements along with a few technical aspects. Let us focus on Non-Functional Testing, in the next screen.
2.41 Non-Functional Testing—Introduction
Non-functional requirements are often referred as qualities of a system. They include all the behavioral requirements not defined in functional requirements such as performance, and stability of the application. These requirements are not directly related to the business functionally, however, are essential to maintain the system. Non-functional testing can be performed at all levels. For example, the functional requirement of delete employee’s record deletes it and should no longer be viewable from the interface. Non-functional requirement of the delete functionality might include a soft delete as the data, though the employee is no longer associated with the company. For example, all employee’s PF numbers needs to be maintained. If a record is soft deleted and a user tries to add it again, a message is displayed informing that the record already exists. However, the record cannot be viewed in the interface as the status is changed in the backend. Let us look at the types of non-functional testing, in the next screen.
2.42 Non-Functional Testing—Types
There are different types of non-functional testing like Baseline testing, Compatibility testing, Compliance testing, Documentation testing, Endurance testing, Load testing, Localization and Internationalization testing, Performance testing, Recovery testing, Usability testing, Scalability testing, and Stress testing. Non-Functional testing depends on the needs of the specific organization and the customer. The names of many non-functional tests are often used interchangeably due to the overlap in scope between various non-functional requirements. For example, software performance is a broad term that includes many specific requirements like reliability and scalability. Let us discuss the common testing types, in the next screen.
2.43 Common Non-Functional Testing Types
Following are the common types of non-functional testing: Performance Testing measures the performance of the system in terms of response time, ability to handle the number of users and so on. Load Testing tests the amount of load a system is able to handle without hampering the response time. Stress Testing involves testing the system behavior beyond normal operational conditions. Usability Testing tests the user friendliness of the system. Security Testing tests how secure the system is to ensure the system cannot be hacked or user information is not leaked. In the following screen, let us discuss the test types with an example.
2.44 Test Types—Example
In an online banking software, the functional testing includes testing whether the funds are transferred accurately, if interest calculations are correct, and if bill payments are done on time. Non-functional testing focus on the security of the system, to ensure all access is safe. It will ensure that the system can handle the load especially during peak time like the start of the month when there are salary disbursements and payments. Let us look at structural testing, in the next screen.
2.45 Structural Testing
Structural testing is also referred as White Box testing, where the testers are required to know the internal implementations of the code. During structural testing, the focus is to test how the code in the software is functioning. For example, a structural technique would check how the loops in the software are working. Different test cases may be derived to exercise each loop once, twice, thrice and so on. This may be done regardless of the functionality of the software. Structural testing is a phased process, which requires work on test data, followed by simulation of the test environment, execution of the prepared cases, verification of code coverage, and the final evaluation of test results, leading to the structural test report. An extensive structure based examination; studies the decisions made by the program, design logic, and source code. Structural testing can be performed at all levels. Tools can also be used to measure the code coverage. Developers use structural testing in component testing and component integration testing. Structural testing is also used in system and acceptance testing, however, the structures are different. For example, the coverage of menu options or major business transactions could be the structural element in system or acceptance testing. The following screen focuses on re-testing and regression testing.
2.46 Re-Testing and Regression Testing
Re-testing is repeatedly testing the same functionality with different sets of input data. Whenever a tester finds a defect and a developer fixes it, the tester needs to retest the functionality. Retesting process can also be termed as ‘Confirmation’ which means that the defect has been resolved. Regression Testing is re-executing the same test case whenever the code is modified or some new functionality is added to it, to ensure no new defects have been introduced. Extent of regression testing is determined based on the risks of the system. Test cases selected for regression testing should be repeatable. In most cases, regression testing is conducted using automated tests, which will enable testers to increase test coverage during regression testing. Both re-testing and regression testing can be performed at all levels and can include both functional and non-functional testing objectives. In most of the cases, Regression testing needs more effort than retesting and retesting is almost followed by regression testing. Let us discuss an example of Re-Testing and Regression Testing, in the next screen.
2.47 Re-Testing and Regression Testing—Example
There are 1000 test cases to be executed in the first build and after executing 100 test cases, 90 pass and 10 fail. After the bugs are fixed and when the second build is released for testing, the tester first conducts re-testing. They test the 10 test cases that had failed earlier. Once the tests have passed, the tester conducts regression testing which executes all the previously passed 90 test cases or few selected ones from the 90 based on calculated risk. This is done to ensure that while fixing the 10 failed tests, any changes made to the system did not impact the already running functionality. In the next screen we will begin with the fourth topic, ‘Maintenance Testing.’
2.48 Maintenance Testing
This topic deals with Maintenance testing. Let us start with the concepts of Software Maintenance, in the next screen.
2.49 Software Maintenance
Software maintenance is a modification of software product after delivery, to correct faults, and improve performance or any other attributes. It is a common perception of maintenance that it merely involves fixing defects. However, a study by Pigosky in 1997 indicated that, over 80% of the maintenance effort is used for non-corrective actions. This perception is propagated by users submitting problem reports that in reality are functionality enhancements to the system. Ensuring any such maintenance activities are carried out as expected is called maintenance testing. This should be carried out before deploying fixes or enhancements to the system into production. In the next screen, we will discuss the importance of maintenance testing.
2.50 Maintenance Testing
Whenever user needs some new features in the existing software which requires modifications to be done, it needs to be tested for functional and non-functional aspects. Regression testing should also be carried out to ensure the existing features are working as expected. Maintenance testing is also required when the software is switched to another latest hardware platform or the environment is changed to a new operating system version and Database version. In this context, maintenance testing involves testing the whole application on new platforms and environment. Most of the systems do not have solitary existence, rather, they interact with many products, systems, networks, and interfaces. Retirement of any link in this chain will again necessitate the need for maintenance testing the products to ensure it still works as expected. Sometimes, this may impose some changes in the interface. So, functional and non-functional testing need to be performed as part of maintenance testing. Let us discuss the types of maintenance testing, in the next screen.
2.51 Maintenance Testing—Types
Different types of maintenance testing exist based on the objectives. These can be preventive, corrective, perfective, and adaptive. When making changes to the existing system, maintenance testing should be performed on the software, to reduce the risk of failure while operating and this is termed as Preventive Maintenance Testing. Sometimes, software may behave unpredictably due to unknown issues. Once these issues are resolved, maintenance testing is required to ensure no new issues have been introduced. This form of testing is called Corrective Testing. Enhancements and testing the enhancements to improve the safety, reliability, efficiency, or cost-effectiveness of operation is termed as Perfective Maintenance Testing. There are many times when you have to adapt the system to address requirements that arises due to changes in the environment or new regulations. Testing environmental or new regulation changes and ensuring they meet the new requirements is adaptation maintenance testing. Let us understand how maintenance testing is different from new application testing, in the next screen.
2.52 Maintenance vs. New Application Testing
Maintenance testing is done after the deployment of software while new application testing is done before. Maintenance testing mainly involves regression testing while testing new application involves all levels of tests including component, integration, system, and acceptance testing. In maintenance testing, number of defects are relatively less while testing new application identifies more defects. Maintenance testing involves huge coverage while testing new application is limited based on objective of testing. The focus of maintenance testing is mainly on functional testing while testing new application can include all test levels and test types. Maintenance testing is done to identify new business advantages whereas new application testing is done to verify whether software is satisfying the business requirements. Maintenance includes Black Box testing whereas new application testing can include all types of testing such as Black Box and White Box techniques. With this, we have reached the end of the lesson. Let us now check your understanding of the topics covered in this lesson.
A few questions will be presented in the following screens. Select the correct option and click Submit to see the feedback.
Here is a quick recap of what we have learned in this lesson: Software development process is divided into several logical stages which allows the company to organize its work efficiently to build a product of the required functionality within a specific time frame and budget. The different levels of testing include component, integration, system, and acceptance testing. The four common types of testing are Functional, Non-Functional, Structural, and Re-testing and Regression. Maintenance testing is done to identify new business advantages whereas new application testing is done to verify whether software satisfies the business requirements.
With this, we have come to the end of the lesson ‘Testing throughout the Software Life Cycle.’ The next lesson is ‘Static Techniques.’
About the On-Demand Webinar
About the Webinar