Executive Overview of Software Testing
Many mistakenly believe that software testing is a small extension of development, and worse, some assume that testing is performed exclusively by software engineers. Others assume that software testing is something that happens at the end of the development process - after the software is developed; it is then tested and shipped. Others equate quality assurance and software testing. All of these beliefs are incorrect.
Software testing is a highly strategic and specialized discipline. It is very different from, but related to, software development (just as sales and marketing are very different but related disciplines). At its most basic, software testing is a concerted attempt to break software so that bugs may be identified and fixed before end-users encounter them.
Testing uses skills, methodologies, insights, and creativity that are very different from those used by software developers.
Software testing is a critically important part of the overall development process.
Because of its strategic importance software testing should be its own organization, separate and distinct from development, with its own budget. This setup can help to enable:
• Effective training
• Continuous skills and career development
• Efficient implementation of testing tools
• More effective communication from the testing team to development and management
The software testing process has several main activities. These include:
• Designing the software tests
• Running the tests
• Identifying problems and defects
• Reporting to management and development on key metrics
Testing is an iterative process of identifying, fixing, and retesting, as well as reacting to design and code changes. Software testing consists of both manual testing and automated testing. While it is true that manual testing should be kept to a minimum; there will always be tasks, such as usability testing, that require manual testing.
Manual Testing: Productivity Objective
No more than 5% of tests should be executed manually.
Software testing is not quality assurance; it is a part of quality assurance. All steps in the development process, from requirements definition, to design and development, share in quality as an objective. The role of software testing in the process is to identify defects so that they may be fixed. You cannot, however, test quality into a poor design (as the saying goes, “bugs may be tested out, but quality must be built in”).
Software testing is also not something that simply happens at the end of the development process. To be effective, software testing needs to be a strategic partner with product marketing and development from the beginning of the development process. Software testing needs to have a clear and full understanding of the goals and objectives of the software under design and development. Having such knowledge will help them to design better tests. Early visibility also helps software testing to develop their testing plans, as well as start to design test cases. It helps to decrease testing time and costs.
In addition to being an activity, software testing generates products that can be viewed as strategic assets to an organization. These products include:
• Test cases that can consolidate the intellectual property of your team members
• Automated tests that can be re-used, becoming assets that reduce costs
Testing metrics provide visibility
Testing metrics must provide visibility into a software product’s quality. Metrics are only useful if they help to make sound business decisions.
Metrics fall into two categories
1. Project management.
2. Process improvement
Some metrics can measure the output of your test team such as
1. Bug reports
2. Test cases written or test cases executed
3. Other metrics, such as coverage metrics, give visibility into how much of the software has been tested.
Executive Overview of Test Automation
Test automation can provide great benefits to the software testing process and improve the quality of the results. The reasons to automate software testing lie in the pitfalls of manual software testing. Manual testing:
• is slow and costly
• does not scale well
• is not consistent and repeatable
• is difficult to manage
While these factors may drive the desire and need to automate testing, it is important to take the right approach to test automation. There are several basic steps to automating testing:
• Use the Action Based Testing (ABT) methodology, a keyword-based, object oriented approach that provides for visibility, reusability, scalability, and maintainability. Visibility, reusability, scalability, and maintainability translate into speed and cost savings.
• Choose the right enabling technologies that support the methodology. The tools need to support extensibility and a team-based global test automation framework, with a solid management and communication platform.
• Put in place the right people with the proper skills and training in methodology, tools, and domain knowledge (knowledge of the software to be tested, the industry for which the software is intended, and end-user expectations).
• Separate test design from test automation so that automation does not dominate test design. Action Based Testing (ABT) creates a hierarchical test development model that allows test engineers (domain experts who may not be skilled in coding) to focus on developing executable tests based on action keywords, while automation engineers (highly skilled technically but who may not be good at developing effective tests) focus on developing the low-level scripts that implement the keyword-based actions used by the test experts. ABT allows an organization to spend more time developing tests and less time actually coding the test cases. This speeds up the whole testing process and helps to reduce costs.
• Lower costs by using less expensive labor than your local team.
Automated testing : productivity objective
No more than 5% of the effort surrounding testing should be expended in automating the tests.
Jumpstart the process with a pre-trained outsourcing partner that knows more about test automation success than you do, and that has a competent, well-trained staff of software testers, automation engineers, test engineers, test leads and project managers.
The most essential element is methodology. The methodology is the foundation upon which everything else rests. The methodology drives tool selection and the rest of the automation process. The methodology also helps to drive the approach to off shoring the “appropriate” pieces of the testing process.
Executive Overview of Offshore Outsourcing
Most companies are convinced of the need to offshore some or all of software testing.
Off shoring offers the promise of significant cost savings. However, off shoring is more than simply moving existing software testing efforts to an offshore outsource partner.
Executives making the decision to offshore testing must understand the possible pitfalls of outsourcing and off shoring, and must plan effective strategies to combat these pitfalls.
Executives also must stay focused on the fact that an effective off shored testing effort is based on the strategic integration of methodology, the latest technologies, and an effective global resource strategy that supports the methodology and tools.
Some of the major pitfalls of off shoring include:
• Problematic communications due to language and cultural barriers, mismatched or miscommunication of expectations, poor metrics selection, and unresponsiveness.
• Insufficient or mismatched skill sets in the software testing organization, such as using entry-level development engineers as software testers, lack of knowledge of the software being tested, and lack of domain knowledge (knowledge in the category of the software being tested).
• Management issues due to the lack of a workable test management process and associated methodology.
• Vendor problems or vendor infrastructure problems such as poor data bandwidth.
• General off shoring risks such as security and protection of intellectual capital.
Some suggestions for dealing with these pitfalls include:
• Finding a trusted partner or building trust in a partner. You need to work with a partner that you know has testing experience, an experienced staff, an understanding of current methodologies, and competent domain knowledge.
Train the test organization. Provide them with knowledge of your application, your expectations, your communication/management platforms and expected domain expertise. Discuss how to recognize and deal with cultural issues.
• Adopt a methodology and tools that support the overall methodology to improve testing, defect tracking, automation, and communications management, focusing on excellent and correct methods, ease of distributed team communication, accessibility, and useful measures.
• Choose carefully what work goes offshore and what remains “at home”. Often it makes sense to keep user-focused scenario development and business process testing at home where you have more knowledge of the domain and the user.
• Get someone local to manage the off shored test effort. A local lead that is part of your team, who understands the culture and communication nuances of the offshore team, can lead the project, effectively communicate progress and metrics, and help to streamline the process.
The most effective way to avoid the pitfalls of test automation and off shoring, and to realize the full time and cost saving benefits of test automation, is to implement a strategy of Global Test Automation.
Global Test Automation strategically integrates the latest test automation methodologies based on Action Based Testing, the latest testing technologies, with an effective and balanced global resource strategy that makes use of both onshore and less expensive offshore resources, on- and off-shore leads, and effective team-based management tools and methodologies.
The benefits of such an approach are scalability, reusability, visibility, and maintainability that ultimately will allow an organization to achieve both time and cost savings while delivering a quality software product. The bottom line is a higher quality product that is delivered faster and more cost effectively.
As you can see in Figure 3 below, Global Test Automation is the strategic integration of:
• Speed achieved through an ABT test methodology and test automation technologies for distributed teams
• Cost control achieved by using a cost structure based on worldwide resources
Choosing a Global Test Automation Partner
Selecting the right testing partner to implement this strategy is more than simply selecting a vendor to develop and run test cases. It entails selecting a partner who fosters innovation to improve test productivity, a partner who understands Global Test Automation, a partner who understands that the methodology drives the use of appropriate tools, a partner who has an effective management and communication strategy, and a partner who makes use of global resources to provide the appropriate mix of on and offshore resources to drive down costs.
The process of test automation can be jump started with the selection of a pre trained strategic partner that knows more about test automation success than you do, and that has a competent, well-trained staff of software testers, test leads, and an in-place global resourcing strategy.
No comments:
Post a Comment