John Overbaugh
Q- Can you explain how you can design test cases using the Cause-Effect Graphing Technique?
A- The Cause-Effect Testing Technique is another of several efforts for mapping input to output/response. In the Cause-Effect Graphing Technique, input and output are modeled as simple text, such as this:
Cause --> intermediate mode --> effect
See the Wikipedia article "Cause-effect graph" for additional information.
Much more information about the Cause-Effect Graphing Technique can be found in the "Cause-Effect Graphing User Guide", which is an entire PDF for the use of Bender RBT testing tool. (Note that this manual was written in 2006, so it might be somewhat out of date.)
The goal of cause-effect graphing is to reduce the number of test cases run. Many input/modifiers/effect combinations have the same output and exercise the same code in the system under test. By reducing the combinations, fewer tests can be run while still achieving the same confidence in quality.
A somewhat related modeling concept is PICT -- Pairwise Interdependent Combinatorial Testing. This technique goes a step beyond cause-effect and actually applies combinatorial theory to reduce the number of combinations needed to be tested.
Each theory has its application. In a highly related system where cause and intermediate effect are tightly coupled, it might be a challenge to model the system in PICT (although some PICT tools, including Microsoft's free PICT modeling tool, do allow you to weight various factors as well as create combinations).
My concern about cause-effect is simple: How could you model cause-effect on a high-availability Web service with 30 methods, each containing 40 or 50 parameters? By manually identifying each cause and effect, you'll spend so much time detailing the test that I'm not convinced you could actually test the application. This is why tools like Bender RBT exist -- to help automate this process.
My biggest complaint about a methodology like cause-effect is its lack of real-world application. I doubt your manager would tolerate your spending five or six days modeling your application into a cause-effect graph. Your manager is more likely to expect you to apply good testing sense and develop cases quickly, but keeping an eye on critical combinations. Still, further research is warranted, and if you can efficiently apply this model to your efforts, you'll be rewarded with higher quality at a reduced input cost.
1 comment:
I want to provide input from someone who _does_ have "real-world experience" with cause-effect graphing (CEG) so as to address some of the concerns raised here.
I will just address three at present:
1. To say that PICT "goes beyond" cause-effect graphing prompts me to ask the question "How?" PICT starts with a description of the independent inputs and constraints for those inputs without any information about what one should expect in response (no "oracle").
As such, the test cases generated are incomplete.
Simple to generate - yes.
Simple to set up for generation - yup.
Effective test cases -- who knows? They simply represent combination for two inputs without any real knowledge what that means to the application in terms of requirements or code covered.
CEG models the logical combination of inputs to the outputs. Generated test cases contain a complete specification for the test in those input and output terms; missing only those details that are not suitable to describe in the CEG model. Further, the model can be analyzed to see where two or more errors in programming may interfere to produce what looks like a correct answer to the tester - this analysis leads us to seek additional output information in the test so as to be sure that we got the correct looking answer for the right reason.
2. In real-world applications the input space can be _quite large_. Here is how I tackle the "high-availability Web service with 30 methods, each containing 40 or 50 parameters" -- I model each method independently in its own CEG -- representing all of its inputs including the State for the object where necessary (the State becomes a valuable _cause_ in object oriented and event driven models). Where possible, I simplify the CEG model by recognizing the Equivalence class an representing those input with a single input node. When the test cases have been generated, I may string them together to form larger test case descriptions across or within the same model space.
CEG modeling is like programming -- there are many equivalent models and there are poor and better approaches to such attributes as achieve readability, maintainability, and be in sync with the requirements specification.
3.While an approach like PICT will generate test cases, it is unlikely to lead the test analyst to uncover either the broad set of missing requirements or the requirements that are logically inconsistent with each other. CEG does provide those outcomes.
CEG is not a simple approach and I would not be attempt it for real-world applications without the support of a tool like BenderRBT. Once mastered however, it is like a swordsman using a Roman short sword versus the best Japanese katana -- a tool that surpasses and utterly defeats the testing analysis problem. I mastered most of CEG working under the guidance of Dick Bender and others trained by him in a matter of a few projects. I have coached others in a similar manner
Post a Comment