Testing: The pair-wise problem and fertile ground for improvement.
Apparently pair-wise testing is a pretty big thing. Hell, I'm not a tester, but I know I've used some of the same techniques writing my own tests. So I guess it was only a matter of time before I learned there was a name for what in the hell was going on. Because there are math problems in pair-wise testing it becomes something that I can really sink my teeth into. Most importantly, I'm really intrigued at the number of tools (fewer than I would expect), and that there is a single tool, usable over the web, that seems to be the master of all that others compare their own algorithms to. Here is the metric I've seen used:
20 features
10 options
High Combination = 235 (AllPairs), Low Combination = 180 (AETG)
Since the AETG guys apparently spearheaded the pattern, I'm guessing that gives them some advantage of time and resources into reducing the problem set... I think we should start with a smaller sample so we can visualize this pair testing... I think a 3 features, 3 options problem should be fine to start, and I like it because we can use a cube to show how it works, along with a tabular layout.
| Feature 1 | Feature 2 | Feature 3 |  |
| 0 | 0 | 0 |
| 0 | 1 | 1 |
| 0 | 2 | 2 |
| 1 | 0 | 1 |
| 1 | 1 | 2 |
| 1 | 2 | 0 |
| 2 | 0 | 2 |
| 2 | 1 | 0 |
| 2 | 2 | 1 |
If you are curious about the cube, it has been rotated counter-clockwise a few degrees. Each tier represents an option of Feature 3 (y-axis), Feature 2 is oriented on the (z-axis) and Feature 1 is oriented on the (x-axis)... You can see an interesting pattern arise in both the table and the image, so I've placed both.
When working with the test cases there is always going to be a simple minimum bounding. The minimum bounding may not be possible, but it gives you a good idea of something approchable. Take the two features with the most options and multiply them together, you'll need to implement at least that many tests (3*3=9) in our case. For an interesting study, it also takes 9 tests in order to solve a 4 feature problem with 3 options per problem. Must be some math in there somewhere? For all square arrays you always wind up with n*n minimum test cases. You can then encode one additional feature that doesn't rotate with the rest of the matrix, but must have the same dimension or number of options.
Well, like I said, lots of room for improvement. This is just an introduction. I'll cover some interesting code samples a bit later and maybe work out a competitive algorithm for those AETG guys. If you like testing stick around for a couple of days. If you are waiting for my binomial coefficient posting, I'm still working on some way to spatially describe the process. Tisn't easy, but I'll get to it.