Testing: Pair-wise, upper and lower boundings

You'd think there would be more math on this. Maybe I'm looking in the wrong places? Well, I wanted to try and formalize some boundings for sequences of features that have identical numbers of options.

f = number of features
n = number of options
n^2 = number of tests required to pair-wise match 2 or more features
Cn^2 = multiplier function for supporting increased numbers of features by adding tests

// Given a constant multipler give the maximum features supported
f = n+(C-1)n^2+1

// Given a number of features and their options, find out the max test boundings
max = ceil((f+1)/2n)n^2, f >=2
min = (ceil((f+1)/2n) - 1)n^2 + 1, f >= n^2

max = ceil((20+1)/2(10))10^2 = 2*100 = 200
min = (ceil((20+1)/2(10)) - 1)10^2 + 1 = 1*100 = 101

These methods are fairly fragile right now, definitely not the rigid generating functions I'd like to have. There is also the issue of large boundings, 101 to 200? What kind of bounding is that. Especially since we know that pairWise(20, 10) is somewhere around 180. pairWise(4, 2) is actually 5 while maxPair(4, 2) = 8 and minPair(4, 2) = 5... Naive attempts to limit pairs at 4, 2 would yield 6, but you can actually achieve 5 using a handy counting routine:

  • Name the features F1...F4... Remember that F1...F3 are already solved, so fix them in place.
  • Now, add F4 to existing line-items. As you add F4 features count each new pair generated with an existing feature.
    • Example: F1-4, F2-2, F3-4
    • Note: I actually ran through the possibilities so the above is a sample of pairWise(4,2) counting
  • At the end, you'll have what we call a deficit of pairings. F1 and F3 are both complete, they have 4 pairings, F2 doesn't.
  • For every 1 in deficit you have to make an extra combination.
    • F1-4, F2-2, F3-4: 2 new combinations need to be made to solve F2's deficit
    • F1-3, F2-3, F3-3: 1 new combination may solve all 3 deficits!

Now that we have a kind of counting routine, maybe it can be extended to geometrically produce optimum pairings. The core of a pairing is easy to generate and is necessary to procedurally generate the remaining combinations. Each core is fixed, while extra-core pairings can have a degree of freedom, similar to arrangements. You'd have to compute the various arrangements of a given test series if you plan on doing a best fit matching to existing tests to determine the minimal number of new tests that are going to be required to fulfill a pair-wise matrix or make recommendations on changing the focus of existing tests.

 

Published Tuesday, September 14, 2004 3:08 AM by Justin Rogers

Comments

Tuesday, September 14, 2004 8:40 AM by Jonathan de Halleux

# re: Testing: Pair-wise, upper and lower boundings

I would be interrested to see a concrete implementation of that algo and compare it with the other approach from TestFu. Are you planning to release any code ?
Tuesday, September 14, 2004 10:05 AM by Justin Rogers

# re: Testing: Pair-wise, upper and lower boundings

As I continue through examining the algorithms bit by by, I'll start to build up a code base... I already have code that generates optimized pair-wise combinations for any number of features that contain 2 options (aka, a trivial case).

I do plan on working through more complex cases, especially in the interest of finding boundings, which is of more interest to me than the generating functions. If you see my later post I've skirted around the concept of solution domains as well. I think there is definitely some value in optimizing existing test coverage, meaning I'll also cover them in more depth.

Leave a Comment

(required) 
(required) 
(optional)
(required)