A failure to do bva

Boundary value Analysis ...

... is a simple test technique.  It is taught on all introductory software testing courses.  The theory is that you split input data into sets of valid and invalid input and then test at the boundary between valid and invalid data.  Easy example, a function accepts an integer between 5 and 10.  Now ignoring the invalid sets as they approach the upper and lower limits of the integer data types you would use the following tests:

• lower bound
• 4 - invalid
• 5 - valid
• 6 - valid
• upper bound
• 9 - valid
• 10 - valid
• 11 - invalid

You obviously also test values in the middle of each set such as 3 or 7.  Assuming that the implementation of the function is approximate to the spec then this should be a reasonable set of tests to run.  Lets try a real world example:

A credit card company has a system that generates card security check (CSC) number (the last 3 numbers on the back of your card) and a system that checks that during a card not present transaction (like when you buy something online) the CSC is valid.  A CSC has the following properties:

• 3 digits long
• has a min value of 001
• has a max value of 999

The test case of a CSC having the value of 000, is that valid or invalid? Not sure? well a test case you should try.

I recently got a new credit card to replace my older ones which were due to have a interest rate rise. The cards arrive and one has a CSC of 000.  I think no more about it apart from, wow that's going to be easy to remember.  Tonight my wife needed to make a purchase online.  Since it was for work I thought we would buy it on the credit card so that when she was reimbursed the money could be applied direct to the credit card.

To my concern the transaction was declined.  I checked online that there was ample credit for the purchase.  There was.  I called the company and asked why the card was declined.  Sanjay (the call taker) advised me that I had mistyped the card security number.  I mentioned that there was NO chance of that since it was so easy to remember (000).  He put me on hold.

Yes Mr Yates.  There is a problem.  The CSC 000 is considered invalid by our system.  As a security precaution we have cancelled all of your cards.  We are sending you new ones in the post.

Excuse me! I replied.  Why is that number considered invalid when it was one of your systems that generated it and printed it onto a card?  Surely this is a boundary value that would have been tested?

Sanjay was very apologetic and credited me £25 in way of an apology (so that's the purchase paid for) and allowed me to use my wifes card to complete the purchase before he canceled all of the cards to process the request for new cards to be issued.

We all make mistakes but there is a compound failure here.  Should the value 000 being fed into the card processing system really cause all the cards associated with an account to be blocked?  I'm not saying who the company is but I wonder if someone else tried a card not present transaction using a CSC of 000 would all their cards be blocked as well?  I'm sure it was never tested as the CSC generation system should never have issued a card with a CSC of 000

So what have we got:

• 2 systems that have the same spec of what should be valid and invalid, however different implementations.  One system considers the edge case 000 valid and the other invalid. 
• A system that doesn't recover from a card not present transaction having a CSC of 000.  Instead defaulting to the 'safest' behavior of blocking all cards associated with the card 
• Potential opening for a test consultant? 

So I am bit annoyed and inconvenienced and I acknowledge that the chance of a card being issued with a CSC is  1/1000 but if a simple test case has been written this wouldn't have been an issue.

Also means I now have a great example when teaching boundary value analysis.