This weekend I wrote a set of classes to [Determine and police password strength]. The interesting goal I set my self was to ensure that I got 100% code coverage using the MS unit test framework. Sounds easy! How hard can that be? Honestly I’ve focused more on TDD tests rather than code coverage.

Well firstly I have to say its not that easy , secondarily it really shows the true benefit of [TDD] – more on that later. So on to the discoveries!

Being a TDD advocate I started off with an interface and then the tests before the code. So far so good – then on to the code. As a Spongebob episode would say “Several hours later” we were done. So crank up the tests and we are looking golden.

Unfortunately the code coverage wasn’t half as good as I had hoped for. So next couple of hours was spent updating the tests for coverage, and a number of edge cases surrounding the throwing of exceptions for invalid parameters. Interestingly a couple of areas that showed up outside of the TDD approach were certain areas in “for loops” where not executed because my test case was really unknowingly best case. The part that was revealing to me was that the TDD is really a “business” driven approach, code coverage is believed to be a programmers indicator of  ‘quality’. So while TDD (at least the first pass I end up doing) meets the business needs it is very hard to get 100% coverage from a business scenario.

Finally I got it up to 99.84% coverage. This last standout was VERY annoying, specifically looking at the code coverage everything was green – no red or yellow sections. Hrrrmm. Ok it was a simple function that has this 1 block of unexecuted code – a switch statement taking a enum in as a parameter.  So lets have a look have a look (note I’ve stripped a lot of the content and comments out for readability:

   public enum PasswordStrengthIndex
   {
      None = 0,
      Weak = 1,
      Medium = 2,
      Strong = 3,
      MostStrong = 4,
   }

      public void ResetToDefinedPolicyStrength(PasswordStrengthIndex strength)
      {
         switch (strength)
         {
            case PasswordStrengthIndex.None:
               MinimumPasswordLength = 0;
               break;
            case PasswordStrengthIndex.Weak:
               MinimumPasswordLength = 4;
               break;
            case PasswordStrengthIndex.Medium:
               MinimumPasswordLength = 6;
               break;
            case PasswordStrengthIndex.Strong:
               MinimumPasswordLength = 8;
               break;
            case PasswordStrengthIndex.MostStrong:
               MinimumPasswordLength = 12;
               break;
         }
         return;
      }

The test case iterated through the enums and verified the output matched the expectations. Still I had this annoying block that was unexecuted. The next step was the Jimmy Neutron “Think Think Think”, a-ha! No default handler – so add one. Hmmm I cant add one that actually can be exercised as I’ve already used all my values in my enum. Well adding a default handler to anyone of those case statements still didnt resolve the issue. Now it was getting personal! I was dangerously close to calling in technical gurus and venting my disgust! So before making the calls time to search Google for “Code Coverage Switch” – and boy Google never ceases to amaze me.

Firstly it confirmed my suspicion that is was indeed the switch statement, second was a link that explained it in detail. Basically the default handler was not getting called – and there were a number of ‘breaking’ workarounds (change the enum to have another element to be used in the default case) in the Google answers, but one one that I ‘liked’ the most was to force the cast (which I would link out to the original author out of respect but I cant find it now ).

      public void ResetToDefinedPolicyStrength(PasswordStrengthIndex strength)
      {
         switch (strength)
         {
            case PasswordStrengthIndex.None:
               MinimumPasswordLength = 0;
               break;
            case PasswordStrengthIndex.Weak:
               MinimumPasswordLength = 4;
               break;
            case PasswordStrengthIndex.Medium:
               MinimumPasswordLength = 6;
               break;
            case PasswordStrengthIndex.Strong:
               MinimumPasswordLength = 8;
               break;
            case PasswordStrengthIndex.MostStrong:
               MinimumPasswordLength = 12;
               break;
            default:
               throw new ArgumentException("Supplied strength is not recognized as enum", "strength");
         }
         return;
      }

      [TestMethod()]
      [ExpectedException(typeof(ArgumentException))]
      public void ResetToDefinedPolicyStrengthBogus()
      {
         PasswordPolicy policy = new PasswordPolicy();
         policy.ResetToDefinedPolicyStrength(<strong>(PasswordStrengthIndex)666</strong>);
      }

Look at the last line of the listing above. Use a cast to force the enum to be a number out side of the range it was looking for – ta da!  On a partially related note this reminds me of the fact Enums are definitely interesting beasts that can bite you if you have compiled against one then change its definition in a different assembly.

So now we have got to 100% code coverage – woo hoo! The realization (which is obvious and well know to many) is that without TDD you would most likely end up writing tests that cover your code – but miss the business requirements. So any programmers out there who think they have good code coverage but are not using TDD are likely to have a false believe in their code because the tool says 100% green (unless they are coding guru’s!) – they are only really testing what they have written and not the business requirements. On the flipside just writing TDD tests without coverage is highly unlikely to meet all the ‘code’ edge cases (different than ‘business’ edge cases). So the conclusion it that the blend of the two are the best, but yet still not perfect. You have to start with TDD, and then move to code coverage – business first, code second.

I’m hoping at some point management teams will start to understand that even with 100% executed code coverage there can still be bugs as its a data world we live in! Obviously I’m an optimist!

Gareth

This entry was posted on Monday, September 14th, 2009 at 10:15 pm and is filed under Development. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.