Unit tests can create a false sense of security

bsoremsugar —  June 23, 2011 — 2 Comments

At SugarCRM we use PHPUnit as our unit testing framework. I like to think we apply it pretty rigorously in terms of always pushing changes with unit tests attached where appropriate. One artifact of this is that unit tests can add a false sense of security.

Here is a real world example. I wrote some code and added a unit test around that functionality.  The problem came from the fact that my unit test didn’t cover both the positive and the negative cases. In other words it was a bad unit test.  My point here is less about that fact, although I take that very seriously. It is more about the fact that even though you have a unit test it doesn’t mean you covered. You need a GOOD unit test to help in that search.

2 responses to Unit tests can create a false sense of security

  1. 

    Very true.  Your post made me think of this article by Steve Sanderson on how to write a good unit test.
    http://blog.stevensanderson.com/2009/08/24/writing-great-unit-tests-best-and-worst-practises/

  2. 

    Roger,
    You are definitely right.

    I did unit testing on C# code, and now I’m doing it for SugarCRM’s project on gitHub….
    Many times we think of testing as a waste of time. Actually this is a bad approach, just because testing can really help You avoiding small or big mistakes, especially when software grows up….
    Build clever tests is really difficult, and only experience can help us…. When You say “Oh f#@k! I broke it”, You understand what Your testing was missing…. At the beginning You understand it too late, but experience will help all of us to improve checks.
    BTW, whenever You write a piece of code, You know You can introduce a bug.
    And whenever You write a unit test, You should know that You could miss a little part of coverage (exactly that one that user Murphy would try immediately…)

    Ciao – FabioG

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s