Does stubbing make your tests brittle?

Stubbing and mocking in unit tests is generally considered a Good Thing. It can make your tests run faster, it helps you not to cross boundaries. But it also makes your tests more brittle.

The reason for this is simple. If you want to stub out some method, you have to do it very precisely. You have to know the underlying implementation and choose a very concrete instance and method for stubbing. This makes your test very tightly coupled to the implementation details of tested class. And tightly coupled is brittle. Let’s look at some trivial example:

class LolCat < ActiveRecord::Base 
  def can_has_cheezburger? 
    0 == LolCat.count( 
           :conditions => [ 
             "id = ? and date(last_cheezburger_date) = ?", 
             self.id, Date.today]) 
  end 
end

This silly simple method checks if LolCat can has its cheezburger: only if it didn’t have one today. How do we test it? For example:

class LolCatTest < Test::Unit::TestCase 
  def test_can_has_cheezburger 
    LolCat.stubs(:count).returns(0) 
    assert @lolcat.can_has_cheezburger? 
  end  

  def test_can_has_no_cheezburger 
    LolCat.stubs(:count).returns(1) 
    assert !@lolcat.can_has_cheezburger? 
  end 
end

This is simple enough. We don’t want to touch the database, so we stub out the count method. Our test should run faster and it doesn’t cross boundaries. And we don’t have to setup any fixtures for the two cases tested thanks to this stubbing. All is good.

Until someone encounters our can_has_cheezburger? method and decides “we don’t need count, we should have used exists? instead”. They proceed with the refactoring, run the tests and discover the failure of our brittle tests.

How to prevent this? I don’t have any good solution. I just try to find a balance. Use stubbing and mocks when it makes sense. When the coupling induced by stubbing pays for itself. In the above example, we avoided setting up a LolCat with and without a cheezburger and touching the database. Was it worth it? Probably.