On Mocking

Tuesday January 26, 2021

I'm generally aware that there is a debate about mocking in tests, but honestly I have not thoroughly researched the arguments. Here I will consider "both sides" of the issue from my naïve standpoint as a thought exercise to improve my own understanding of testing and mocking.

For the purposes of this (hopefully short) article, I'm going to ignore the differences between mocks, fakes, stubs, etc. The differences are certainly important, but ultimately I just don't want this to devolve into a semantic argument.

The Case Against Mocking

The basic case against mocking is simple: mocks are not your code. Ergo if you mock in test, you're not testing your code.

Here are some more specific criticisms I can come up with.

Mocking is a code (test?) smell

Needing to mock is an indication you need to refactor your code and/or tests:

  • Your tests are too big: test smaller pieces of code.
  • If you can't make your tests smaller, your code is too tightly coupled: separate the concerns.

Simple! Ultimately good code should be relatively easilty testable. Of course there are exceptions to this, but we are in the business of writing code, not tests! Let's ignore for now that tests are code.

Mocking is dangerous

Mocks make it easier for tests to get out of sync with your code, such that they're useless or even dangerous. Bad tests can be worse than no tests at all by instilling false confidence, wasting build time, and hiding breaking changes.

Mocking is a burden

Testing is overhead, but it is meant to enhance developer productivity. As mentioned above, mocks can make it easier for tests to become stale or out of sync with your code. The more time you spend rewriting tests, the more overhead they become. Tests should be simple so that when a test fails, you know to fix the code not the test.

The Case For Mocking

We live in a real, flawed world. We write bad code. We write code without tests!

Of course we'd never actually do that, but we uh...inherit code without tests! (Sometimes from our younger, more foolish selves.)

I will concede that mocks seem inelegant, but that doesn't mean they're not useful. Just because we're not testing all our code doesn't mean we're not adding value. Likewise regarding the dangers - most useful tools are somewhat dangerous, but we learn how to use them and how to be careful.

Here are just some circumstances where I can envision using mocks (some from experience, some hypothetical):

Mock early in an untested project

Just as premature optimization is a mistake, premature refactoring is a mistake. The extreme version is the infamous rewrite. You can't and shouldn't always dive into refactoring, and some tests are generally better than none tests, so just get them in there even if you have to use a mock or two.

A huge benefit of testing is refactoring without fear. Getting a few tests in, even with mocks, will make it easier to start refactoring, ultimately getting you to that fully-tested mock-free Nirvana sooner.

Mock out parts that are already tested

How about mocking a third party library? Sure it's open source so theoretically you could contribute, but if it's a big well supported project just mock it and leave the testing to them.

Even in your own project, if you have some costly function with a strong API that is already tested - just mock that puppy! Sure, your test will stop testing properly if the API changes, but you should have a good idea when that happens. Maybe you can even do something fancy like automatically generate the mock with the same API (maybe that makes it a fake or stub or a crash test dummy or something).

Mock a big fixture

Say you want to test against a server, but you don't want to test against the actual live server. You could stand up your own version as a test fixture, but this comes with many of the same problems as mocking and then some. Save yourself the time and energy and Just Mock It™.

Mock some hardware

I work on software for a "device", but this applies to more situations than that. For example, you could mock some API calls into a GPU library if your test environment doesn't have one.