> I find it funny that in most discussions around mocking, a database is the given example. 100% of the time when I need a database for "unit tests", I use a database. Not a mock. I don't want to know that when I do select blar from flar I get back the value 7, and hardcode it. I want to actually load the data and see it happen. Because if I don't, how do I know that my code will actually work in production?
That is not a unit test. What if your database is corrupted? What if your mock Netflix API is buggy? The second you do anything like that you're no longer unit testing. The whole point is you're testing a single method/class in isolation -- Mockito gives you the power to assume external structures (the database) work as expected.
If you don't like unit testing, sure. But unit testing is what Mockito is all about.
What if my DB is corrupted - Why did it get corrupted? Sounds like you have a good failure to chase down. It would be even worse if your prod DB got corrupt!
For most use cases, a given piece needs 1 test around it. "The test". If you can test your logic + view + db all in 1 test, you get a pretty huge efficiency gain.. over sitting there writing "unit tests" where you mock what a select * from a database returns. Life is too short to do that.
Having just a single set of tests has a lot of draw backs compared to splitting up your tests.
Consider this, we have a simple CRUD app that uses a datastore.
Since you want everyone to be able to run the test suite locally, setting up a local datastore manually isn't an option. You need a script/function that will setup the database.
Because you want your tests to be as deterministic as possible, and you don't want the order that tests are executed in to change the results, you need to setup the database before every test, and then clean it up again after the test.
However, setting up the datastore and cleaning it up again can take significant amount of time. Even if it only takes 5 seconds, when you multiply it out for each test your test suite can now take 100x longer to run.
This means you have now limited yourself to being able to run tests after you have completed large changes, and not incrementally.
Also when a test breaks, you now need to determine if it is broken because your DB isn't configured correctly, or because something is wrong with your code.
You can help mitigate these problems by creating 2 sets of tests. Unit tests and integration test. Unit tests use mocks, and run quickly. Integration tests will create a environment that mimics what you have in prod, and will take a while to complete.
I think you also overestimate how long it takes to write mock tests.
Hopefully this provides some insight into why I think unit + integration tests are the way to go.
> Since you want everyone to be able to run the test suite locally, setting up a local datastore manually isn't an option. You need a script/function that will setup the database.
Since we want our environment to be reproducible and stored in git, this is a requirement anyway. If your schema is not versioned in Git, and cannot automatically be applied in all environments, including prod, you are doing it wrong.
> Because you want your tests to be as deterministic as possible, and you don't want the order that tests are executed in to change the results, you need to setup the database before every test, and then clean it up again after the test.
No not really. Lots of ways to solve this. For example DJango will run a test in a transaction, ensure it passes, then roll back the transaction. Order thus does not matter, the DB is cleaned up for free.
> However, setting up the datastore and cleaning it up again can take significant amount of time. Even if it only takes 5 seconds, when you multiply it out for each test your test suite can now take 100x longer to run.
This means you have now limited yourself to being able to run tests after you have completed large changes, and not incrementally.
So the old wisdom on this is that unit tests should be able to run in 5 minutes or less. Just a silly Django crud app with a parallel test runner, you can do something like 50k-100k tests in 5 minutes (Most tests take about 1/10th of a second, with several going on in parallel). Past the 50-100k test mark it gets harder to do locally, but is trivial to do in CI (for example, there are several CI as a service services that split your tests between containers, so you can run any number of parallel containers).
> Also when a test breaks, you now need to determine if it is broken because your DB isn't configured correctly, or because something is wrong with your code.
Because database state and schema is stored in git, and applied programmatically, we already ruled out the DB. So if a test breaks, it is the code.
> You can help mitigate these problems by creating 2 sets of tests. Unit tests and integration test. Unit tests use mocks, and run quickly. Integration tests will create a environment that mimics what you have in prod, and will take a while to complete.
To me, you had a problem (Database not consistent), and rather solve it in a sane way (store schema in source control, apply automatically) - You solved it in a very roundabout way (worry about which class of a tests a given test is in, write two types of tests, still fight bad schema in prod issues because you didn't solve the root problem (schema not in source control) )
I'm just pointing out that your problem is with unit tests, not with Mockito. I think unit tests are invaluable on large projects, so I find Mockito to be an incredibly valuable tool. I'm also not arguing against integration testing, just that unit testing is not the time to worry about your prod environment.
Why should your tools having to spin up a database in 3 seconds conflate local dev testing and your prod environment? Not advocating a 8 database cluster with hot failover on your laptop.. just a simple database.
Would all those scenarios really be a part of a "unit" test? I generally approach unit tests to ensure that given pre-conditions of the "external structures" you have a defined set of post-conditions that the test can exercise.
The cases you've mentioned - I'd most likely leave up at integration & failure injection testing. Still handled by a CI/CD infrastructure but done at a level where the availability of an entire environment (DBs, multiple nodes, h/w, etc.) can be expected.
That is not a unit test. What if your database is corrupted? What if your mock Netflix API is buggy? The second you do anything like that you're no longer unit testing. The whole point is you're testing a single method/class in isolation -- Mockito gives you the power to assume external structures (the database) work as expected.
If you don't like unit testing, sure. But unit testing is what Mockito is all about.