Why “green tests” don’t mean your code actually works — and how to fix that before production does it for you
It was 2:07 AM when my phone buzzed.
“Prod is down. Employee onboarding is broken. Customers can’t add new hires.”
I sat up, opened my laptop, and stared at the dashboard like it owed me money. Stack trace after stack trace. A NullPointerException deep inside EmployeeServiceImpl, triggered by a repository call that, in my test suite, had never once returned null.
Here’s the part that still stings a little: my test suite was green. All 214 tests passing. A little badge on my GitHub repo proudly displaying “build: passing” like it had done its job.
It hadn’t.
That night I learned the difference between tests that make you feel safe and tests that actually keep you safe. And honestly, it’s a distinction most of us — myself very much included — get wrong for years before someone (or something) forces the lesson on us.
So let’s talk about it. Not the textbook version. The real one.
The Trap: “I Have Tests” vs. “I Have Good Tests”
Here’s a confession: for the first couple of years of writing Spring Boot applications, my definition of “well-tested” was embarrassingly simple. If a class had a corresponding test class sitting next to it, I considered the job done. Green checkmark, ship it, move on.
The problem is that Spring Boot makes it dangerously easy to write tests that look thorough but test almost nothing meaningful. You spin up @SpringBootTest, autowire half your application context, call a method, assert it didn’t throw, and pat yourself on the back. Congratulations — you’ve written a test that mostly verifies Spring can start.
That’s not a unit test. That’s barely an integration test. It’s a false sense of security wearing a lab coat.
To understand where things go wrong, we have to actually separate the two things most developers blur together: unit tests and integration tests. They are not the same tool, and using one where the other belongs is exactly how bugs like mine slip through a “fully tested” codebase.
Unit Testing: Testing One Brick, Not the Whole Wall
A unit test has one job: verify that a single piece of logic — usually a method, sometimes a small cluster of tightly related methods — behaves correctly in complete isolation. No database. No web server. No other services. Just your code, and whatever fake objects you hand it.
Here’s what that actually looks like for a service class that depends on a repository:
@ExtendWith(MockitoExtension.class)
class EmployeeServiceTest {
@Mock
private EmployeeRepository employeeRepository;
@InjectMocks
private EmployeeServiceImpl employeeService;
@Test
void whenValidId_thenEmployeeShouldBeFound() {
// Arrange
Employee mockEmployee = new Employee(1L, "Alex", "[email protected]");
when(employeeRepository.findById(1L)).thenReturn(Optional.of(mockEmployee));
// Act
Employee result = employeeService.getEmployeeById(1L);
// Assert
assertThat(result.getName()).isEqualTo("Alex");
}
}
Notice something: no @SpringBootTest. No Spring context loading. This test uses MockitoExtension, not the Spring test runner. It doesn’t need Spring at all — it’s plain Java, plain Mockito, and it runs in milliseconds. That speed matters more than people give it credit for. A unit test suite that finishes in 4 seconds gets run constantly. One that takes 3 minutes because it keeps bootstrapping application contexts gets run once before lunch and never again.
This is where Mockito earns its keep. Real dependencies — databases, external APIs, other microservices — are slow, flaky, and honestly none of a unit test’s business. Mockito lets you say “when this method is called, pretend it returned this,” so you can focus entirely on your logic, not your dependency’s behavior.
The mistake most people make here isn’t using mocks — it’s testing the mock instead of the logic. If your test only verifies that employeeRepository.findById() was called, and never actually checks what your service does with that result, you’ve written a test that protects nothing.
Integration Testing: Making Sure the Bricks Actually Fit Together
This is where things get more interesting, and where my 2 AM incident actually lived.
An integration test checks whether multiple real components — your controller, your service, your repository, sometimes a real (or in-memory) database — work correctly together. This is where Spring Boot’s testing annotations start doing heavy lifting.
For testing the persistence layer specifically, @DataJpaTest is the tool of choice:
@DataJpaTest
class EmployeeRepositoryTest {
@Autowired
private EmployeeRepository employeeRepository;
@Test
void testFindByEmail_whenEmailExists_thenReturnEmployee() {
// Arrange
Employee employee = Employee.builder()
.name("Himanshu")
.email("[email protected]")
.build();
employeeRepository.save(employee);
// Act
List<Employee> result = employeeRepository.findByEmail(employee.getEmail());
// Assert
assertThat(result).isNotEmpty();
assertThat(result.get(0).getEmail()).isEqualTo(employee.getEmail());
}
}
@DataJpaTest spins up an in-memory H2 database automatically, configures Hibernate, and scans your JPA entities — without loading your entire application. It’s focused. That’s the whole point.
For the controller layer, @WebMvcTest gives you a similarly narrow slice — it boots up just the web layer, not your services or repositories:
@WebMvcTest(EmployeeController.class)
class EmployeeControllerTest {
@Autowired
private MockMvc mockMvc;
@MockBean
private EmployeeService employeeService;
@Test
void givenEmployees_whenGetEmployees_thenReturnJsonArray() throws Exception {
Employee alex = new Employee(1L, "Alex", "[email protected]");
given(employeeService.getAllEmployees()).willReturn(List.of(alex));
mockMvc.perform(get("/api/employees")
.contentType(MediaType.APPLICATION_JSON))
.andExpect(status().isOk())
.andExpect(jsonPath("$[0].name", is("Alex")));
}
}
And then there’s @SpringBootTest, the big one — it loads your entire application context, and it’s genuinely useful when you want a real end-to-end check across layers:
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.MOCK)
@AutoConfigureMockMvc
class EmployeeRestControllerIntegrationTest {
@Autowired
private MockMvc mvc;
@Test
void givenEmployees_whenGetEmployees_thenStatus200() throws Exception {
mvc.perform(get("/api/employees"))
.andExpect(status().isOk());
}
}
This is powerful. It’s also the exact tool that gets misused constantly — which brings us back to that stack trace at 2 AM.
The Climax: Where My Test Suite Lied to Me
Here’s what actually happened. Somewhere in EmployeeServiceImpl, a method looked roughly like this:
public Employee getEmployeeByEmail(String email) {
return employeeRepository.findByEmail(email).get(0);
}In my unit test, I had mocked employeeRepository.findByEmail() to always return a list with exactly one employee in it. Every single time. My test never once considered what happens when the list comes back empty — which, in production, it did, because someone searched for an email that didn’t exist yet.
.get(0) on an empty list. IndexOutOfBoundsException dressed up as a NullPointerException by the time it bubbled through three layers of abstraction. My “comprehensive” test suite had 214 green checkmarks and exactly zero tests for the one scenario that actually happened in the real world.
That’s the “most people do it wrong” part. Not that people skip testing — most engineers I know test plenty. The problem is what they test. They test the happy path, mock it into submission, and never ask “what’s the boring, unglamorous, annoying edge case here?” Empty lists. Null values. Timeouts. Duplicate entries. The stuff that isn’t fun to write tests for is exactly the stuff that ends up in your pager alerts.
Integration tests using @DataJpaTest and @SpringBootTest are what would have caught this — because they hit a real (in-memory) database, and a real empty result set behaves like an empty result set, not like whatever convenient value I told Mockito to hand back.
The Fix: A Testing Pyramid That Actually Holds Weight
The lesson wasn’t “write more tests.” It was “write the right tests, at the right layer, for the right reason.”
Here’s the mental model that’s held up for me since:
Unit tests (the bulk of your suite): fast, isolated, mock everything external, and — critically — test your edge cases, not just your happy path. Empty results. Nulls. Exceptions being thrown. These should run in seconds and run constantly.
Integration tests (a smaller but essential slice): real database interactions via @DataJpaTest, real HTTP request/response cycles via @WebMvcTest or @SpringBootTest, verifying that your layers actually talk to each other the way you assume they do.
End-to-end tests (the thinnest layer): full application, simulating an actual user journey, usually the slowest and most expensive, reserved for your critical paths only.
None of these annotations are decoration. @MockBean exists so you can isolate a layer while still running inside a Spring context. @TestConfiguration exists so you can swap in test-specific beans without polluting your real configuration. @AutoConfigureMockMvc exists so you’re not spinning up an actual servlet container just to hit an endpoint. Each one has a job. Using @SpringBootTest for everything because it’s the one annotation you remember is like using a sledgehammer to hang a picture frame — it’ll technically work, right up until it doesn’t.
Where I Ended Up
These days, before I write a single test, I ask myself one question: what am I actually trying to prove here? If the answer is “this one method handles bad input correctly,” that’s a unit test, mocked and isolated. If the answer is “these three layers actually cooperate,” that’s an integration test, and I’m not afraid to let it touch a real (in-memory) database.
And every single time — every time — I make myself write the test for the case that annoys me the most to think about. The empty list. The null. The timeout. The thing I’m tempted to skip because “that probably won’t happen.” My 2 AM phone call happened precisely because I skipped that thing once.
I put together a lot of these annotations, patterns, and the “which one do I actually use here” decision tree into a single reference I keep pinned in my browser tabs, because honestly, I still forget the difference between @DataJpaTest and @JdbcTest more often than I’d like to admit. If you’re the kind of person who’d rather have that cheat sheet than relearn it the hard way at 2 AM, I put it together here: Ultimate Spring Boot Interview & Development Cheat Sheet. It’s the same annotations and gotchas from this article, plus the interview angle, in one place I actually reuse.
Testing isn’t about the badge on your GitHub repo. It’s about whether the code holds up when someone you’ve never met does something you never imagined at an hour you’d rather be asleep for.
Write the boring test. Especially the boring test.
Happy Coding!
The 2 AM Bug That Taught Me I Was Testing Spring Boot All Wrong was originally published in Javarevisited on Medium, where people are continuing the conversation by highlighting and responding to this story.
This post first appeared on Read More