Automated tests have become crucial in the field of software engineering in the last few years, even more than in the past. In fact, automated testing is now part of the Continuous Integration / Continuous Delivery (CI/CD) process, so tests may run in different shapes and environments throughout the development of software artefacts. While Unit Tests can be executed by taking apart a core component or class, by mocking every external dependency (DB included), Integration Tests and End-to-End tests require at least one real external component, in order to be as realistic as possible.
Scope
In this blog post we’ll discuss Integration Tests (ITs from now on) when one of the components involved is Neo4j. More specifically, we’ll funnel our effort on components written in Java and we will show how Testcontainers for Java can leverage successful Tests suites and, consequently, being enablers of the stability of your software.
This is not the first time that GraphAware contributes with a blog post about Integration Tests, Java applications and Testcontainers: a few years ago, Frantisek Hartmann had already given more than an overview of the landscape of Testcontainers working together with Neo4j. (https://graphaware.com/blog/neo4j/integration-testing-with-docker-neo4j-image-and-testcontainers.html); since then, the ecosystem of Testcontainers evolved quite a lot, by introducing new modules (included a Neo4j module) and new features. Last but not least, AtomicJar, the company that gave birth to Testcontainers, is now part of Docker Inc. and it’s used around the world from big companies like Netflix, Uber, Spotify, DoorDash, Google, GitHub, Capital One, Skyscanner, Wise and many more.
Agenda
This blogpost is divided into three parts:
- At first, we’ll give an overview of Neo4j Harness, with pros and cons
- Secondly, we’ll take a deep dive into Testcontainers, with a special focus on the Neo4jContainer module and some of its possible configurations. a. We’ll compare Neo4jContainers with GenericContainers and show how beneficial the first one is. b. We’ll come up with a few tips and tricks to optimise the number of Testcontainers we can spin up in a test suite.
- The third part is a practical example taken from hands-on situations: we’ll build an API that must interact and support Neo4j 4.4 and 5, at the same time.
Versions
Any examples described here following are based on the latest version of Neo4j, at the moment we’re writing, which is 5.16, built on top of Java 17.
Testcontainers latest version while typing is 1.19.4. We are going to adopt such a version.
Lastly, we’ll adopt Junit Juniper 5.
Neo4j Harness
Neo4j Harness is a library that enables developers to test several scenarios, including Spring Applications which adopt Neo4j as Graph Database, but also Neo4j-OGM, Neo4j plugins, etc. Such a library gives the opportunity to spin up a lightweight Neo4j instance, and it was a great option when the only alternative was using an actual Neo4j server dedicated to testing purposes.
Due to its lightweight nature, it has definitely two main advantages.
- Short startup time of the embedded Neo4j instance.
- Quick deployments of plugins, when you’re developing user-defined procedures, functions or aggregation function But, unfortunately, there are also drawbacks:
- it doesn’t guarantee the behaviour of the embedded Neo4j target instance will 100% match the corresponding actual Neo4j server version.
- It pollutes your dependencies with other required dependencies, that might conflict with production ones.
- You can test only one Neo4j target version per time: in fact, the target instance is set only in Maven/Gradle dependencies.
Also, since you can reuse Testcontainers between test executions (read “keep them running from one test run to another”), for CI purposes, keeping on adopting Harness could seem pointless for many scenarios.
However, when it comes to developing Neo4j plugins, adopting Neo4j Test Harness during the development has a huge impact in terms of time spent, compared with Testcontainers. In fact, if you’re about to deploy a small change to your code, with Testcontainers you have to restart an actual instance, which is time consuming. Rather, the lightweight nature of Neo4j Test Harness enables deployment and being able to debug your plugin in a few seconds, so we strongly suggest it in such scenarios.
An example of usage of Neo4j Harness is available here.
Neo4j Module
Set up Testcontainer dependencies
Let’s now start with a few examples of usage of Testcontainers, with a special attention to the Neo4j database as a target.
Any Docker image available on Docker Hub or on a private registry can turn into a testcontainer, by instantiating a GenericContainer. However, when custom modules are available, which are subclasses of the GenericContainer class, they provide out-of-the-box configurations, making life easier for developers.
The Neo4j Module is one of them, so we will quickly show the main advantages of it.
To run all the examples below on, as a prerequisite, you need the Docker Engine already installed. If not, please follow the Official Documentation.
Then, add the following dependencies to your Java project:
We are explicitly excluding Junit 4 from dependencies, as Testcontainers bring it out of the box (will be removed with Testcontainers 2, but for now we have to deal with a hack to get rid of such a library, that I’ll explain soon).
Also, we would need another one for bringing in the Neo4j Module
Last, but not the least, the Neo4j Java driver
In the GitHub repository you will find further dependencies, but they’re specific to the examples we’re showing. Amid them:
It’s basically the Neo4j server. Since our example consists in a Neo4j plugin, we assume that the underlying infrastructure is a Neo4j instance (hence the scope is provided).
And then
- datafaker:2.0.1 : for generating random mock data
- assertj-core:3.24.2 : for better assertions, not mandatory.
Exclude JUnit 4
WARNING: As we have already mentioned, to make sure we will use only JUnit 5 for our tests, we have added the exclusion for Junit 4 artefacts.
However, it’s not enough.
In fact, if we ran:
We would get a compilation failure because of missing classes
cannot access org.junit.rules.TestRule
This class and a few others from JUnit4 are not actually used, so in order to make sure we can compile, we will mock them by reproducing the following folder structure and files under test/java
Don’t forget to follow such hierarchical structure and add TestRule (Interface) and Statement (Class), which are actually empty. Here you can find an example that contains the snippets below.
No more JUnit4 dependencies, long life to JUnit5!
Run Our First Test
Now we’re ready to give a try, for the first time, to a Neo4j Container:
Please note that the statement
With the GenericContainer, instead of the single line above, it would have been something like:
Pretty verbose! It’s obvious that the Neo4j Containers keep us away from boilerplate code, by bringing all of these things out-of-the-box. Therefore it makes the configuration easier and reduces the risk of mistakes.
In the GitHub repository you can find more than one example of comparison between Neo4jContainers and GenericContainers.
Use an Alternative Docker Registry
Docker Hub applies a rate limiter strategy to the number of pulls,based on your account type (https://docs.docker.com/docker-hub/download-rate-limit/). Free plans, for instance, are allowed to pull 100 times every 6 hours (anonymous users), which is more than acceptable if you’re working on your laptop, but it may turn out to be too restrictive in a CI pipeline, where a team triggers several jobs concurrently. In such cases, you have two options: the first one is a zero-maintenance solution, that is an upgrade of your Docker Hub account towards a paid one that better fits your usage goals. The second one is mirroring to your own registry server (https://docs.docker.com/registry/deploying/ ); if this is your choice, the creation of the Neo4jContainer should change a little bit to be like the following.
Neo4j Enterprise Edition
Neo4jContainer offers a shortcut for testing against a Neo4j Enterprise instance:
Definitely the shortest way for spinning up a Neo4j Enterprise instance. Nevertheless, there is a small drawback: you should include a container-license-acceptance.txt file to the root of your classpath, containing the text neo4j:4.4-enterprise in one line, regardless of the target version of Neo4j.
Another limitation is that the withEnterprise() clause does not support unofficial Docker registries.
In fact, instantiating a Neo4j container pointing a Neo4j Enterprise version on a registry different than Docker Hub will return an exception:
Exception returned:
java.lang.IllegalStateException: Cannot use enterprise version with alternative image my.mirror.com/neo4j:5.16.0-enterprise
In order to avoid such limitation, give a try to the following workaround:
It won’t require any txt file in your classpath.
Neo4j With Plugins
Sometimes our test cases rely on a specific library or it’s testing our own custom plugins (custom functions, procedures, etc). Neo4j Plugins are all shipped as jar files, however we must distinguish between Neo4jLabs plugins and plugins from third parties.
Neo4j Labs Plugins
In the first case, there are the following shortcut methods which allow the automatic download from the Neo4j CDN and the deployment of the required official plugins. For instance, if we’re testing a scenario with both APOC and GDS libraries required, we’d write:
Other available plugins are APOC Core, Bloom, Streams, N10, Neo Semantics
Custom Plugins
In this scenario, the jar must be placed somewhere in the Docker host filesystem or in the classpath. In the following example we chose the first option
Please notice that if we prefer to use our local copy of Neo4j Labs plugins, we can manually download the jar beforehand and use the .withPlugins() option.
Mixed Mode (Labs and custom plugins at the same time)
It’s always possible to use both approaches:
How to Reuse Containers
By definition, Testcontainers (not only Neo4jContainer) are disposable, but their lifespan can vary quite a lot, according to the scenario, the expected performance and complexity.
From the shortest to the longest duration, below you can find how long a Testcontainer can live, namely the time interval between the creation and the destruction of the container:
- (Shortest) a single test case execution: Every test method in a class will start up and tear down a container.
- All the test cases of a class: the container starts up before the first test case execution and it’s destroyed after the last test case.
- Singleton Containers Pattern: containers that can be shared among several test classes, ideally alongside the entire test suite.
- Reuse containers among different test executions of the same test suite.
Single Test Case Execution
Running a new container for every test is not recommended because it’s a resource intensive operation.
Moreover, there’s an easier way to clean up your database before each test, which are @BeforeEach and @AfterEach annotated methods.
When you declare a container like the following (not static), its lifespan will end after the execution of each test case
Container Shared Among Rests of a Class
The Testcontainer must be declared as a static field for the class: it’ll keep it alive among test cases. Also, don’t forget to clean the database if you don’t want the current test to be polluted from previous ones.
There are two options available:
1. Using JUnit 5 Lifecycle Callback Methods:
We can use the JUnit 5 @BeforeAll and @AfterAll lifecycle callback methods to start and stop the containers. We suggest to do not explicitly stop containers, the Ryuk container will do it automatically on your behalf
2. Using JUnit 5 Extension Annotations:
The JUnit 5 Extension offered by the Testcontainers library streamlines the initiation and cessation of containers through annotations. To utilise this extension, ensure the inclusion of the org.testcontainers:junit-jupiter test dependency in your project.
Singleton Containers
The Testcontainers library is frequently used for crafting integration tests involving essential services encapsulated within Docker containers. As the count of test classes grows, the frequency of container initialization also rises, which means time spent growing up.
To address this, it can be beneficial to initiate all containers collectively in a shared base class, ensuring consistency across your integration tests. To implement this approach, have your integration test class extend this base class, following the Singleton Containers Pattern.
Base abstract class:
Test class:
Reuse Containers Among Different Executions
Imagine the following scenario: you are a developer and you need to run a bunch of tests against a singleton Neo4j instance multiple times; to do so, you’re working on your IDE and, when some of them fail, you fix and retry, over and over again.
It’s generally not acceptable to wait all of these times for a Neo4j startup: you’ll definitely lose the thread!
A combination of the techniques described above, plus the usage of the withReuse() flag can resolve this issue, by extending the lifespan of a container even after the execution of the entire suite of tests (or a subset of it).
How to enable reusable Testcontainers:
- enable Reusable Containers in ~/.testcontainers.properties, by adding testcontainers.reuse.enable=true
- define a container and subscribe to reuse the container using withReuse(true)
Example: how to test multiple versions of Neo4j
It’s pretty common that production databases are sometimes not up-to-date with the latest version of a DBMS, Neo4j or not; some clients have an intrinsic resistence to upgrade to the latest version of Neo4j because of several reasons: keeping a working version, complex upgrade process or just laziness are just a few examples.
For this reason, among your clients you may deal with clients with an older version of Neo4j (let’s say, 4.4 LTS) and clients which are always up-to-date.
Nevertheless, due to this disparity, your applications must ensure compatibility with diverse systems.
As a consequence, if your app supports several versions of Neo4j, it’s very important to test against all of these versions.
For example, as at the moment we write both Neo4j 4.4 is LTS and 5.16 is the last release, so in GitHub you can find a full example, which is an API for writing movies on Neo4j and retrieve them all (in a nutshell, it’s a subset of CRUD).
Some highlights:
- Dynamically set the Bolt URL into the System Properties, in order to connect the application under test to connect to.
- Used @ParameterizedTest to pass the Neo4j version to test, and consequently the Testcontainer to run against. The Bolt URL is set dynamically, according to the target Testcontainer.
- Adopted some of the techniques above described (Singleton above all)
Besides, bear in mind to:
- Keep the list of supported versions as short as possible, because multiple instances of Neo4j will run on your machine at the same time, so it’ll impact the performance of your machine.
- Don’t forget to kill them manually once you’re done: release memory for other containers, in order to prevent any OOM Errors.
Summary
- Neo4j Harness is useful when you write custom procedures for Neo4j, in order to ease the continuous development loop: in fact, you don’t need to restart Neo4j to deploy a new version of the plugin under development.
- For the rest of scenarios, the Neo4jContainer module helps you to spin up Neo4j test containers with as little code as possible. Thanks to default properties of the Neo4jContainer, a lot of settings come for free, so you can focus on specific settings that diverge from default ones. With GenericContainer you had to set every single property manually, but also readiness probes.
- For intensive cycles of development, the withReuse() clause can prevent developers from waiting for container startups, by keeping the containers running even when the test suite has finished its execution. Don’t forget to drop running containers at the end of your working day.
- Thanks to a combination of the techniques described above, plus a wise use of JUnit5 ParameterizedTests, it’s possible to run the same tests against different versions of Neo4j. However, since the upper limit is the memory of your machine, it’s recommended to keep the number of versions low, because you’ll have a container running for each version, at the same time.
