Automated testing is the cornerstone of any successful software project. Applications using the Neo4j database are no exception. This blog post shows how to use the Neo4j Docker image and the Testcontainers library for integration testing in Java using JUnit.
This blog post shows examples in Java. Testcontainers library has been ported to many other languages so the same approach and principles can be applied. Check out theTestcontainersgithub page.
Motivation
Neo4j already provides a testing harness to start a temporary database within tests, either manually or through a JUnit rule. To use this harness one must include the neo4j-harness maven artifact, together with whole Neo4j database as a test dependency to the project. This inevitably pollutes the test classpath leading to various issues, for example
- having conflicting versions of libraries on the classpath (the culprits are usually Lucene, Jetty or Scala), see for example this issue
- Spring auto-configuration is affected by presence of certain classes on classpath
- non-determinism – your code either sometimes fails, or fails only in certain environments
To avoid these issues we will use the Testcontainers library with the official Neo4j Docker image to start the database for our tests.
Testcontainers is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container.
How to use Testcontainers
Let’s start with a simplest possible example of an application using the Neo4j Java driver to connect to a Neo4j database running in server mode.
First you need Docker installed on your machine. If you don’t have Docker installed already follow the official Docker documentation.
Then in your project, add the Testcontainers dependency. Use test scope so you don’t distribute the dependency with your application.
<dependency><groupId>org.testcontainers</groupId><artifactId>testcontainers</artifactId><version>1.10.2</version><scope>test</scope></dependency>and the Neo4j Java driver
<dependency><groupId>org.neo4j.driver</groupId><artifactId>neo4j-java-driver</artifactId><version>1.7.2</version></dependency>Note that there are no other Neo4j dependencies required in the project.
A short while ago there has been amerged PRto the Testcontainers library with special support for Neo4j in form of a Neo4jContainer class. It provides some of the features we cover in this blog post – namely authentication configuration, getting the bolt url and configuring the enterprise edition. We are looking forward to trying it out once it is released!.
Then in your JUnit tests, use the Testcontainers library to start a container with the Neo4j database
publicclassApplicationIT{@ClassRulepublicstaticGenericContainerneo4j=newGenericContainer("neo4j:3.5.0").withExposedPorts(7687@TestpublicvoidshouldAnswerWithOne(){Stringuri="bolt://"+neo4j.getContainerIpAddress()+":"+neo4j.getMappedPort(7687try(Driverdriver=GraphDatabase.driver(uri,AuthTokens.basic("neo4j","neo4j"))){try(Sessionsession=driver.session()){StatementResultresult=session.run("RETURN 1 AS value"intvalue=result.single().get("value").asIntassertThat(value).isEqualTo(1}}}}The rule GenericContainer starts the given docker image before all tests and shuts it down after all tests finish. The setting withExposedPorts exposes a given port from the inside of the container on a random port available from the outside. Then neo4j.getMappedPort() returns this random port.
Once the bolt uri is constructed, the test creates an instance of Neo4j Java driver as usual. See full example on github.
You can also run your tests against enterprise version of Neo4j provided you have a license. Just change the docker image version to x.y.z-enterprise. Of course, you need to have a valid enterprise license
Configuring Neo4j
Neo4j Docker image provides a way to change the Neo4j configuration through environment variables. For details of the naming convention see the official Neo4j documentation. Testcontainers provides the withEnv method to pass a variable to the container, for example, the usual way of disabling authentication in neo4j.conf is
dbms.security.auth_enabled=false With Testcontainers this can be done with
newGenericContainer("neo4j:3.5.0").withEnv("NEO4J_dbms_security_auth__enabled","false")Turning off authentication isn’t a best practice. Fortunately the Neo4j Docker image supports setting password via a special environment variable (this is specific to the image, not Neo4j).
newGenericContainer("neo4j:3.5.0").withEnv("NEO4J_AUTH","neo4j/Password123")The GenericContainer class from Testcontainers library has also few configuration options. For example to reduce timeout during container startup to make debugging faster you can use withStartupTimeout.
newGenericContainer("neo4j:3.5.0").withStartupTimeout(ofSeconds(5))Container scope
Using the @ClassRule annotation specifies that the container is started before the test class and is shut down after all tests are executed. By using the @Rule annotation instead, it is possible to limit the scope to a single test method only. This can be useful for tests which leave the database in unusable state when finished.
In most cases, however, you will likely want to extend the scope of the container beyond a single test class, for example to avoid repeated start of the container and hence reducing the test execution time. Currently there is no native support in the Testcontainers library for this, but it is easily achievable by the following snippet of custom code.
publicclassNeo4jContainerSupport{privatestaticGenericContainerneo4j=newGenericContainer("neo4j:3.5.0").withEnv("NEO4J_dbms_security_auth__enabled","false").withExposedPorts(7687publicstaticvoidstart(){if(!neo4j.isRunning()){neo4j.start}}publicstaticStringuri(){return"bolt://"+neo4j.getContainerIpAddress()+":"+neo4j.getMappedPort(7687}}And in your tests ensure that the container is started.
@BeforeClasspublicstaticvoidsetUpClass(){Neo4jContainerSupport.start}Because a single instance of the database is used for all tests, you must perform a cleanup between tests manually to have the tests independent. This might be tricky if indexes and/or constraints are involved and unless you have a strong requirements about performance using per-class or per-method container might be easier.
Pro tip: clean up your database in @Before method, not @After – you will be able to inspect the state of your database after running a single test.
Notice there is no need to stop the container anywhere, it will be destroyed at JVM exit by the Testcontainers library.
Custom procedure or plugin
Sometimes an application needs a custom Cypher procedure or a Neo4j plugin to run. The Neo4j Docker image loads plugins from the /plugins directory inside the container. This can be achieved with binding the jar file using withFileSystemBind method on the GenericContainer.
Plugins usually fall into one of following categories, which further complicates things
- no dependencies
- has dependencies, can be packaged as fat jar
- has dependencies, cannot be packaged as fat jar
No dependencies
This is the simplest case. Your fully functioning plugin jar is prepared by the time the integration test phase runs. Simply bind the final jar to the plugins folder. See the example on github.
Fat jar
Similar to previous case, just bind the fat jar, instead of the regular jar. See the example on github.
Jar with dependencies
While Neo4j itself shows in the manual how to package a plugin jar with dependencies, there are reasons why you might not want to do that (for example you may simply loathe fat jars or you have been burned with classpath issues). In such cases, you can copy all dependencies to a folder and bind the folder to the plugins. See the example on github.
In any case, don’t forget to whitelist your procedures if required.
Existing docker compose
Do you already have a docker compose file describing your Neo4j container with all plugins and configuration? By using DockerComposeContainer you can take advantage of it and start Neo4j (and others services) in the same way as you do in your development and production environments.
Summary
Testcontainers works really well together with Neo4j and is a handy tool to have in one’s toolbox.
It provides several advantages
- easy to use
- cleans up your dependencies, removes conflicts
- tests execution is closer/identical to the production environment
and has some disadvantages
- might have worse performance
- need to have Docker installed on your local and CI machines, with all it’s caveats.