For the Global GraphHack 2019, we extended JMeter to support the Bolt protocol and do load testing on Cypher queries.
Apache JMeter is a popular, open source application to load test functionality and measure performance. It started off with the intent of testing Web Applications but its widespread use and availability of reports and visualizers saw it expand to supporting the testing of various protocols including databases via JDBC.
JMeter would be very suitable to test Neo4j procedures as well — often, procedures are not load tested early enough and there aren’t very straightforward ways of testing them for performance. So to celebrate the first ever Global GraphHack 2019, we extended JMeter to provide support for the Bolt protocol and the execution of Cypher queries, and hence, procedures.
This extension has been integrated in the JMeter codebase and is available in the latest release.
The JMeter Bolt protocol support
The new bolt protocol support adds 2 new elements to the JMeter UI: the Bolt connection configuration and the Bolt Sampler.
Database connection configuration
The first thing to do is to configure Neo4j connection parameters.
This is done through the Bolt Connection Configuration, which you just need to add at the root of your test plan. It will automatically handle the database driver initialization and make sure it is cleaned up after the test.
At the moment, the configuration is fairly basic and you cannot configure additional parameters. But the driver defaults are quite good out of the box and allow you to run a reasonable benchmark. Feel free to open an issue or send a pull request here if you need more!
Sampler
Once the configuration part is done, we can now define what our actual load test will run. In JMeter this is done through samplers, and in our Neo4j case, with Bolt Request samplers. Each sampler contains a cypher query, along with its parameters.
The parameters have to be provided in json format, where the keys must match the query parameters. The json values in the example above are JMeter variables, and in this case, are dynamically replaced with the values coming from a CSV dataset. Other forms of JMeter variable substitution are also supported, such as User Defined Variables.
We strongly recommend using query parameters, as they allow the database to cache the query execution plans and get better performance.
There is also an additional option Record Query Results, in case you’d like to store/display the data returned by the query. By default, this option is disabled to minimize memory usage, and avoid iterating through the entire resultset.
Note that a sampler has to be nested within a thread group that controls the concurrency of the test. In this example, we have JMeter spawn 20 threads that will execute our query in parallel.
Running the test plan
Once the sampler(s) are ready, we can add a couple of reporting elements to gather statistics at runtime, and start the load test.
In the screenshot below, while our test running, we can monitor the number of queries executed, the response times, error rates, etc.
Load testing best practices
In this blog post we went through a very simple example. If you are going to do more advanced testing, make sure to follow JMeter best practices such as running the test in CLI mode, disabling the View Results Tree listener, or do proper validation on query results.
Make sure to check them out on the JMeter website.
What’s next
The bolt support is available right now in JMeter 5.2, released a few days ago.
The example presented here is available at Github, feel free to use it as a bootstrap for your tests.
We welcome feedback and suggestions, and hope you find this useful.
Happy load testing!
