JarCastingNeo4j Migrations
This is a tool for defining Java based migrations that should be applied and recorded against a Neo4j instance. The only dependencies are the Neo4j Java driver and ClassGraph, the later being used to find migrations on the classpath.
Neo4j Migrations is inspired by FlywayDB, which is an awesome tool for migration of relational databases.
Usage
As standalone CLI
Since 0.0.5, you can grab binaries of neo4j-migrations CLI version at Maven central: neo4j-migrations-0.1.1.zip
Unzip them (unzip neo4j-migrations-cli-0.1.1.zip), go the the director (cd neo4j-migrations-0.1.1) and execute the program (neo4j-migrations on Unix-like operating systems and neo4j-migrations.cmd on Windows):
./neo4j-migrations -h
Usage: neo4j-migrations [-hvV] -p[=<password>] [-p[=<password>]]...
[-a=<address>] [-d=<database>]
[--transaction-mode=<transactionMode>] [-u=<user>]
[--location=<locationsToScan>]...
[--package=<packagesToScan>]... [COMMAND]
Migrates Neo4j databases.
-a, --address=<address> The address this migration should connect to. The
driver supports bolt, bolt+routing or neo4j as
schemes.
-d, --database=<database> The database to migration (Neo4j 4.0+).
-h, --help Show this help message and exit.
--location=<locationsToScan>
Location to scan. Repeat for multiple locations.
-p, --password[=<password>]
The password of the user connecting to the database.
--package=<packagesToScan>
Package to scan. Repeat for multiple packages.
--transaction-mode=<transactionMode>
The transaction mode to use.
-u, --username=<user> The login of the user connecting to the database.
-v Log the configuration and a couple of other things.
-V, --version Print version information and exit.
Commands:
info Retrieves all applied and pending informations, prints them and
exits.
migrate Retrieves all pending migrations, verify and applies them.
Here’s an example that looks for migrations in a Java package and it’s subpackages and in a filesystem location for Cypher based migrations. It uses the info command to tell you which migrations have been applied and which not:
./neo4j-migrations -uneo4j -psecret --package some.migrations --location file:$HOME/Desktop/foo info
Database: Neo4j/4.0.0@localhost:7687
+---------+-----------------------------+--------+--------------+----+----------------+---------+--------------------------------------------------------+
| Version | Description | Type | Installed on | by | Execution time | State | Source |
+---------+-----------------------------+--------+--------------+----+----------------+---------+--------------------------------------------------------+
| 001 | FirstMigration | JAVA | | | | PENDING | some.migrations.changeset1.V001__FirstMigration |
| 002 | AnotherMigration | JAVA | | | | PENDING | some.migrations.changeset1.V002__AnotherMigration |
| 023 | NichtsIstWieEsScheint | JAVA | | | | PENDING | some.migrations.changeset2.V023__NichtsIstWieEsScheint |
| 025 | SlowMigration | JAVA | | | | PENDING | some.migrations.changeset3.V025__SlowMigration |
| 030 | Something based on a script | CYPHER | | | | PENDING | V030__Something_based_on_a_script.cypher |
| 042 | The truth | CYPHER | | | | PENDING | V042__The truth.cypher |
+---------+-----------------------------+--------+--------------+----+----------------+---------+--------------------------------------------------------+
You can repeat both --package and --location parameter for fine grained control. Use migrate to apply migrations:
./neo4j-migrations -uneo4j -psecret --package some.migrations.changeset1 --package some.migrations.changeset2 migrate
Applied migration 001 ("FirstMigration")
Applied migration 002 ("AnotherMigration")
Applied migration 023 ("NichtsIstWieEsScheint")
Database migrated to version 023.
If we go back to the info example above and grab all migrations again, we find the following result:
./neo4j-migrations -uneo4j -psecret --package some.migrations --location file:$HOME/Desktop/foo info
Database: Neo4j/4.0.0@localhost:7687
+---------+-----------------------------+--------+-------------------------------+---------------+----------------+---------+--------------------------------------------------------+
| Version | Description | Type | Installed on | by | Execution time | State | Source |
+---------+-----------------------------+--------+-------------------------------+---------------+----------------+---------+--------------------------------------------------------+
| 001 | FirstMigration | JAVA | 2020-01-17T15:34:16.388Z[UTC] | msimons/neo4j | PT0S | APPLIED | some.migrations.changeset1.V001__FirstMigration |
| 002 | AnotherMigration | JAVA | 2020-01-17T15:34:16.406Z[UTC] | msimons/neo4j | PT0S | APPLIED | some.migrations.changeset1.V002__AnotherMigration |
| 023 | NichtsIstWieEsScheint | JAVA | 2020-01-17T15:34:16.417Z[UTC] | msimons/neo4j | PT0S | APPLIED | some.migrations.changeset2.V023__NichtsIstWieEsScheint |
| 025 | SlowMigration | JAVA | | | | PENDING | some.migrations.changeset3.V025__SlowMigration |
| 030 | Something based on a script | CYPHER | | | | PENDING | V030__Something_based_on_a_script.cypher |
| 042 | The truth | CYPHER | | | | PENDING | V042__The truth.cypher |
+---------+-----------------------------+--------+-------------------------------+---------------+----------------+---------+--------------------------------------------------------+
Another migrate - this time with all packages - gives us the following output and result:
./neo4j-migrations -uneo4j -psecret --package some.migrations --location file:$HOME/Desktop/foo migrate
Skipping already applied migration 001 ("FirstMigration")
Skipping already applied migration 002 ("AnotherMigration")
Skipping already applied migration 023 ("NichtsIstWieEsScheint")
Applied migration 025 ("SlowMigration")
Applied migration 030 ("Something based on a script")
Applied migration 042 ("The truth")
Database migrated to version 042.
./neo4j-migrations -uneo4j -psecret --package some.migrations --location file:$HOME/Desktop/foo info
Database: Neo4j/4.0.0@localhost:7687
+---------+-----------------------------+--------+-------------------------------+---------------+----------------+---------+--------------------------------------------------------+
| Version | Description | Type | Installed on | by | Execution time | State | Source |
+---------+-----------------------------+--------+-------------------------------+---------------+----------------+---------+--------------------------------------------------------+
| 001 | FirstMigration | JAVA | 2020-01-17T15:34:16.388Z[UTC] | msimons/neo4j | PT0S | APPLIED | some.migrations.changeset1.V001__FirstMigration |
| 002 | AnotherMigration | JAVA | 2020-01-17T15:34:16.406Z[UTC] | msimons/neo4j | PT0S | APPLIED | some.migrations.changeset1.V002__AnotherMigration |
| 023 | NichtsIstWieEsScheint | JAVA | 2020-01-17T15:34:16.417Z[UTC] | msimons/neo4j | PT0S | APPLIED | some.migrations.changeset2.V023__NichtsIstWieEsScheint |
| 025 | SlowMigration | JAVA | 2020-01-17T15:36:06.899Z[UTC] | msimons/neo4j | PT0.503S | APPLIED | some.migrations.changeset3.V025__SlowMigration |
| 030 | Something based on a script | CYPHER | 2020-01-17T15:36:07.001Z[UTC] | msimons/neo4j | PT0.004S | APPLIED | V030__Something_based_on_a_script.cypher |
| 042 | The truth | CYPHER | 2020-01-17T15:36:07.016Z[UTC] | msimons/neo4j | PT0.003S | APPLIED | V042__The truth.cypher |
+---------+-----------------------------+--------+-------------------------------+---------------+----------------+---------+--------------------------------------------------------+
From your build tool
Maven Plugin
You can trigger Neo4j Migrations from your build via plugin:
<plugin>
<groupId>eu.michael-simons.neo4j</groupId>
<artifactId>neo4j-migrations-maven-plugin</artifactId>
<version>0.1.1</version>
<executions>
<execution>
<id>migrate</id>
<goals>
<goal>migrate</goal>
</goals>
<configuration>
<user>neo4j</user>
<password>secret</password>
<address>bolt://localhost:${it-database-port}</address>
<verbose>true</verbose>
</configuration>
</execution>
</executions>
</plugin>
By default, the plugin will look in neo4j/migrations resource. You can change that via locationsToScan:
<locationsToScan>
<locationToScan>file://${project.build.outputDirectory}/custom/path/to/migrate</locationToScan>
</locationsToScan>
Add multiple elements for multiple locations. The plugin has the same parameters as the standalone or CLI version.
Inside your application
In a Spring Boot application
We provide a starter with automatic configuration for Spring Boot. Declare the following dependency in your Spring Boot application:
<dependency>
<groupId>eu.michael-simons.neo4j</groupId>
<artifactId>neo4j-migrations-spring-boot-starter</artifactId>
<version>0.1.1</version>
</dependency>
That starter itself depends on the Neo4j Java Driver. The driver is managed by Spring Boot since 2.4 and you can enjoy configuration support directly through Spring Boot. For Boot versions prior to Spring Boot 2.4, please have a look at version 0.0.13 of this library.
Neo4j Migrations will automatically look for migrations in classpath:neo4j/migrations and will fail if this location does not exists. It does not scan by default for Java based migrations.
Here’s an example on how to configure the driver and the migrations:
spring.neo4j.authentication.username=neo4j
spring.neo4j.authentication.password=secret
spring.neo4j.uri=bolt://localhost:7687
# Add configuration for your migrations, for example, additional packages to scan
org.neo4j.migrations.packages-to-scan=your.changesets, another.changeset
# Or disable the check if the location exists
org.neo4j.migrations.check-location=false
The following configuration properties are supported:
| Name | Type | Default | Description |
|---|---|---|---|
|
java.lang.Boolean |
true |
Whether to check that migration scripts location exists. |
|
java.lang.String |
|
The database that should be migrated (Neo4j 4.0+ only). Leave {@literal null} for using the default database. |
|
java.lang.Boolean |
true |
Whether to enable Neo4j migrations or not. |
|
java.nio.charset.Charset |
UTF-8 |
Encoding of Cypher migrations. |
|
java.lang.String |
System user |
Username recorded as property {@literal by} on the MIGRATED_TO relationship. |
|
java.lang.String[] |
|
Locations of migrations scripts. |
|
java.lang.String[] |
An empty array |
List of packages to scan for Java migrations. |
|
TransactionMode |
|
The transaction mode in use (Defaults to "per migration", meaning one script is run in one transaction). |
|
Note
|
Migrations can be disabled by setting org.neo4j.migrations.enabled to false. |
Other applications
Declare the extension as Maven dependency:
<dependency>
<groupId>eu.michael-simons.neo4j</groupId>
<artifactId>neo4j-migrations</artifactId>
<version>0.1.1</version>
</dependency>
Put your migrations as Java classes into your project:
import ac.simons.neo4j.migrations.core.JavaBasedMigration;
import ac.simons.neo4j.migrations.core.MigrationContext;
import org.neo4j.driver.Driver;
import org.neo4j.driver.Session;
public class V001__MyFirstMigration implements JavaBasedMigration {
@Override
public void apply(MigrationContext context) {
try (Session session = context.getSession()) { // (1)
// Steps necessary for a migration
}
}
}
-
It is important that you use the supplied session (or the
SessionConfigif you want to use another type ofSession) for your session retrieval, otherwise you may run that migration in a different database than in which the tool itself is run. However, you are free in which database you run this. Your mileage may vary.
The class names must start with a V followed by digits followed by __ and than some valid Java class name.
To use them create a Migrations instance to scan your project and apply all found migrations:
Migrations migrations = new Migrations(
MigrationsConfig.builder().withPackagesToScan("org.company.changeset1").build(),
GraphDatabase.driver("bolt://localhost:7687", AuthTokens.basic("neo4j", "secret"))
);
migrations.apply();
You’re migrations will be recorded as a chain of applied migrations (as nodes with the label __Neo4jMigration). They can use the driver any way they like.
There’s no rollback yet. If any migration fails, the chain will stop, but will not rollback previous migrations.
Cypher script based migrations
You can put Cypher scripts ending with .cypher inside your classpath resources under neo4j/migrations. From there on they’ll be picked up automatically.
Here’s an example:
CREATE (agent:`007`) RETURN agent;
UNWIND RANGE(1,6) AS i
WITH i CREATE (n:OtherAgents {idx: '00' + i})
RETURN n
;Scripts can contain multiple statements, separated by a ; followed by a newline. Statements will be executed in one transaction by default. That behaviour can be changed as follows:
Migrations migrations= new Migrations(
MigrationsConfig.builder()
.withTransactionMode(MigrationsConfig.TransactionMode.PER_STATEMENT)
.build(),
driver
);
migrations.apply();
If you want Migrations to look at other places, configure it as follows:
Migrations migrations = new Migrations(
MigrationsConfig.builder()
.withLocationsToScan(
"classpath:my/awesome/migrations",
"file:/path/to/migration"
).build(),
GraphDatabase.driver("bolt://localhost:7687", AuthTokens.basic("neo4j", "secret"))
);
migrations.apply();
For the adventurous
The Neo4j Java driver and this application supports native compilation with GraalVM so that you can create a native executable. Read more about it here.
After installing at least GraalVM 20.1.0, prepare your environment as follows:
export GRAALVM_HOME=/Library/Java/JavaVirtualMachines/graalvm-ce-java11-20.1.0/Contents/Home
export JAVA_HOME=$GRAALVM_HOME
The paths are probably different on your system.
The neo4j-migrations-cli module has a build profile create-native-image that you use to create a binary for your OS. Run it with:
./mvnw -Pcreate-native-image clean package
The resulting migration tool can only be used to load Cypher script based migrations in file locations. It won’t find classes, as those are instantiated dynamically via reflection, which is not supported in full in a native image.