JarCastingCassandra Binary Store
Allows storing large binary files in Cassandra via the Vert.x event bus keeping file chunks in a binary format. The files are kept in two Cassandra tables:
- files - contains file information such as filename, contentType, length, chunkSize, and additional metadata
- chunks - contains the chunks of binary data
Dependencies
This module requires a Cassandra 1.2+ server to be available on the network.
Configuration
The Cassandra Binary Store module takes the following configuration:
{
"address": <address>,
"keyspace": <keyspace>
}
For example:
{
"address": "test.my_persistor",
"keyspace": "my_keyspace"
}
Let's take a look at each field in turn:
addressThe main address for the module. Every module has a main address. Defaults toet.cassandra.binarystore.keyspaceThe keyspace name in Cassandra instance to use. Defaults tobinarystore. This can also be set via env var BINARYSTORE_KEYSPACE
Operations
The module supports the following operations
Get File
Returns file information for a given id.
Send a JSON message to the module main address:
{
"action": "getFile",
"id": <id>
}
Where:
idis the UUID of the file. This field is mandatory.
An example would be:
{
"action": "getFile",
"id": "51d864754728011036adc575"
}
When the getFile completes successfully, a reply message is sent back to the sender with the following data:
{
"status": "ok",
"filename": <filename>,
"contentType": <contentType>,
"length": <length>,
"chunkSize": <chunkSize>,
"uploadDate": <uploadDate>,
"metadata": <metadata>
}
Where:
filenameis the filename provided when savingcontentTypeis the content type (ex. image/jpeg)lengthis the total file length in byteschunkSizeis the size in bytes of each chunkuploadDateis the long time of the upload in milliseconds since 1 Jan 1970metadatais an optional json object with additional metadata
If an error occurs in saving the document a reply is returned:
{
"status": "error",
"message": <message>
}
Where message is an error message.
Get Chunk
Returns a file chunk
Send a JSON message to the module main address:
{
"action": "getChunk",
"files_id": <files_id>,
"n": <n>,
"reply": <reply>
}
Where:
files_idis the UUID of the filenis the chunk number (first chunk is 0).replyis a boolean flag indicating a reply message handler should be added to send the next chunk
An example would be:
{
"action": "getChunk",
"files_id": "51d864754728011036adc575",
"n": 0,
"reply": true
}
When the get chunk completes successfully, a reply message with the chunk data byte[] in the message body is returned.
If an error occurs when getting the chunk, a json message is returned:
{
"status": "error",
"message": <message>
}
Where message is an error message.
Save File
Saves the file information.
Send a JSON message to the module main address:
{
"action": "saveFile",
"id": <id>,
"length": <length>,
"chunkSize": <chunkSize>,
"uploadDate": <uploadDate>,
"filename": <filename>,
"contentType": <contentType>
}
Where:
idis the UUID of the file.lengthis the total file length in byteschunkSizeis the size in bytes of each chunkuploadDateis the long time of the upload in milliseconds since 1 Jan 1970. The field is optional.filenameis the filename provided when saving. This field is optional.contentTypeis the content type (ex. image/jpeg). This field is optional (but recommended).
An example would be:
{
"action": "saveFile",
"id": "51d864754728011036adc575",
"length": 161966,
"chunkSize": 102400,
"contentType": "image/jpeg"
}
When the save completes successfully, a reply message is sent back to the sender with the following data:
{
"status": "ok"
}
If an error occurs when saving the file information a reply is returned:
{
"status": "error",
"message": <message>
}
Where message is an error message.
Save Chunk
Saves a chunk of binary data.
Send a Buffer message to the module main address + "/saveChunk".
The Buffer is made up of 3 parts:
- The first four bytes are an int defining the length of a UTF-8 encoded json string.
- The json bytes are next
- The remaining bytes are the chunk to be saved
The json contains the following fields:
{
"files_id": <files_id>,
"n": <n>
}
Where:
files_idis the UUID of the filenis the chunk number (first chunk is 0).
An example would be:
{
"files_id": "51d864754728011036adc575",
"n": 0
}
An example of generating the message would be:
public Buffer getMessage(String files_id, int chunkNumber, byte[] data) {
JsonObject jsonObject = new JsonObject()
.putString("files_id", files_id)
.putNumber("n", chunkNumber);
byte[] json = jsonObject.encode().getBytes("UTF-8");
byte[] data = getData(chunkNumber);
Buffer buffer = new Buffer();
buffer.appendInt(json.length);
buffer.appendBytes(json);
buffer.appendBytes(data);
return buffer;
}
When the save completes successfully, a reply message is sent back to the sender with the following data:
{
"status": "ok"
}
