RocksDb

RocksDb Store

The RocksDb store implementation will persist the application state to a RocksDb database within the local filesystem. To use it, ensure that the nstream.persist.store.rocksdb module is available on the module path. Additionally, it always requires the key-value adapter to be available on the module path.

Dependencies

Gradle

implementation 'io.nstream:nstream-persist-kernel:4.13.21'
implementation 'io.nstream:nstream-persist-kv:4.13.21'
implementation 'io.nstream:nstream-store-rocksdb:4.13.21'
implementation 'io.nstream:nstream-store-s3snapshot:4.13.21'

Maven

<dependency>
  <groupId>io.nstream</groupId>
  <artifactId>nstream-persist-kernel</artifactId>
  <version>4.13.21</version>
  <type>module</type> <!-- Remove or comment this line for non-modular projects -->
</dependency>
<dependency>
  <groupId>io.nstream</groupId>
  <artifactId>nstream-persist-kv</artifactId>
  <version>4.13.21</version>
  <type>module</type> <!-- Remove or comment this line for non-modular projects -->
</dependency>
<dependency>
  <groupId>io.nstream</groupId>
  <artifactId>nstream-store-rocksdb</artifactId>
  <version>4.13.21</version>
  <type>module</type> <!-- Remove or comment this line for non-modular projects -->
</dependency>
<dependency>
  <groupId>io.nstream</groupId>
  <artifactId>nstream-store-s3snapshot</artifactId>
  <version>4.13.21</version>
  <type>module</type> <!-- Remove or comment this line for non-modular projects -->
</dependency>

N.B. The dependency on nstream-store-s3snapshot is only required if snapshotting is used.

Configuration

The RocksDB store is configured as follows:

storeName: @store {
    implName: KeyValueAdapter
    parameters: {
        implName: RocksDB
        parameters: {
            path: "/path/to/db"
        }
    }
}

Here, /path/to/db is the path in the local filesystem where the database files will be located.

Snapshots

By default, the RocksDB database storing the state is kept in the local file system only. However, it is also possible to periodically copy snapshots to a remote repository. When the application is restarted, if a copy of the database is not available locally, it can be restored from the most recent snapshot. Currently, the only supported remote repository is S3 (or any other object store that uses the S3 protocol. Snapshot support is configured as follows:

storeName: @store {
    implName: KeyValueAdapter
    parameters: {
        implName: RocksDB
        parameters: {
            enableSnapshots: true
            snapshotInterval: 120000
            remoteSnapshotsRoot: "s3://bucket_name/snapshots/"
            snapshotStoreConfig: {
                # Parameters for the store implementation.
            }
            path: "/path/to/local/repository"
        }
    }
}

Setting enableSnapshots to true will cause the RocksDB tore to try to find a snapshotting implementation on the module/classpath. The URI specified in remoteSnapshotsRoot determines the location where the remote snapshots will be stored and is used to select an appropriate snapshotting implementation if multiple are available.

The path argument is used to specify where in the local filesystem the database should be stored. In contrast to the case without snapshots, the store will expect there to be sub-directories called db and checkpoints under the path. If they do not exist at startup, it will attempt to create them. On startup, if a snapshot exists in the remote repository, the store will attempt to restore it to the db directory. The checkpoints directory is used to create RocksDB checkpoints that are used to create the snapshots. By default, these checkpoints are deleted immediately after the snapshot completes.

The snapshotInterval parameter determines how frequently the snapshots are taken (in ms).

Configuration Options Table

The following additional parameters control how the store implementation uses RocksDB.

All of these directly determined parameters in RocksDB. For the details of what they do, consult the RocksDB documentation.

Additionally, when snapshots are enabled, the following additional parameters are available.

S3 Snapshots

The S3 snapshot implementation will be chosen for URIs of the form s3://bucket_name/path/to/snapshots/ where bucket_name is the name of the bucket to use and /path/to/snapshots is used as the prefix for all objects in the store. The S3 store is configured as follows:

snapshotStoreConfig: {
    tag: "server1",
    credentials: @basic {
        accessKey: "ACCESS_KEY"
        secretKey: "SECRET KEY"
    }
    region: "aws-global"
    endpoint: "http://host:9000"
}

The tag is an arbitrary string that is appended to the root path to distinguish snapshot repositories for different servers within a single store. ACCESS_KEY and SECRET_KEY should be replaced with your credentials to connect. The region selects the appropriate S3 region to use (this is irrelevant for other S3 compatible stores).

The endpoint parameter is optional. If it is not specified, AWS S3 will be assumed. Otherwise, the client will attempt to connect to an S3 compatible store, hosted at the URL specified here.