Gradoop Accumulo Store

Accumulo Store for Gradoop

Apache Accumulo is a key/value store based on the design of Google's BigTable.

With this adapter implementation you can use Apache Accumulo as DataSource or DataSink for your graph data.

Getting start

1. Adding a Accumulo Runtime Iterator

Before using gradoop accumulo, you must put gradoop-accumulo-iterator.jar into your accumulo runtime library.

Run the instructions below to build your accumulo runtime library:

cd gradoop-store/gradoop-accumulo && mvn clean install

Then copy gradoop-accumulo/target/iterator/*.jar to your accumulo runtime library path (local filesystem or hdfs).

If you use a native external lib , just copy it to $ACCUMULO_HOME/lib/ext.

For more details about accumulo's iterator setting, please visit Apache Accumulo Manual.

2. Using gradoop accumulo

Compile gradoop accumulo with

mvn clean install -DskipTests=true

Copy gradoop-accumulo/target/gradoop-accumulo-<ver>.jar into your client lib.

Or you can simply use maven pom as below:

<!-- Maven Gradoop Accumulo -->
<dependency>
    <groupId>org.gradoop</groupId>
    <artifactId>gradoop-accumulo</artifactId>
    <version>${gradoop.version}</version>
</dependency>

Creation of an Accumulo based Graph-store

// flink execution env
ExecutionEnvironment env = ExecutionEnvironment.createLocalEnvironment();

// create gradoop accumulo configuration
GradoopAccumuloConfig config = GradoopAccumuloConfig.create(env)
  .set(GradoopAccumuloConfig.ACCUMULO_USER, {user})
  .set(GradoopAccumuloConfig.ACCUMULO_INSTANCE, {instance})
  .set(GradoopAccumuloConfig.ZOOKEEPER_HOSTS, {comma separated zookeeper host list})
  .set(GradoopAccumuloConfig.ACCUMULO_PASSWD, {password})
  .set(GradoopAccumuloConfig.ACCUMULO_TABLE_PREFIX, {table prefix});

// create store
AccumuloStore graphStore = new AccumuloStore(config);


> let's just add some graph elements

graphStore.writeGraphHead(graphHead); graphStore.wirteVertex(vertex); graphStore.writeEdge(edge);

graphStore.flush();

## Accessing Data
> Example for DataSink & DataSource

// data source DataSource accumuloDataSource = new AccumuloDataSource(config);

GraphCollection result = accumuloDataSource.cypher( "MATCH (u1:Person)<-[:hasModerator]-(f:Forum)" + "(u2:Person)<-[:hasMember]-(f)" + "WHERE u1.name = "Alice"");

// data sink DataSink accumuloSink = new AccumuloDataSink(config);

accumuloSink.write(result);

## Store Layout

Gradoop store instances(`GradoopAccumuloStore`) are divided by their table name prefix (`GradoopAccumuloConfig.ACCUMULO_TABLE_PREFIX`)

GraphData (Table graph)

------------------------------------------------------------------------------------ row | cf | cq | timestamp | value ------------------------------------------------------------------------------------ | label | | | {label} {id} -------------------------------------------------------------------------- | property | property key | | {property} ------------------------------------------------------------------------------------

VertexData (Table vertex)

------------------------------------------------------------------------------------ row | cf | cq | timestamp | value ------------------------------------------------------------------------------------ | label | | | {label} -------------------------------------------------------------------------- {id} | property | property key | | {property} -------------------------------------------------------------------------- | graph | {graph id} | | ------------------------------------------------------------------------------------

EdgeData (Table edge)

------------------------------------------------------------------------------------ row | cf | cq | timestamp | value ------------------------------------------------------------------------------------ | label | | | {label} -------------------------------------------------------------------------- | source | | | {vertex id} -------------------------------------------------------------------------- {id} | target | | | {vertex id} -------------------------------------------------------------------------- | property | property key | | {property} -------------------------------------------------------------------------- | graph | {graph id} | | ------------------------------------------------------------------------------------