JarCasting#Prairie - Embedded Environment for Testing
###Overview This framework can help you to write tests for your big data and not only products. It supports majority modern BigData technologies.
###Look closer Framework consists of units. Unit is the service which responsible for one of the technologies(yarn, hive, etc). Also they have useful methods which can be use in you tests. Complex of units emulate cluster behavior. Units can have dependencies to another units. For example yarn depends on hdfs. You don't need to maintain this dependencies, they'll be created automatically.
####Example of hive unit with junit
@RunWith(PrairieRunner.class)
public class Hive2UnitTest {
@PrairieUnit
private Hive2Unit hive2Unit;
@Test
public void testForDemo() throws Exception {
hive2Unit.execute("create table prairie_test_table (id bigint, name string)");
hive2Unit.execute("insert into prairie_test_table values (1, 'first')");
List<Map<String, String>> tableContent = hive2Unit.executeQuery("select * from prairie_test_table");
Map<String, String> expectedRaw = new HashMap<>();
expectedRaw.put("prairie_test_table.id", "1");
expectedRaw.put("prairie_test_table.name", "first");
Assert.assertEquals(expectedRaw, tableContent.get(0));
}
}
More examples you can find in test directories for each unit:
Hdfs, Yarn, Hive2, Pig, Oozie, Flume, Zookeeper, Kafka Cmd HBase Cassandra
####Example of hive unit
public class Hive2UnitDemo {
@PrairieUnit
private Hive2Unit hive2Unit;
public void runDemo() throws SQLException {
hive2Unit.execute("create table prairie_test_table (id bigint, name string)");
hive2Unit.execute("insert into prairie_test_table values (1, 'first')");
List<Map<String, String>> tableContent = hive2Unit.executeQuery("select * from prairie_test_table");
for (Map<String, String> row : tableContent) {
System.out.println(row);
}
}
public static void main(String[] args) throws PrairieException, SQLException {
DependencyResolver dependencyResolver = new DependencyResolver();
Hive2UnitDemo hive2UnitDemo = new Hive2UnitDemo();
dependencyResolver.resolve(hive2UnitDemo);
hive2UnitDemo.runDemo();
dependencyResolver.destroy(hive2UnitDemo);
}
}
###Maven To use one of units you need to put dependency to you pom.xml
<dependency>
<groupId>com.intropro.prairie</groupId>
<artifactId>${unit.id}</artifactId>
<version>1.1.0</version>
</dependency>
and replace unit.id with data from table below
| Unit ID | Unit Version |
|---|---|
| hdfs-unit | 1.2.0 |
| yarn-unit | 1.2.0 |
| hive2-unit | 1.2.0 |
| oozie-unit | 1.2.0 |
| flume-unit | 1.2.0 |
| zookeeper-unit | 1.2.0 |
| kafka-unit | 1.2.0 |
| pig-unit | 1.2.0 |
| cmd-unit | 1.2.0 |
| hbase-unit | 1.2.0 |
| cassandra-unit | 1.2.0 |
###Technologies 
| Technology | Version |
|---|---|
| Hadoop | 2.2.0 or higher |
| Hive | 1.0.1 or higher |
| Oozie | 4.0.0 or higher |
| Flume | 1.5.0 or higher |
| Zookeeper | 3.4.5 or higher |
| Kafka | 0.8.2.2 or higher |
| Pig | 0.12.0 or higher |
| HBase | 1.2.0 |
| Cassandra | 3.9 |
###Supported Platforms
| Platform | Version |
|---|---|
| CDH | 5.2.0 or higher |
| HDP | 2.2.0 or higher |
###JUnit Framework have junit runner PrairieRunner which initialize all units, and resolve inner dependencies. To use it you need to put dependency to your pom.xml:
<dependency>
<groupId>com.intropro.prairie</groupId>
<artifactId>junit-runner</artifactId>
<version>1.2.0</version>
</dependency>
Licensing
This software is licensed under the terms in the file named "LICENSE" in this directory.
