com.intropro.prairie:units

Embedded Environment for Testing

License

License

GroupId

GroupId

com.intropro.prairie
ArtifactId

ArtifactId

units
Last Version

Last Version

1.2.0
Release Date

Release Date

Type

Type

pom
Description

Description

Embedded Environment for Testing

Download units

Filename Size
units-1.2.0.pom 929 bytes
Browse

How to add to project

<!-- https://jarcasting.com/artifacts/com.intropro.prairie/units/ -->
<dependency>
    <groupId>com.intropro.prairie</groupId>
    <artifactId>units</artifactId>
    <version>1.2.0</version>
    <type>pom</type>
</dependency>
// https://jarcasting.com/artifacts/com.intropro.prairie/units/
implementation 'com.intropro.prairie:units:1.2.0'
// https://jarcasting.com/artifacts/com.intropro.prairie/units/
implementation ("com.intropro.prairie:units:1.2.0")
'com.intropro.prairie:units:pom:1.2.0'
<dependency org="com.intropro.prairie" name="units" rev="1.2.0">
  <artifact name="units" type="pom" />
</dependency>
@Grapes(
@Grab(group='com.intropro.prairie', module='units', version='1.2.0')
)
libraryDependencies += "com.intropro.prairie" % "units" % "1.2.0"
[com.intropro.prairie/units "1.2.0"]

Dependencies

There are no dependencies for this project. It is a standalone project that does not depend on any other jars.

Project Modules

  • unit-common
  • hadoop-common
  • hdfs-unit
  • yarn-unit
  • hive2-unit
  • oozie-unit
  • flume-unit
  • zookeeper-unit
  • kafka-unit
  • pig-unit
  • cmd-unit
  • hbase-unit
  • cassandra-unit

#Prairie - Embedded Environment for Testing

###Overview This framework can help you to write tests for your big data and not only products. It supports majority modern BigData technologies.

###Look closer Framework consists of units. Unit is the service which responsible for one of the technologies(yarn, hive, etc). Also they have useful methods which can be use in you tests. Complex of units emulate cluster behavior. Units can have dependencies to another units. For example yarn depends on hdfs. You don't need to maintain this dependencies, they'll be created automatically.

####Example of hive unit with junit

@RunWith(PrairieRunner.class)
public class Hive2UnitTest {

    @PrairieUnit
    private Hive2Unit hive2Unit;

    @Test
    public void testForDemo() throws Exception {
        hive2Unit.execute("create table prairie_test_table (id bigint, name string)");
        hive2Unit.execute("insert into prairie_test_table values (1, 'first')");
        List<Map<String, String>> tableContent = hive2Unit.executeQuery("select * from prairie_test_table");
        Map<String, String> expectedRaw = new HashMap<>();
        expectedRaw.put("prairie_test_table.id", "1");
        expectedRaw.put("prairie_test_table.name", "first");
        Assert.assertEquals(expectedRaw, tableContent.get(0));
    }
}

More examples you can find in test directories for each unit:

Hdfs, Yarn, Hive2, Pig, Oozie, Flume, Zookeeper, Kafka Cmd HBase Cassandra

####Example of hive unit

public class Hive2UnitDemo {

    @PrairieUnit
    private Hive2Unit hive2Unit;

    public void runDemo() throws SQLException {
        hive2Unit.execute("create table prairie_test_table (id bigint, name string)");
        hive2Unit.execute("insert into prairie_test_table values (1, 'first')");
        List<Map<String, String>> tableContent = hive2Unit.executeQuery("select * from prairie_test_table");
        for (Map<String, String> row : tableContent) {
            System.out.println(row);
        }
    }

    public static void main(String[] args) throws PrairieException, SQLException {
        DependencyResolver dependencyResolver = new DependencyResolver();
        Hive2UnitDemo hive2UnitDemo = new Hive2UnitDemo();
        dependencyResolver.resolve(hive2UnitDemo);
        hive2UnitDemo.runDemo();
        dependencyResolver.destroy(hive2UnitDemo);
    }
}

###Maven To use one of units you need to put dependency to you pom.xml

<dependency>
    <groupId>com.intropro.prairie</groupId>
    <artifactId>${unit.id}</artifactId>
    <version>1.1.0</version>
</dependency>

and replace unit.id with data from table below

Unit ID Unit Version
hdfs-unit 1.2.0
yarn-unit 1.2.0
hive2-unit 1.2.0
oozie-unit 1.2.0
flume-unit 1.2.0
zookeeper-unit 1.2.0
kafka-unit 1.2.0
pig-unit 1.2.0
cmd-unit 1.2.0
hbase-unit 1.2.0
cassandra-unit 1.2.0

###Technologies

Technology Version
Hadoop 2.2.0 or higher
Hive 1.0.1 or higher
Oozie 4.0.0 or higher
Flume 1.5.0 or higher
Zookeeper 3.4.5 or higher
Kafka 0.8.2.2 or higher
Pig 0.12.0 or higher
HBase 1.2.0
Cassandra 3.9

###Supported Platforms

Platform Version
CDH 5.2.0 or higher
HDP 2.2.0 or higher

###JUnit Framework have junit runner PrairieRunner which initialize all units, and resolve inner dependencies. To use it you need to put dependency to your pom.xml:

<dependency>
    <groupId>com.intropro.prairie</groupId>
    <artifactId>junit-runner</artifactId>
    <version>1.2.0</version>
</dependency>

Licensing

This software is licensed under the terms in the file named "LICENSE" in this directory.

com.intropro.prairie

intropro

Versions

Version
1.2.0
1.1.0
1.0.1