JarCastinghive_test: A simple way to test Hive scripts.
Usage
hive_test gives us an embedded Hive including an embedded Derby database, and a local HiveThriftService. This allows us to create unit tests very easily.
Hive scripts testing can be done in a similar way to PigUnit. For more examples check this class
@Test
public void testScriptListInput() {
Map<String, List<String>> input = Maps.<String, List<String>>newHashMap();
List<String> lines = Lists.newArrayList();
lines.add("msmith,10");
lines.add("mjohnson,2");
lines.add("mwilliamson,7");
lines.add("mjones,4");
lines.add("mdavies,5");
input.put("$INPUT1", lines);
Response output =
HiveBuilder.create()
.hiveTestWithEmbeddedHive("src/test/resources/scripts/passing-scores.hql")
.outputForInput(input);
List<String> expected = Lists.newArrayList();
expected.add("msmith,10");
expected.add("mwilliamson,7");
assertEquals(ResponseStatus.SUCCESS, output.getResponseStatus());
assertEquals(expected, output.getResult());
}
Maven
Available from Maven central
Declare the Maven dependency in your pom.xml file
<dependencies>
...
<dependency>
<groupId>io.teknek</groupId>
<artifactId>hiveunit</artifactId>
<version>0.0.3</version>
<classifier>jar-with-dependencies</classifier>
</dependency>
...
<dependencies>
Execution profiles
Without a local Hadoop install
By default, we're set to download a local copy of Hadoop when you first build Hive Test, or whenever the project is cleaned, just before running our test cases.
You can force a redownload and installation of Hadoop by manually activating the download-hadoop profile
mvn --activate-profiles download-hadoop test
You can also perform the download and extraction process independent of testing.
Download Hadoop (into the maven target directory)
mvn --activate-profiles download-hadoop wagon:download-single
Extract Hadoop (into the maven target directory)
mvn --activate-profiles download-hadoop exec:exec
With a local Hadoop install
We'll skip attempting to download and use a local copy of Hadoop if any of the following are true
- set your HADOOP_HOME environment variable to a hadoop distribution
- hadoop tar extracted to $home/hadoop/hadoop-0.20.2_local
Hive Test will work so long as you have Hadoop in your path, i.e. /usr/bin/hadoop. In this case, you'll want to deactivate the hadoop download.
mvn --activate-profiles -hadoop-home-defined test
Contribute!
Contributions are welcome! You can contribute by
- starring this repo!
- adding new features
- enhancing existing code
- testing
- enhancing documentation
- bringing suggestions and reporting bugs
- spreading the word / telling us how you use it!