JarCasting
JarCasting JarCasting

legstar.avro.distrib

Translates COBOL structures to Avro schemas. Converts Mainframe data to Avro records.

License

License
GroupId

GroupId

com.legsem.legstar
ArtifactId

ArtifactId

legstar.avro.distrib
Last Version

Last Version

0.4.2
Release Date

Release Date
Type

Type

jar
Description

Description

legstar.avro.distrib
Translates COBOL structures to Avro schemas. Converts Mainframe data to Avro records.
Project Organization

Project Organization

LegSem

Download legstar.avro.distrib

Filename Size
legstar.avro.distrib-0.4.2.pom
legstar.avro.distrib-0.4.2.zip 29 MB
legstar.avro.distrib-0.4.2-sources.jar 40 KB
legstar.avro.distrib-0.4.2-javadoc.jar 225 KB
Browse

How to add to project

<!-- https://jarcasting.com/artifacts/com.legsem.legstar/legstar.avro.distrib/ -->
<dependency>
    <groupId>com.legsem.legstar</groupId>
    <artifactId>legstar.avro.distrib</artifactId>
    <version>0.4.2</version>
</dependency>
// https://jarcasting.com/artifacts/com.legsem.legstar/legstar.avro.distrib/
implementation 'com.legsem.legstar:legstar.avro.distrib:0.4.2'
// https://jarcasting.com/artifacts/com.legsem.legstar/legstar.avro.distrib/
implementation ("com.legsem.legstar:legstar.avro.distrib:0.4.2")
'com.legsem.legstar:legstar.avro.distrib:jar:0.4.2'
<dependency org="com.legsem.legstar" name="legstar.avro.distrib" rev="0.4.2">
  <artifact name="legstar.avro.distrib" type="jar" />
</dependency>
@Grapes(
@Grab(group='com.legsem.legstar', module='legstar.avro.distrib', version='0.4.2')
)
libraryDependencies += "com.legsem.legstar" % "legstar.avro.distrib" % "0.4.2"
[com.legsem.legstar/legstar.avro.distrib "0.4.2"]

Dependencies

compile (5)

Group / Artifact Type Version
com.legsem.legstar : legstar.avro.generator jar 0.4.2
com.legsem.legstar : legstar.avro.cob2avro jar 0.4.2
com.legsem.legstar : legstar.avro.cob2avro.hadoop jar 0.4.2
org.apache.hadoop : hadoop-common jar 2.4.1
org.apache.hadoop : hadoop-mapreduce-client-core jar 2.4.1

test (1)

Group / Artifact Type Version
junit : junit jar 4.11

Project Modules

There are no modules declared in this project.

legstar.avro

A COBOL to Apache Avro translator based on LegStar

The idea is that mainframe data might be useful in environments where Apache Avro records are commonplace. One such environment is Hadoop where MapReduce jobs can read/write Avro records.

Legstar.avro helps deliver mainframe data as Avro records.

Objectives

  • Provide a Generator to produce artifacts from a COBOL copybook:

    • Avro Schema matching the COBOL copybook

    • Conversion logic to turn mainframe records to Avro Generic records

    • Avro specific records compiled from the Avro schema

  • Readers for pure Java, or Hadoop, that read mainframe files and deliver an Avro record for each mainframe record

Overview

Requirements

  • Java JDK 6 and above (It is important that this is a JDK not a simple JRE)

  • Maven 3 (If you build from sources)

  • Ant 1.9.x (To run samples)

  • Hadoop 2.4.1 (To run the Hadoop sample)

Build from sources

  1. Clone the GIT repository

  2. From a command window, while located in the folder where you cloned the repo, type:

mvn clean install

Run the samples

If you built the project from sources, you will find the distribution zip file under legstar.avro/legstar.avro.distrib/target.

Otherwise you can get the latest released zip here.

Unzip the zip file in a location of your choice.

1. Run the CustdatReader sample

Go to the samples folder and type:

ant

You can get more information on the CustdatReader sample provided on the wiki

2. Run the CustdatHadoopReader sample

Go to the samples folder and type:

ant -f build-hadoop.xml

You can get more information on the CustdatHadoopReader sample provided on the wiki

Versions

Version
0.4.2
0.4.1
0.3.0