com.homeaway.streamplatform:stream-registry-core

Stream Registry Parent POM

License

License

Categories

Categories

ORM Data
GroupId

GroupId

com.homeaway.streamplatform
ArtifactId

ArtifactId

stream-registry-core
Last Version

Last Version

0.5.2
Release Date

Release Date

Type

Type

jar
Description

Description

com.homeaway.streamplatform:stream-registry-core
Stream Registry Parent POM

Download stream-registry-core

How to add to project

<!-- https://jarcasting.com/artifacts/com.homeaway.streamplatform/stream-registry-core/ -->
<dependency>
    <groupId>com.homeaway.streamplatform</groupId>
    <artifactId>stream-registry-core</artifactId>
    <version>0.5.2</version>
</dependency>
// https://jarcasting.com/artifacts/com.homeaway.streamplatform/stream-registry-core/
implementation 'com.homeaway.streamplatform:stream-registry-core:0.5.2'
// https://jarcasting.com/artifacts/com.homeaway.streamplatform/stream-registry-core/
implementation ("com.homeaway.streamplatform:stream-registry-core:0.5.2")
'com.homeaway.streamplatform:stream-registry-core:jar:0.5.2'
<dependency org="com.homeaway.streamplatform" name="stream-registry-core" rev="0.5.2">
  <artifact name="stream-registry-core" type="jar" />
</dependency>
@Grapes(
@Grab(group='com.homeaway.streamplatform', module='stream-registry-core', version='0.5.2')
)
libraryDependencies += "com.homeaway.streamplatform" % "stream-registry-core" % "0.5.2"
[com.homeaway.streamplatform/stream-registry-core "0.5.2"]

Dependencies

compile (33)

Group / Artifact Type Version
com.homeaway.streamplatform : schema-library-events jar 0.5.2
org.slf4j : slf4j-api jar 1.7.25
com.google.guava : guava jar 23.4-jre
org.apache.commons : commons-lang3 jar 3.8.1
javax.validation : validation-api jar 1.1.0.Final
io.dropwizard : dropwizard-core jar
io.dropwizard : dropwizard-configuration jar
io.dropwizard : dropwizard-lifecycle jar
io.dropwizard : dropwizard-util jar
io.dropwizard : dropwizard-client jar
io.dropwizard : dropwizard-jetty jar
io.dropwizard : dropwizard-jersey jar
io.dropwizard : dropwizard-jackson jar
io.dropwizard.metrics : metrics-core jar
io.dropwizard.metrics : metrics-healthchecks jar
io.dropwizard.metrics : metrics-servlets jar
io.dropwizard.metrics : metrics-annotation jar
com.fasterxml.jackson.core : jackson-databind jar
com.fasterxml.jackson.core : jackson-annotations jar
com.fasterxml.jackson.datatype : jackson-datatype-guava jar
org.apache.kafka : kafka_2.11 jar 1.1.1-cp2
org.apache.kafka : kafka-clients jar 1.1.1-cp2
org.apache.kafka : kafka-streams jar 1.1.1-cp2
org.rocksdb : rocksdbjni jar 5.7.3
com.101tec : zkclient jar 0.10
org.apache.avro : avro jar 1.8.2
io.confluent » kafka-avro-serializer jar 4.1.3
io.confluent » kafka-streams-avro-serde jar 4.1.3
io.confluent » kafka-schema-registry-client jar 4.1.3
io.swagger : swagger-annotations jar 1.5.21
io.swagger : swagger-jersey2-jaxrs jar 1.5.21
com.smoketurner : dropwizard-swagger jar 1.1.2-1
javax.ws.rs : javax.ws.rs-api jar 2.1

provided (1)

Group / Artifact Type Version
org.projectlombok : lombok jar 1.18.4

runtime (1)

Group / Artifact Type Version
javax.xml.bind : jaxb-api jar 2.4.0-b180830.0359

test (12)

Group / Artifact Type Version
junit : junit jar
org.hamcrest : hamcrest-core jar 1.3
org.hamcrest : hamcrest-library jar 1.3
org.mockito : mockito-core jar 2.23.4
io.dropwizard : dropwizard-testing jar
org.apache.curator : curator-test jar 2.9.0
org.apache.kafka : kafka_2.11 jar 1.1.1-cp2
org.apache.kafka : kafka-clients jar 1.1.1-cp2
org.apache.kafka : kafka-streams jar 1.1.1-cp2
io.confluent » kafka-schema-registry jar 4.1.3
net.bytebuddy : byte-buddy-agent jar 1.8.5
net.bytebuddy : byte-buddy jar 1.8.5

Project Modules

There are no modules declared in this project.

stream-registry Build Status

StreamRegistryLogo

Announcement: 12th April 2019

We wanted to let you know that there are going to be some exciting developments with the Stream Registry project in the very near future. Stream Registry is being adopted by many brands at Expedia Group as a critical component of its digital nervous system for key streams across Expedia Group. Therefore, Vrbo stream registry is finding a new home.

What is changing

  • We will be investing in the project by expanding the existing team with full-time resources in several locations across Expedia Group. Expect greatly increased project activity: contributors, commits, issues, features, releases
  • The repository will relocate to the ExpediaGroup open source GitHub org in its entirety, preserving the history and community

What isn't changing

  • The original vision of Stream Registry as a Stream Discovery and Stream Orchestration platform
  • The project will remain open source, and will be joined shortly by other supporting Expedia Group stream platform components
  • Licenses, conduct and contribution guidelines will remain unchanged
  • The value of your contributions - please keep them coming!

We expect the start of this journey to be a little bumpy, but please bear with us as we work towards the first release of the Expedia Group Stream Registry!

About

A Stream Registry is what its name implies: it is a registry of streams. As enterprises increasingly scale in size, the need to organize and develop around streams of data becomes paramount. Synchronous calls are attracted to the edge, and a variety of synchronous and asynchronous calls permeate the enterprise. The need for a declarative, central authority for discovery and orchestration of stream management emerges. This is what a stream registry provides. In much the same way that DNS provides a name translation service for an ip address, by way of analogy, a Stream Registry provides a “metadata service” for streams. By centralizing stream metadata, a stream translation service for producer and/or consumer stream coördinates becomes possible. This centralized, yet democratized, stream metadata function thus streamlines operational complexity via stream lifecycle management, stream discovery, stream availability and resiliency.

Why Stream Registry?

We believe that as the change to business requirements accelerate, time to market pressures increase, competitive measures grow, migrations to cloud and different platforms are required, and so on, systems will increasingly need to become more reactive and dynamic in nature.

The issue of state arises.

We see many systems adopting event-driven-architectures to facilitate the changing business needs in these high stakes environments. We hypothesize there is an emerging need for a centralized "stream metadata" service in the industry to help streamline the complexities and operations of deploying stream platforms that serve as a distributed federated nervous system in the enterprise.

What is Stream Registry?

Put simply, Stream Registry is a centralized service for stream metadata.

The stream registry can answer the following question:

  1. Who owns the stream?
  2. Who are the producers and consumers of the stream?
  3. Management of stream replication across clusters and regions
  4. Management of stream storage for permanent access
  5. Management of stream triggers for legacy stream sources

Architecture

StreamRegistryArchitecture

See the architecture/northstar documentation for more details.

Building locally

Stream Registry is built using OpenJDK 11 and Maven.

Stream Registry is currently packaged as a shaded JAR file. We leave specific deployment considerations up to each team since this varies from enterprise to enterprise.

To build Stream Registry as a JAR file, please run

./mvnw clean package

Start Stream Registry

Required Local Environment
The local 'dev' version of Stream Registry requires a locally running version of Apache Kafka and Confluent's Schema Registry on ports 9092 and 8081, respectively.

To quickly get a local dev environment set up, we recommend to use Docker Compose.

Alternatively, one can start Confluent Platform locally after downloading the Confluent CLI and running the following command. Note: The confluent command is currently only available for macOS and Linux. If using Windows, you'll need to use Docker, or run ZooKeeper, Kafka, and the Schema Registry all individually.

confluent start zookeeper
confluent start kafka
confluent start schema-registry

Stream Registry can then be started.

Once Stream Registry has started, check that the application's GraphiQL server is running at http://localhost:8080/graphiql

Kafka Version Compatibility

Stream Registry development and initial deployment started with Kafka 0.11.0 / Confluent Platform 3.3.0, and has also been deployed against Kafka 1.1.1 / Confluent Platform 4.1.2.
As per the Kafka Compatibility Matrix, we expect Stream Registry to be compatbile with Kafka 0.10.0 and newer, and the internal Java Kafka clients used by Stream Registry can be found in the pom.xml.

Run Unit Tests

./mvnw clean test

Contributors

Special thanks to the following for making stream-registry possible at Vrbo and beyond!

Adam Westerman
Adam Westerman

💻
Arun Vasudevan
Arun Vasudevan

💻 🎨
Nathan Walther
Nathan Walther

💻 👀
Jordan Moore
Jordan Moore

💻 💁
Carlos Cordero
Carlos Cordero

💻
Ishan Dikshit
Ishan Dikshit

💻 📖
Vinayak Ponangi
Vinayak Ponangi

💻 📢 🎨 👀
Prabhakaran Thatchinamoorthy
Prabhakaran Thatchinamoorthy

💻 🎨
Rui Zhang
Rui Zhang

💻
Miguel Lucero
Miguel Lucero

💻 💁
René X Parra
René X Parra

💻 📖 📝 📢 🎨 👀

This project follows the all-contributors specification.

Legal

This project is available under the Apache 2.0 License.

Copyright 2018-2019 Expedia, Inc.

com.homeaway.streamplatform

HomeAway

Versions

Version
0.5.2
0.5.1
0.5.0
0.4.10
0.4.9
0.4.8
0.4.7
0.4.6
0.4.5
0.4.4
0.4.3
0.4.2