io.backpackcloud:fakeomatic

Fake-O-Matic is a library for generating and ingesting fake data.

License

License

GroupId

GroupId

io.backpackcloud
ArtifactId

ArtifactId

fakeomatic
Last Version

Last Version

0.9.1
Release Date

Release Date

Type

Type

jar
Description

Description

Fake-O-Matic is a library for generating and ingesting fake data.
Project URL

Project URL

https://github.com/backpackcloud/fakeomatic
Source Code Management

Source Code Management

https://github.com/backpackcloud/fakeomatic

Download fakeomatic

How to add to project

<!-- https://jarcasting.com/artifacts/io.backpackcloud/fakeomatic/ -->
<dependency>
    <groupId>io.backpackcloud</groupId>
    <artifactId>fakeomatic</artifactId>
    <version>0.9.1</version>
</dependency>
// https://jarcasting.com/artifacts/io.backpackcloud/fakeomatic/
implementation 'io.backpackcloud:fakeomatic:0.9.1'
// https://jarcasting.com/artifacts/io.backpackcloud/fakeomatic/
implementation ("io.backpackcloud:fakeomatic:0.9.1")
'io.backpackcloud:fakeomatic:jar:0.9.1'
<dependency org="io.backpackcloud" name="fakeomatic" rev="0.9.1">
  <artifact name="fakeomatic" type="jar" />
</dependency>
@Grapes(
@Grab(group='io.backpackcloud', module='fakeomatic', version='0.9.1')
)
libraryDependencies += "io.backpackcloud" % "fakeomatic" % "0.9.1"
[io.backpackcloud/fakeomatic "0.9.1"]

Dependencies

compile (11)

Group / Artifact Type Version
io.quarkus : quarkus-resteasy jar
io.quarkus : quarkus-resteasy-qute jar
io.quarkus : quarkus-resteasy-jackson jar
io.quarkus : quarkus-rest-client-jackson jar
io.quarkus : quarkus-rest-client jar
io.quarkus : quarkus-smallrye-metrics jar
io.vertx : vertx-web-client jar
io.smallrye.reactive : smallrye-mutiny-vertx-web-client jar
io.quarkus : quarkus-vertx jar
io.quarkus : quarkus-picocli jar
com.fasterxml.jackson.dataformat : jackson-dataformat-yaml jar 2.10.4

test (2)

Group / Artifact Type Version
io.quarkus : quarkus-junit5 jar
io.quarkus : quarkus-junit5-mockito jar

Project Modules

There are no modules declared in this project.

Fake-O-Matic

Fake-O-Matic is a utility tool for creating and sending fake payloads to endpoints in order to test throughput. On each iteration, Fake-O-Matic will generate a payload based on a provided template and a set of rules for generating fake data.

Fake-O-Matic is published in Quay.io, and the library is published in Central Maven Repo under the following properties:

groupId

io.backpackcloud

artifactId

fakeomatic

Quick Example

Navigate to https://requestbin.com/ and create a request bin (uncheck the private option and choose the old bin). Copy the Endpoint url.

Now build a JVM version of Fake-O-Matic with mvn package and run it (yeah, I know…​ downloading the internet is not quick):

$ java -jar target/fakeomatic-$VERSION-runner.jar \
      --endpoint-url https://YOUR_REQUEST_BIN_ID.x.pipedream.net/ \
      --template examples/example1.json
Note
You may also download the binaries from the releases page.

How to Build

Fake-O-Matic is build with Quarkus and supports native compilation. THe build is a good and old mvn package command. If you wish to build a native binary, just activate the native profile (mvn -P native package).

It’s also possible to build Fake-O-Matic using source-to-image:

s2i build . quay.io/quarkus/ubi-quarkus-native-s2i:19.3.2-java11 backpackcloud/fakeomatic

How to Configure

Fake-O-Matic is configured through a set of yaml files that forms a hierarchy. If some configuration is not found on one file, then the next file is used. That also means that you can override configurations by leaving the default values as the last in the hierarchy.

A Fake-O-Matic configuration has:

endpoints: # (1)
  endpoint_name:
    # configuration ...

samples: # (2)
  sample_name:
    # configuration

placeholders: # (3)
  "PLACEHOLDER_CHARACTER": sample_name
  1. At least one endpoint

  2. Ideally as much samples as needed

  3. Optional placeholders for creating expressions

The configuration for each part is covered through this document.

Configuration Values

Some configuration values can be externalized by different ways additionally to the normal String value:

env

Fetches the value from an Environment Variable.

param:
  env: PARAM # reads the value from the PARAM environment variable (export PARAM=foo)
property

Fetches the value from a System Property.

param:
  property: my.parameter # reads the value from the my.parameter system property (-Dmy.parameter=foo)
file

Reads the value from a file.

param:
  file: /tmp/myfile # reads the value from the /tmp/myfile content
resource

Reads the value using a class.getResourceAsStream.

param:
  resource: /com/example/configuration.yaml # reads the value from the configuration.yaml
                                            # file inside the com.example package
url

Reads the value from the given URL by using GET url.

param:
  url: http://something.example.com/some-param
value

Supplies a raw value.

param:
  value: bar # uses "bar" as the value if the PARAM environment variable is not present

It is also possible to combine different approaches for getting the value by passing all the desired options. The first one that holds a value will be used (see the priority bellow):

param:
  env: PARAM
  property: my.parameter
  value: bar

The priority for getting the value is:

  1. env

  2. property

  3. file

  4. resource

  5. url

  6. value

It’s also possible to supply a raw value without specifying the type value if it’s the only value:

param: some value # implies a configuration of type "value"

Sample as Configuration

Some samples take other samples as configuration. In most cases, there are two ways of passing a sample as configuration:

  1. Referring through a ref property.

  2. Directly through a sample property.

Endpoint Connections

An endpoint is primarily used to send generated payloads but can also be used as a source of data for the samples.

Configuration

url

A configuration value defining the API endpoint.

method

Which HTTP method to use (defaults to GET).

payload

An optional payload object to use for calling the API. Useful for POST requests.

template

A configuration defining which template to use

content-type

The content type of the template (defaults to application/json).

insecure

Defines if the certificates should be trusted without checking (defaults to false).

concurrency

A configuration int value that defines the max ongoing connections to this endpoint. (Defaults to 10.)

buffer

A configuration int value that defines how many requests can be enqueued when the max ongoing requests number is reached. (Defaults to 10.)

This is useful in cases where generating the payloads can take more time than it takes to send them (like depending on several external APIs).

headers

A string,configuration map containing the headers to use.

params

A string,configuration map containing all the values that forms the url (example: /api/{uuid} will be replaced by the value from the path_var uuid).

Built In

Fake-O-Matic defines a default endpoint which can be customized via configuration values:

endpoints:
  default:
    url:
      env: ENDPOINT_URL
      property: endpoint.url
      value: http://localhost:8080
    payload:
      content-type:
        env: ENDPOINT_CONTENT_TYPE
        property: endpoint.content_type
        value: application/json
      template:
        env: ENDPOINT_TEMPLATE
        property: endpoint.template
        resource: /META-INF/resources/config/payload.json
    method:
      env: ENDPOINT_METHOD
      property: endpoint.method
      value: POST
    concurrency:
      env: ENDPOINT_CONCURRENCY
      property: endpoint.concurrency
      value: 10
    buffer:
      env: ENDPOINT_BUFFER
      property: endpoint.buffer
      value: 10
    insecure:
      env: ENDPOINT_INSECURE
      property: endpoint.insecure
      value: false

Please see Configuration Properties for more information.

Data Generation

The rules for generating data can contain sample and placeholders. Samples are a set of data that Fake-O-Matic can randomly pick and placeholders are characters that can be associated with the samples for to allow the use of expressions for generating data.

Samples

Characters

This sample can pick any character from a given string. Useful for defining a set of characters that can be used to produce IDs or any other information that is not meant to be read.

Configuration
type

chars

value

String that holds the chars

Example
samples:
  letter:
    type: chars
    value: "abcdefghijklmnopqrstuvwxyz"
  digit:
    type: chars
    value: "0123456789"
List

This sample can pick any item from a given list of objects.

Configuration
type

list

values

List of raw values to use.

samples

List of Samples to use.

source

A configuration pointing to where to locate the list of values.

Note
You need to supply only one way of loading the values (values, sample or source).
Example
samples:
  cause:
    type: list
    values:
      - "clock speed"
      - "solar flares"
      - "electromagnetic radiation from satellite debris"
      - "static from nylon underwear"
      - "static from plastic slide rules"
      - "global warming"
      - "poor power conditioning"
      - "static buildup"
      - "doppler effect"
  first_name:
    type: list
    samples:
      - man_name
      - woman_name
  story:
    type: list
    source:
      env: STORIES
      property: stories
      default: stories.txt
API

This sample actually calls a given API to get data to use every time it’s asked for a data.

Warning
Due to the nature of this sample, it’s not possible to reproduce the same payloads without relying on the dependent API.
Configuration
type

api

endpoint

An endpoint configuration for the api to use.

Note
Although this sample uses an Endpoint object, which supports concurrency and buffers, the sample is synchronous so configuring those values will not have any effect on this sample.
Example
samples:
  chuck_norris:
    type: api
    endpoint:
      url: https://api.chucknorris.io/jokes/random
  example:
    type: api
    endpoint:
      url: https://api.example.com/{version}/some/path
      headers:
        token:
          env: EXAMPLE_API_TOKEN
          property: example.api.token
      params:
        version:
          env: EXAMPLE_API_VERSION
          property: example.api.version
          value: v1
JSON Pointer

This sample extracts a value from a JSON object. Useful when used in combination with the API Sample.

Configuration
type

json

path

A JSON Pointer to specify which value to take from the JSON object.

source

Which sample sample should be used to obtain the JSON object.

Example
samples:
  chuck_norris:
    type: json
    source:
      sample:
        type: api
        endpoint:
          url: https://api.chucknorris.io/jokes/random
    path: /value
Universally Unique Identifier

This sample will produce a universally unique identifier.

Configuration
type

uuid

Example
samples:
  uuid:
    type: uuid
Join

This sample will gather other samples and join them into one data.

Configuration
type

join

samples

Which samples to join

separator

Which separator to use (defaults to an empty string).

Example
samples:
  full_name:
    type: join
    separator: " "
    samples:
      - first_name
      - last_name
Weight

A sample that allows you to define specific weights to each element.

Configuration
type

weight

values

The list of values.

weight

The weight of the value.

value

The value to use.

source

Which sample sample should be used instead of the value.

Example
samples:
  color:
    type: weight
    values:
      - weight: 30
        value: blue
      - weight: 45
        value: yellow
      - weight: 10
        value: red
      - weight: 20
        value: brown
      - weight: 25
        value: cyan
Note
The sum of the weights don’t necessary need to be 100, but using a total weight of 100 helps to see the weights as percentage.
Range

This sample generates numbers from a given interval.

Configuration
type

range

min

The minimum value.

max

The maximum value.

Example
samples:
  grade:
    type: range
    min: 0
    max: 10
  temperature:
    type: range
    min: -10
    max: 20
Expression

A sample that generates data based on expressions.

Configuration
type

expression

sample

The sample to use as an expression.

expression

The expression to use.

Note
You need to supply a sample or an expression.
Example
samples:
  address_expression:
    type: list
    values:
      - "Some Street ##"
      - "Another Street ###"
      - "Galaxy ###"
  address:
    type: expression
    sample: address_expression
  credit_card:
    type: expression
    expression: "################"
Date

A sample that can generate dates based on a given interval.

Configuration
type

date

from

Defines the start date.

to

Defines the end date.

period

Defines a period instead of an end date. See the docs for java.time.Period#parse.

format

The format to parse the supplied dates. Defaults to dd-MM-yyyy. See the docs for java.time.format.DateTimeFormatter#ofPatter.

inclusive

Sets if the end date is part of the interval or not.

Note
It is possible to use today, yesterday or tomorrow instead of the actual date values.
Example
samples:
  day_in_2020:
    type: date
    from: 2020-01-01
    to: 2021-01-01
  day_in_quarter:
    type: date
    from: 2020-01-01
    period: P3M
  yesterday_to_tomorrow:
    type: date
    from: yesterday
    to: tomorrow
    inclusive: true
Cache

A sample that caches the value. Useful when used with an API sample that posts data in order to create a data dependency.

Configuration
type

cache

source

Which sample sample should be used to obtain the value to cache.

ttl

How many hits the value should last until the cache gets another one. This is a configuration value.

Example
samples:
  new_person:
    type: api
    url: http://api.example.com/persons
    method: POST
    payload:
      template: person.json
    return: /id
  person:
    type: cache
    source:
      ref: new_person

Placeholders

The placeholders are a single character that can be associated with any of the configured sample. Bellow is an example of a configuration file:

placeholders:
  "#": digit   (1)
  "%": letter  (2)

samples:
  letter:
    type: chars
    value: "abcdefghijklmnopqrstuvwxyz"
  digit:
    type: chars
    value: "0123456789"
  1. Associated with the digit sample

  2. Associated with the letter sample

Payload Template

Fake-O-Matic uses Qute templates to produce the payloads. A couple of methods can be used to get a fake data, the main one are:

some(sampleName)

Gets a random sample from the given sample name.

expression(placeholders)

Gets a random data produced by replacing each placeholder by a random sample associated with it.

Configuration Properties

The following properties can be configured as a JVM argument (prefix -D), environment variable (with upper cases and underscores), or a command line parameter:

endpoint.name|ENDPOINT_NAME|--endpoint-name

Which endpoint in the configuration to use. Defaults to default.

endpoint.url|ENDPOINT_URL|--endpoint-url

The endpoint url to use for the default endpoint. Defaults to http://localhost:8080.

endpoint.method|ENDPOINT_METHOD|--method

Which HTTP method to use for calling the endpoint. Defaults to POST.

endpoint.concurrency|ENDPOINT_CONCURRENCY|--concurrency

The maximum number of concurrent requests to the endpoint. Defaults to 10.

endpoint.buffer|ENDPOINT_BUFFER|--buffer

How many payloads should be buffered while we have ongoing requests. Defaults to 10.

endpoint.insecure|ENDPOINT_INSECURE|--insecure

Marks the endpoint as insecure or not. An insecure endpoint will not have its certificate check. Defaults to false.

endpoint.template|ENDPOINT_TEMPLATE|--template

Where to locate the template for generating the payloads. Defaults to ./payload.json.

endpoint.content_type|ENDPOINT_CONTENT_TYPE|--content-type

Which Content-Type to pass to the endpoint. Defaults to application/json.

generator.total|GENERATOR_TOTAL|--total

The number of generated payloads. Defaults to 10.

generator.config|GENERATOR_CONFIG|--config

Which configuration files should be used. Fake-O-Matic allows you to define parent configurations, so you can reuse them in the way it suits you better. The configurations should be comma separated from the most specific to the least specific (so the last configuration file becomes the parent one). The built-in configuration can be included with a simple fakeomatic name. The built-in configuration can be found at src/main/resources/META-INF/config/fakeomatic.yaml.

Note
The --config param can be also used once for each file to include.
generator.seed|GENERATOR_SEED|--seed

The seed to use for the random functions. Fake-O-Matic will generate one if empty.

fakeomatic.events.log.level|FAKEOMATIC_EVENTS_LOG_LEVEL|--events-log-level

Sets the log level for the events. To see all the payloads and responses, set the log level to DEBUG.

Note
You can still make use of the endpoint properties if you use the same approach as the built-in one for defining an endpoint.

Example

The following examples can be used with the built-in configuration.

{
  "id": "{some('uuid')}",
  "cause": "{some('error_cause')}",
  "device": "{expression('%%-#####')}"
}
{
  "level": "{oneOf('INFO', 'WARN', 'ERROR')}",
  "message": "{some('business_bullshit')}"
}
io.backpackcloud

Backpack Cloud

Ideas worth packing!

Versions

Version
0.9.1
0.9.0
0.7.0
0.6.0
0.5.0
0.3.0
0.2.1
0.2.0
0.1.0