JarCastingPerformance MQTT Test Suite
Getting started
Now you have an opportunity to independently evaluate the performance of IoTBroker.Cloud. Besides this test suite can be used to measure the performance of your own software. The following instruction will explain how to run the performance tests by yourself.
Prerequisites
The following programs should be installed before starting to clone the project:
- JDK (version 7+);
- Eclipse;
- Maven.
Installation
First of all, you should clone Performance MQTT Test Suite.
Then you have to build the project. For this purpose, open the Performance Test Suite project in Eclipse, right-click mqtt-test-suite and go to Maven / Update Project. Then you should again right-click mqtt-test-suite and go to Run As / Maven clean. Finally you should right-click mqtt-test-suite and go to Run As / Maven install.
Now you have the controller (in mqtt-test-suite/controller/target folder) and the test runner (in mqtt-test-suite/runner/target folder) jar files on your computer. To make the work more convenient, create performance_test folder which will contain controller-jar-with-dependencies.jar and runner-jar-with-dependencies.jar. Also you should add JSON files to this very performance_test folder and controller.params file (you can find it in mqtt-test-suite/controller folder).
Test run
First you should open the terminal and cd to performance_test folder. You should start the controller by running the command which is given below (do not forget to indicate your path):
Now you can start the controller by running the following command :
java -Xmx2048m -Xms512m -jar controller.jar http://127.0.0.1:9998/ /home/username/performance_test/controller.params
Here is a brief explanation:
Xmx2048m – maximum memory allocation;
Xms512m – initial memory allocation;
controller.jar – controller which is inside the performance_test folder;
http://192.168.1.1:9998/ - IP address and port of controller;
/home/username/performance_test/controller.params – path to controller.params file.
Now you should open the second terminal window and cd to performance_test folder. Now you can run the test by running the following command:
java -jar test-runner.jar pipeline.json
The command mentioned above is an example of running the test scenario which is described in pipeline.json file.
Each JSON file contains different test scenarios. You can separately run each test scenario by indicating the name of a specific JSON file. When the test is over you will get the report for each test scenario:
+---------- Scenario-ID: df2b3214-650e-41ed-b753-83862a473e6f ---------- Result: SUCCESS ----------+
| Start Time | 2016-12-21 18:36:49.526 | 1482338209526 |
| Finish Time | 2016-12-21 18:51:53.762 | 1482339113762 |
| Current Time | 2016-12-21 18:51:59.951 | 1482339119951 |
+---------------------------------+--------------------------------+--------------------------------+
| Total clients 500 | Total commands 451000 | Errors occured 0 |
| Successfuly finished 500 | Successfuly finished 451000 | Duplicates received 0 |
| Failed 0 | Failed 0 | Duplicates sent 0 |
+--------------- Outgoing counters ---------------+--------------- Incoming counters ---------------+
| Counter Name | Counter Value | Counter Name | Counter Value |
| CONNECT | 500 | CONNACK | 500 |
| SUBSCRIBE | 0 | SUBACK | 0 |
| UNSUBSCRIBE | 0 | UNSUBACK | 0 |
| PINGREQ | 500 | PINGRESP | 500 |
| PUBLISH | 450000 | PUBLISH | 0 |
| PUBACK | 0 | PUBACK | 0 |
| PUBREC | 0 | PUBREC | 450000 |
| PUBREL | 450000 | PUBREL | 0 |
| PUBCOMP | 0 | PUBCOMP | 450000 |
| DISCONNECT | 500 | DISCONNECT | 0 |
+------------------------+------------------------+------------------------+------------------------+
+---------- Scenario-ID: 68dd3a83-6aca-48fd-80f6-ec79bf4ddcc7 ---------- Result: SUCCESS ----------+
| Start Time | 2016-12-21 18:36:49.514 | 1482338209514 |
| Finish Time | 2016-12-21 18:51:53.794 | 1482339113794 |
| Current Time | 2016-12-21 18:51:59.970 | 1482339119970 |
+---------------------------------+--------------------------------+--------------------------------+
| Total clients 500 | Total commands 451000 | Errors occured 0 |
| Successfuly finished 500 | Successfuly finished 451000 | Duplicates received 0 |
| Failed 0 | Failed 0 | Duplicates sent 0 |
+--------------- Outgoing counters ---------------+--------------- Incoming counters ---------------+
| Counter Name | Counter Value | Counter Name | Counter Value |
| CONNECT | 500 | CONNACK | 500 |
| SUBSCRIBE | 0 | SUBACK | 0 |
| UNSUBSCRIBE | 0 | UNSUBACK | 0 |
| PINGREQ | 500 | PINGRESP | 500 |
| PUBLISH | 450000 | PUBLISH | 0 |
| PUBACK | 0 | PUBACK | 0 |
| PUBREC | 0 | PUBREC | 450000 |
| PUBREL | 450000 | PUBREL | 0 |
| PUBCOMP | 0 | PUBCOMP | 450000 |
| DISCONNECT | 500 | DISCONNECT | 0 |
+------------------------+------------------------+------------------------+------------------------+
Each test can be run in its current form. Besides you can change the existing test scenarios or add the new ones. The information given here will help you to get into these test requests. If you are interested in making considerable changes, the Structure of Performance Test Suite will help you to get into the structure of this test suite.
Performance MQTT Test Suite is developed by Mobius Software.
