hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Elek, Marton (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDDS-143) Provide docker compose files to measure performance in a pseudo cluster
Date Fri, 01 Jun 2018 20:30:00 GMT
Elek, Marton created HDDS-143:

             Summary: Provide docker compose files to measure performance in a pseudo cluster
                 Key: HDDS-143
                 URL: https://issues.apache.org/jira/browse/HDDS-143
             Project: Hadoop Distributed Data Store
          Issue Type: Improvement
          Components: Tools
            Reporter: Elek, Marton
            Assignee: Elek, Marton
             Fix For: 0.2.1

Currently we have one docker-compose file to run local pseudo cluster with Ozone/HDDS. But
it's hard to do longer performance tests as we have no metrics collector in the compose files.

This patch provide a simple solution: it includes a prometheus server and a consul component.
With the init script you can download the required java agent: the agent will publish jmx
metrics on a prometheus compatible endpoint and register the metrics endpoint in the consul.
Prometheus collects the data from the registered endpoints.

More details about the usage are included in the README.md.

I run this cluster in my development machine (SSD, 16G ram) with one freon and 3 datanodes.
The average rate of key creation was around 180-220 key / seconds.

The compose file could also help to check the available metrics or compare performance before/after
a patch.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

View raw message