hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rui Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8968) New benchmark throughput tool for striping erasure coding
Date Thu, 05 Nov 2015 12:17:27 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14991596#comment-14991596
] 

Rui Li commented on HDFS-8968:
------------------------------

Hi [~rakeshr],
One more question. What do you mean by running a real cluster inside {{ErasureCodeBenchmarkThroughput}}?
I checked {{BenchmarkThroughput}} and seems it only runs against local FS and MiniDFSCluster.
I mentioned I can run {{ErasureCodeBenchmarkThroughput}} with MiniDFSCluster in a unit test.
And that's only intended to test the new tool. To really benchmark performance, user has to
deploy a real cluster, right?
Thanks.

> New benchmark throughput tool for striping erasure coding
> ---------------------------------------------------------
>
>                 Key: HDFS-8968
>                 URL: https://issues.apache.org/jira/browse/HDFS-8968
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Kai Zheng
>            Assignee: Rui Li
>         Attachments: HDFS-8968-HDFS-7285.1.patch, HDFS-8968-HDFS-7285.2.patch, HDFS-8968.3.patch
>
>
> We need a new benchmark tool to measure the throughput of client writing and reading
considering cases or factors:
> * 3-replica or striping;
> * write or read, stateful read or positional read;
> * which erasure coder;
> * striping cell size;
> * concurrent readers/writers using processes or threads.
> The tool should be easy to use and better to avoid unnecessary local environment impact,
like local disk.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message