hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7847) Modify NNThroughputBenchmark to be able to operate on a remote NameNode
Date Thu, 19 Nov 2015 06:16:11 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15012903#comment-15012903
] 

Colin Patrick McCabe commented on HDFS-7847:
--------------------------------------------

bq. If understand this correctly, you are not targeting to build a benchmark, but rather unit
tests of NameNode RPC. Isn't there a bunch of those in HDFS source tree already? Just trying
to understand what is different.

Excuse me, I mis-spoke.  {{NNThroughputBenchmark}} is a benchmark, not a unit test.

bq. Have you looked at org.apache.hadoop.fs.loadGenerator.LoadGenerator. Would its functionality
suffice the testing needs you have in mind?

{{LoadGenerator}} is certainly much more similar to {{NNThroughputBenchmark}} than any of
the other things you mentioned.  However, it performs a mix of only list, read, and write
operations.  {{NNThroughputBenchmark}} performs a wider range of operations.  In general,
I would like to avoid read and write operations in my testing of NN RPC performance, since
those operations involve the DataNode heavily.  I think it's good to have multiple different
benchmarks testing different things.

> Modify NNThroughputBenchmark to be able to operate on a remote NameNode
> -----------------------------------------------------------------------
>
>                 Key: HDFS-7847
>                 URL: https://issues.apache.org/jira/browse/HDFS-7847
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.8.0
>            Reporter: Colin Patrick McCabe
>            Assignee: Charles Lamb
>             Fix For: 2.8.0
>
>         Attachments: HDFS-7847.000.patch, HDFS-7847.001.patch, HDFS-7847.002.patch, HDFS-7847.003.patch,
HDFS-7847.004.patch, HDFS-7847.005.patch, make_blocks.tar.gz
>
>
> Modify NNThroughputBenchmark to be able to operate on a NN that is not in process. A
followon Jira will modify it some more to allow quantifying native and java heap sizes, and
some latency numbers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message