hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7847) Modify NNThroughputBenchmark to be able to operate on a remote NameNode
Date Tue, 17 Nov 2015 22:30:11 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15009722#comment-15009722
] 

Colin Patrick McCabe commented on HDFS-7847:
--------------------------------------------

bq. Shouldn't we be specifying the namenode using the standard generic option -fs?

That makes sense to me.  Would you like to post a follow-on JIRA?

bq. I guess I did not fully understand your motivation here?

The motivation was to have a simple unit test of raw NameNode RPC throughput in the HDFS source
code.  DFSIO is not a test of pure NN RPC performance, since it is a MapReduce job with all
the attendant overheads.  SLive is an external project not in the HDFS source tree.

If you want to benchmark the NameNode's performance without RPC overhead, you can still do
that with NNThroughputBenchmark, by using the right command-line options.

> Modify NNThroughputBenchmark to be able to operate on a remote NameNode
> -----------------------------------------------------------------------
>
>                 Key: HDFS-7847
>                 URL: https://issues.apache.org/jira/browse/HDFS-7847
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.8.0
>            Reporter: Colin Patrick McCabe
>            Assignee: Charles Lamb
>             Fix For: 2.8.0
>
>         Attachments: HDFS-7847.000.patch, HDFS-7847.001.patch, HDFS-7847.002.patch, HDFS-7847.003.patch,
HDFS-7847.004.patch, HDFS-7847.005.patch, make_blocks.tar.gz
>
>
> Modify NNThroughputBenchmark to be able to operate on a NN that is not in process. A
followon Jira will modify it some more to allow quantifying native and java heap sizes, and
some latency numbers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message