hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7847) Modify NNThroughputBenchmark to be able to operate on a remote NameNode
Date Tue, 17 Nov 2015 23:35:11 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15009837#comment-15009837
] 

Konstantin Shvachko commented on HDFS-7847:
-------------------------------------------

??The motivation was to have a simple unit test of raw NameNode RPC throughput in the HDFS
source code.??

If understand this correctly, you are not targeting to build a benchmark, but rather unit
tests of NameNode RPC.
Isn't there a bunch of those in HDFS source tree already? Just trying to understand what is
different.

??DFSIO is not a test of pure NN RPC performance, since it is a MapReduce job??

It can be used to test NameNode's performance, because the MR overhead is subtracted. It is
in MapReduce though, true.

??SLive is an external project??

SLive is in the same place as DFSIO. External to what?

Have you looked at [{{org.apache.hadoop.fs.loadGenerator.LoadGenerator}}|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/loadGenerator/LoadGenerator.java].
Would its functionality suffice the testing needs you have in mind?


> Modify NNThroughputBenchmark to be able to operate on a remote NameNode
> -----------------------------------------------------------------------
>
>                 Key: HDFS-7847
>                 URL: https://issues.apache.org/jira/browse/HDFS-7847
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.8.0
>            Reporter: Colin Patrick McCabe
>            Assignee: Charles Lamb
>             Fix For: 2.8.0
>
>         Attachments: HDFS-7847.000.patch, HDFS-7847.001.patch, HDFS-7847.002.patch, HDFS-7847.003.patch,
HDFS-7847.004.patch, HDFS-7847.005.patch, make_blocks.tar.gz
>
>
> Modify NNThroughputBenchmark to be able to operate on a NN that is not in process. A
followon Jira will modify it some more to allow quantifying native and java heap sizes, and
some latency numbers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message