hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9436) Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes by default
Date Thu, 26 Nov 2015 01:06:11 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027931#comment-15027931
] 

Konstantin Shvachko commented on HDFS-9436:
-------------------------------------------

In {{BlockReportStats}} 
- {{numOpsRequired}} means the total number of block reports processed by NN from all DNs
- {{numThreads}} is the number of DNs, sending the reports, each as a separate thread
- {{addBlock()}} appends a new block to the list of {{blocks}}, which represents replicas
to be reported. The list is formed from "real" blocks first and then padded by fake ones.
- {{blockReportList}} is a sequence of longs, representing the actual report of all the replicas
in {{blocks}}, which is sent to the NN

Hope this clarifies. A better JavaDoc is always welcome.

I think everything works as expected, except the EMPTY array introduced as you mentioned by
HDFS-7435. We should fix it in a different jira.

> Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes by default
> ----------------------------------------------------------------------------
>
>                 Key: HDFS-9436
>                 URL: https://issues.apache.org/jira/browse/HDFS-9436
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: test
>            Reporter: Mingliang Liu
>            Assignee: Mingliang Liu
>            Priority: Minor
>             Fix For: 2.8.0
>
>         Attachments: HDFS-9436.000.patch, HDFS-9436.001.patch
>
>
> This is a follow-up of [HDFS-9379].
> Though for actual benchmarking the defaults are rarely used, it would be good to change
the default for {{numThreads}} as a >=10 value and may be {{numOpsRequired}} in {{BlockReportStats}}
just to make sure the condition in [HDFS-9379] is tested in the future.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message