hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7165) Separate block metrics for files with replication count 1
Date Mon, 13 Oct 2014 23:35:35 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170237#comment-14170237
] 

Andrew Wang commented on HDFS-7165:
-----------------------------------

Hi Zhe, generally looks good. A few review comments:

* Need javadoc on new MXBean method
* For naming, you could try "MissingBlocksWithReplOne" as a shorter alternative to Phil's
suggestion.
* One whitespace-only change in TestMissingBlocksAlert
* UnderReplicatedBlocks looks like a standalone class, so we happily might be able to write
some actual unit tests. TestUnderReplicatedBlockQueues has an example. Would be good to test
remove and update in addition to test, this will simulate block deletion and setrep (up and
down).
* TestUnderReplicatedBlockQueues also does something lazy and extends Assert rather than doing
the static imports, it'd be cool to fix this up too if you edit this file.

> Separate block metrics for files with replication count 1
> ---------------------------------------------------------
>
>                 Key: HDFS-7165
>                 URL: https://issues.apache.org/jira/browse/HDFS-7165
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Andrew Wang
>            Assignee: Zhe Zhang
>         Attachments: HDFS-7165-20141003-v1.patch, HDFS-7165-20141009-v1.patch, HDFS-7165-20141010-v1.patch
>
>
> We see a lot of escalations because someone has written teragen output with a replication
factor of 1, a DN goes down, and a bunch of missing blocks show up. These are normally false
positives, since teragen output is disposable, and generally speaking, users should understand
this is true for all repl=1 files.
> It'd be nice to be able to separate out these repl=1 missing blocks from missing blocks
with higher replication factors..



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message