hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2554) Add separate metrics for missing blocks with desired replication level 1
Date Mon, 20 Aug 2012 21:44:37 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13438243#comment-13438243

Eli Collins commented on HDFS-2554:

bq. Maybe I'm missing something, but this seems to be the same change you suggested in a comment
above dated 07/Aug/12. I responded to it above, it seems much more natural to me to provide
values A and B which add to give C than to provide A and C which subtracted give B.

I was thinking in terms of the naming. Eg getMissingBlocks and getMissingBlocksWithRepl1 seems
more clear than R1 and RN. We should at least do something like getMissingBlocksWithRepl1
and getMissingBlocksWithReplGreaterThan1, ie that indicates what is being returned (ie just
from the javadoc w/o reading what R1 means).

Getting the "total count" via {{getMissingBlocksWithRepl1() + getMissingBlocksWithReplGreaterThan1()}}
seems clunkier than finding "missing blocks with > 1 replica" via  {{getMissingBlocks()
- getMissingBlocksWithRepl1()}} because in the 2nd case the user expects to have to do arithmetic.
I don't feel strongly. In practice we'll want to know all values so either way the user is
doing arithmetic. 

> Add separate metrics for missing blocks with desired replication level 1
> ------------------------------------------------------------------------
>                 Key: HDFS-2554
>                 URL: https://issues.apache.org/jira/browse/HDFS-2554
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>    Affects Versions: 2.0.0-alpha
>            Reporter: Todd Lipcon
>            Assignee: Andy Isaacson
>            Priority: Minor
>         Attachments: hdfs-2554-1.txt, hdfs-2554.txt
> Some users use replication level set to 1 for datasets which are unimportant and can
be lost with no worry (eg the output of terasort tests). But other data on the cluster is
important and should not be lost. It would be useful to separate the metric for missing blocks
by the desired replication level of those blocks, so that one could ignore missing blocks
at repl 1 while still alerting on missing blocks with higher desired replication.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message