hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7165) Separate block metrics for files with replication count 1
Date Tue, 21 Oct 2014 18:16:35 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178773#comment-14178773
] 

Zhe Zhang commented on HDFS-7165:
---------------------------------

Thanks, that makes sense.

bq. In ClientProtocol#getStats, it mentions "total used space of the block pool", and I see
that being set in HeartbeatManager, but AFAICT it's dropped in the PB layer on the server
side. If it's not being used, let's remove it.

It is indeed dropped and the new patch has it removed.

{{NameNodeMXBean}} line 148 is a correction of an existing comment not related to this patch,
but in the neighborhood.

> Separate block metrics for files with replication count 1
> ---------------------------------------------------------
>
>                 Key: HDFS-7165
>                 URL: https://issues.apache.org/jira/browse/HDFS-7165
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Andrew Wang
>            Assignee: Zhe Zhang
>         Attachments: HDFS-7165-20141003-v1.patch, HDFS-7165-20141009-v1.patch, HDFS-7165-20141010-v1.patch,
HDFS-7165-20141015-v1.patch
>
>
> We see a lot of escalations because someone has written teragen output with a replication
factor of 1, a DN goes down, and a bunch of missing blocks show up. These are normally false
positives, since teragen output is disposable, and generally speaking, users should understand
this is true for all repl=1 files.
> It'd be nice to be able to separate out these repl=1 missing blocks from missing blocks
with higher replication factors..



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message