hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arpit Agarwal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7896) HDFS Slow disk detection
Date Thu, 22 Oct 2015 23:37:28 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14970109#comment-14970109

Arpit Agarwal commented on HDFS-7896:

Thanks for taking this up [~brahmareddy].

It's overkill and invasive to run slow disk detection for every disk error. It should be sufficient
to do this once on DN startup to begin with. Also we just need to run it once per volume,
not separately for each combination of \{block pool\} × \{finalized, rbw, tmp}.

It doesn't look like YARN ships with a default disk checker script, so we can look into adding
one in a separate Jira.

> HDFS Slow disk detection
> ------------------------
>                 Key: HDFS-7896
>                 URL: https://issues.apache.org/jira/browse/HDFS-7896
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>            Reporter: Arpit Agarwal
>         Attachments: HDFS-7896.00.patch
> HDFS should detect slow disks. To start with we can flag this information via the NameNode
web UI. Alternatively DNs can avoid using slow disks for writes.

This message was sent by Atlassian JIRA

View raw message