hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Elliott Clark (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12973) make DU pluggable
Date Wed, 30 Mar 2016 20:10:25 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15218747#comment-15218747
] 

Elliott Clark commented on HADOOP-12973:
----------------------------------------

bq.Hi Elliott Clark I wonder what was the concern about replacing the call to DU?
My understanding is that replacing du with df means that we will lose the ability to tell
how large the distributed cache is. So for people who rely on MR/Yarn a lot then it's possible
to get into a bad situation. I haven't really looked into that too much. My primary use case
is around HBase where a du running and walking all the inodes causes a significant IO spike
and latency outliers.

bq.With regard to the issues described in HDFS-9923, do you think that your solution also
fixes the inconsistent exception handling of DU?
I didn't change the exception handling at all, but it is weird and changing it to just log
a warn seems like a good idea to me.

> make DU pluggable
> -----------------
>
>                 Key: HADOOP-12973
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12973
>             Project: Hadoop Common
>          Issue Type: Sub-task
>            Reporter: Elliott Clark
>            Assignee: Elliott Clark
>         Attachments: HADOOP-12973v0.patch, HADOOP-12973v1.patch, HADOOP-12973v2.patch,
HADOOP-12973v3.patch, HADOOP-12973v5.patch, HADOOP-12973v6.patch
>
>
> If people are concerned about replacing the call to DU. Then an easy first step is to
make it pluggable. Then it's possible to replace it with something while leaving the default
alone.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message