hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kihwal Lee (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-9945) Datanode command for evicting writers
Date Fri, 11 Mar 2016 14:37:10 GMT
Kihwal Lee created HDFS-9945:
--------------------------------

             Summary: Datanode command for evicting writers
                 Key: HDFS-9945
                 URL: https://issues.apache.org/jira/browse/HDFS-9945
             Project: Hadoop HDFS
          Issue Type: New Feature
          Components: datanode
            Reporter: Kihwal Lee


It will be useful if there is a command to evict writers from a datanode. When a set of datanodes
are being decommissioned, they can get blocked by slow writers at the end.  It was rare in
the old days since mapred jobs didn't last too long, but with many different types of apps
running on today's YARN cluster, we are often see very long tail in datanode decommissioning.

I propose a new dfsadmin command, {{evictWriters}}, to be added. I initially thought about
having namenode automatically telling datanodes on decommissioning, but realized that having
a command is more flexible. E.g. users can choose not to do this at all, choose when to evict
writers, or whether to try multiple times for whatever reasons.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message