hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7411) Refactor and improve decommissioning logic into DecommissionManager
Date Mon, 02 Feb 2015 23:30:35 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14302319#comment-14302319
] 

Colin Patrick McCabe commented on HDFS-7411:
--------------------------------------------

I don't see any benefit to splitting the patch further.  The old logic is seriously flawed...
it's not effective at rate limiting and often goes way too fast or too slow, because it is
based on number of nodes rather than number of blocks.

I think it's great that [~andrew.wang] took on this task, which has been a maintenance problem
for us for a while.  This is a great example of someone who really cares about the project
making things better by working on something which is "boring" (it will never make it into
a list of new features or exciting research talks) but essential to our users.

There is no benefit to keeping around multiple broken implementations of things to do the
same job.  Hadoop has enough dead and obsolete code as is... more than enough.  Things like
the {{RemoteBlockReader}} / {{RemoteBlockReader2}} split increase our maintenance burden and
confuse users and potential contributors.  I only allowed the {{BlockReaderLocalLegacy}} /
{{BlockReaderLocal}} split because we didn't have platform support for file descriptor passing
on Windows.  But since there are no platform support issues here, there is no reason to increase
our maintenance burden.

If we are concerned about stability, we can let this soak in trunk for a while.

It has been through three months of review.  [~mingma], [~arpitagarwal], and myself have reviewed
this.   It's ready to go in, and I think it should.  +1.  Let's commit this today if there
are no other comments about the patch.

> Refactor and improve decommissioning logic into DecommissionManager
> -------------------------------------------------------------------
>
>                 Key: HDFS-7411
>                 URL: https://issues.apache.org/jira/browse/HDFS-7411
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 2.5.1
>            Reporter: Andrew Wang
>            Assignee: Andrew Wang
>         Attachments: hdfs-7411.001.patch, hdfs-7411.002.patch, hdfs-7411.003.patch, hdfs-7411.004.patch,
hdfs-7411.005.patch, hdfs-7411.006.patch, hdfs-7411.007.patch, hdfs-7411.008.patch, hdfs-7411.009.patch,
hdfs-7411.010.patch
>
>
> Would be nice to split out decommission logic from DatanodeManager to DecommissionManager.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message