hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1351) Make it possible for BlockPlacementPolicy to return null
Date Tue, 24 Aug 2010 23:19:16 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12902146#action_12902146
] 

Konstantin Shvachko commented on HDFS-1351:
-------------------------------------------

Deletion hint is the Balancer logic. Balancer copied a replica to another node, and is requesting
to delete the source replica. So if chooseReplicaToDelete() would ever intentionally return
null, the delHint will be lost, and balancing will be only increasing block replication in
your use case.
I am a bit confused that you state this as a a step in future work, rather than a bug.
Your test fails on current trunk with NullPointerException because chooseReplicaToDelete()
returned null, right? And this seems to be an attempt to fix it. It is good to fix bugs.
If it is intended as a bug fix, then you can fix it without changing the order of choosing
replicas.
If not, then it would be really good if you elaborate on why do you need this. Why e.g. just
bringing the NN into safe mode doesn't work. Basically I am wondering what is coming next,
what is the design of this hypothetical feature. Is there an intention to build one soon.

> Make it possible for BlockPlacementPolicy to return null
> --------------------------------------------------------
>
>                 Key: HDFS-1351
>                 URL: https://issues.apache.org/jira/browse/HDFS-1351
>             Project: Hadoop HDFS
>          Issue Type: Test
>          Components: name-node
>    Affects Versions: 0.22.0
>            Reporter: Dmytro Molkov
>            Assignee: Dmytro Molkov
>         Attachments: HDFS-1351.patch
>
>
> The idea is to modify FSNamesystem.chooseExcessReplicates code, so it can accept a null
return from chooseReplicaToDelete which will indicate that NameNode should not be deleting
extra replicas.
> One possible usecase - if there are nodes being added to the cluster that might have
corrupt replicas on them you do not want to delete other replicas until the block scanner
finished scanning every block on the datanode.
> This will require additional work on the implementation of the BlockPlacementPolicy,
but with this JIRA I just wanted to create a basis for future improvements.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message