hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uma Maheswara Rao G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7310) Mover can give first priority to local DN if it has target storage type available in local DN
Date Tue, 25 Nov 2014 17:39:16 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224891#comment-14224891
] 

Uma Maheswara Rao G commented on HDFS-7310:
-------------------------------------------

Hi Vinay, Patch looks good to me. I have one question before my +1 on this.

Seems like now when you plan to move block across storage in a DN and if it is not on transient
storage, you are not doing checksum calculation. my point is, we may  need to compute checksum
for having data integrity when moving across storage? For some reason, that is not necessary,
please let me know and also in such case we need to change below log though

{code}
if (LOG.isDebugEnabled()) {
      LOG.debug("Copied " + srcMeta + " to " + dstMeta +
          " and calculated checksum");
      LOG.debug("Copied " + srcFile + " to " + dstFile);
    }
{code}


> Mover can give first priority to local DN if it has target storage type available in
local DN
> ---------------------------------------------------------------------------------------------
>
>                 Key: HDFS-7310
>                 URL: https://issues.apache.org/jira/browse/HDFS-7310
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: balancer & mover
>    Affects Versions: 3.0.0
>            Reporter: Uma Maheswara Rao G
>            Assignee: Vinayakumar B
>         Attachments: HDFS-7310-001.patch, HDFS-7310-002.patch, HDFS-7310-003.patch
>
>
> Currently Mover logic may move blocks to any DN which had target storage type. But if
the src DN has target storage type then mover can give highest priority to local DN. If local
DN does not contains target storage type, then it can assign to any DN as the current logic
does.
>   This is a thought, have not go through the code fully yet.
> Thoughts?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message