hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "surendra singh lilhore (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8540) Mover should exit with NO_MOVE_BLOCK if no block can be moved
Date Sat, 13 Jun 2015 09:45:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14584531#comment-14584531
] 

surendra singh lilhore commented on HDFS-8540:
----------------------------------------------

Thanks [~szetszwo] for review..

I have one doubt

{code}
    ExitStatus getExitStatus(){
      return isNoBlockMoved()? ExitStatus.NO_MOVE_BLOCK
          : isHasRemaining()? ExitStatus.IN_PROGRESS: ExitStatus.SUCCESS;
    }
{code}

This will return "NO_MOVE_BLOCK"  when user execute mover but nothing to move and all the
file in proper storage policy.

I think this should be 

{code}
    ExitStatus getExitStatus() {
      return isHasRemaining() ? isNoBlockMoved()
          ? ExitStatus.NO_MOVE_BLOCK : ExitStatus.IN_PROGRESS
          : ExitStatus.SUCCESS;
    }
{code}

We should first check {{isHasRemaining()}}, if it is false we should return SUCCESS.


> Mover should exit with NO_MOVE_BLOCK if no block can be moved
> -------------------------------------------------------------
>
>                 Key: HDFS-8540
>                 URL: https://issues.apache.org/jira/browse/HDFS-8540
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: balancer & mover
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: surendra singh lilhore
>         Attachments: HDFS-8540.patch, HDFS-8540_1.patch
>
>
> When there are files not satisfying their storage policy and no move is possible, Mover
exits with SUCCESS.  It should exit with NO_MOVE_BLOCK.
> The bug seems in the following code.  When StorageTypeDiff is not empty and scheduleMoves4Block
return false, it does not update hasRemaining.  Also, there is no indication of "No block
can be moved" for the entire iteration.
> {code}
> //Mover.processFile(..)
>         if (!diff.removeOverlap(true)) {
>           if (scheduleMoves4Block(diff, lb, ecSchema)) {
>             hasRemaining |= (diff.existing.size() > 1 &&
>                 diff.expected.size() > 1);
>           }
>         }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message