hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jonsolaris (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-6736) FSNamesystem.appendFile() try to remove block from replication queue, but It may not work
Date Wed, 23 Jul 2014 04:03:39 GMT

     [ https://issues.apache.org/jira/browse/HDFS-6736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

jonsolaris updated HDFS-6736:
-----------------------------

    Description: 
the process appendFile() seems to remove block from replication queue, but It may not  work
, with the code:

FSNamesystem.appendFile()
{

...
          // Remove block from replication queue.
<br>

          updateNeededReplications(last, 0, 0); // last is the block appending
...
}

UnderReplicatedBlocks.updateNeededReplications(last, 0, 0) {
    NumberReplicas repl = countNodes(last);
    int curExpectedReplicas = getReplication(last);
    neededReplications.update(last, 
                              repl.liveReplicas(), 
                              repl.decommissionedReplicas(),
                              curExpectedReplicas,
                              0, 0);
  }

UnderReplicatedBlocks.update(Block block, int curReplicas, 
                           int decommissionedReplicas,
                           int curExpectedReplicas,
                           int curReplicasDelta, int expectedReplicasDelta)
{
     //while the curReplicasDelta & expectedReplicasDelta are all be 0,
     //the oldReplicas and oldExpectedReplicas while be the same
     // then the  curPri == oldPri 

    int oldReplicas = curReplicas-curReplicasDelta;
    int oldExpectedReplicas = curExpectedReplicas-expectedReplicasDelta;
    int curPri = getPriority(block, curReplicas, decommissionedReplicas, curExpectedReplicas);
    int oldPri = getPriority(block, oldReplicas, decommissionedReplicas, oldExpectedReplicas);

    //  the if condition may get false , coz  oldPri == curPri
    // this will never remove(), and it may call the add()
    if(oldPri != LEVEL && oldPri != curPri) {
      remove(block, oldPri);
    }
    if(curPri != LEVEL && priorityQueues.get(curPri).add(block)) {
    ... //log()
    }
}




  was:
the process appendFile() seems to remove block from replication queue, but It may not  work
, with the code:

FSNamesystem.appendFile()
{

...
          // Remove block from replication queue.


          updateNeededReplications(last, 0, 0); // last is the block appending
...
}

UnderReplicatedBlocks.updateNeededReplications(last, 0, 0) {
    NumberReplicas repl = countNodes(last);
    int curExpectedReplicas = getReplication(last);
    neededReplications.update(last, 
                              repl.liveReplicas(), 
                              repl.decommissionedReplicas(),
                              curExpectedReplicas,
                              0, 0);
  }

UnderReplicatedBlocks.update(Block block, int curReplicas, 
                           int decommissionedReplicas,
                           int curExpectedReplicas,
                           int curReplicasDelta, int expectedReplicasDelta)
{
     //while the curReplicasDelta & expectedReplicasDelta are all be 0,
     //the oldReplicas and oldExpectedReplicas while be the same
     // then the  curPri == oldPri 

    int oldReplicas = curReplicas-curReplicasDelta;
    int oldExpectedReplicas = curExpectedReplicas-expectedReplicasDelta;
    int curPri = getPriority(block, curReplicas, decommissionedReplicas, curExpectedReplicas);
    int oldPri = getPriority(block, oldReplicas, decommissionedReplicas, oldExpectedReplicas);

    //  the if condition may get false , coz  oldPri == curPri
    // this will never remove(), and it may call the add()
    if(oldPri != LEVEL && oldPri != curPri) {
      remove(block, oldPri);
    }
    if(curPri != LEVEL && priorityQueues.get(curPri).add(block)) {
    ... //log()
    }
}





> FSNamesystem.appendFile() try to remove block from replication queue, but It may not
 work
> ------------------------------------------------------------------------------------------
>
>                 Key: HDFS-6736
>                 URL: https://issues.apache.org/jira/browse/HDFS-6736
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 1.2.1
>            Reporter: jonsolaris
>   Original Estimate: 12h
>  Remaining Estimate: 12h
>
> the process appendFile() seems to remove block from replication queue, but It may not
 work , with the code:
> FSNamesystem.appendFile()
> {
> ...
>           // Remove block from replication queue.
> <br>
>           updateNeededReplications(last, 0, 0); // last is the block appending
> ...
> }
> UnderReplicatedBlocks.updateNeededReplications(last, 0, 0) {
>     NumberReplicas repl = countNodes(last);
>     int curExpectedReplicas = getReplication(last);
>     neededReplications.update(last, 
>                               repl.liveReplicas(), 
>                               repl.decommissionedReplicas(),
>                               curExpectedReplicas,
>                               0, 0);
>   }
> UnderReplicatedBlocks.update(Block block, int curReplicas, 
>                            int decommissionedReplicas,
>                            int curExpectedReplicas,
>                            int curReplicasDelta, int expectedReplicasDelta)
> {
>      //while the curReplicasDelta & expectedReplicasDelta are all be 0,
>      //the oldReplicas and oldExpectedReplicas while be the same
>      // then the  curPri == oldPri 
>     int oldReplicas = curReplicas-curReplicasDelta;
>     int oldExpectedReplicas = curExpectedReplicas-expectedReplicasDelta;
>     int curPri = getPriority(block, curReplicas, decommissionedReplicas, curExpectedReplicas);
>     int oldPri = getPriority(block, oldReplicas, decommissionedReplicas, oldExpectedReplicas);
>     //  the if condition may get false , coz  oldPri == curPri
>     // this will never remove(), and it may call the add()
>     if(oldPri != LEVEL && oldPri != curPri) {
>       remove(block, oldPri);
>     }
>     if(curPri != LEVEL && priorityQueues.get(curPri).add(block)) {
>     ... //log()
>     }
> }



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message