hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yongjun Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-6577) Failure to create FinalizeCommand
Date Fri, 20 Jun 2014 14:57:25 GMT

     [ https://issues.apache.org/jira/browse/HDFS-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Yongjun Zhang updated HDFS-6577:
--------------------------------

    Description: 
I visually see a bug in NameNodeRpcServer.java, 
{code}
 @Override // DatanodeProtocol
  public DatanodeCommand blockReport(DatanodeRegistration nodeReg,
      String poolId, StorageBlockReport[] reports) throws IOException {
    verifyRequest(nodeReg);
    if(blockStateChangeLog.isDebugEnabled()) {
      blockStateChangeLog.debug("*BLOCK* NameNode.blockReport: "
           + "from " + nodeReg + ", reports.length=" + reports.length);
    }
    final BlockManager bm = namesystem.getBlockManager();
    boolean noStaleStorages = false;
    for(StorageBlockReport r : reports) {
      final BlockListAsLongs blocks = new BlockListAsLongs(r.getBlocks());
      noStaleStorages = bm.processReport(nodeReg, r.getStorage(), poolId, blocks);
      metrics.incrStorageBlockReportOps();
    }

    if (nn.getFSImage().isUpgradeFinalized() &&
        !nn.isStandbyState() &&
        noStaleStorages) {
      return new FinalizeCommand(poolId);
    }

    return null;
  }
{code}

noStaleStorage is initialized to false, then in the for loop {{ for(StorageBlockReport r :
reports) {}} it could be changed to true. 

The problem is, since it's a loop of all reports, the value returned from the last blockreport
(the last iteration) will set it to the final value. Even if it was set to true in the middle
of a loop, the final iteration could overwrite it to false. Similarly, if it was set to false
in the middle, and the final iter could overwrite it to true.





  was:
I visually see a bug in NameNodeRpcServer.java, 
{code}
 @Override // DatanodeProtocol
  public DatanodeCommand blockReport(DatanodeRegistration nodeReg,
      String poolId, StorageBlockReport[] reports) throws IOException {
    verifyRequest(nodeReg);
    if(blockStateChangeLog.isDebugEnabled()) {
      blockStateChangeLog.debug("*BLOCK* NameNode.blockReport: "
           + "from " + nodeReg + ", reports.length=" + reports.length);
    }
    final BlockManager bm = namesystem.getBlockManager();
    boolean noStaleStorages = false;
    for(StorageBlockReport r : reports) {
      final BlockListAsLongs blocks = new BlockListAsLongs(r.getBlocks());
      noStaleStorages = bm.processReport(nodeReg, r.getStorage(), poolId, blocks);
      metrics.incrStorageBlockReportOps();
    }

    if (nn.getFSImage().isUpgradeFinalized() &&
        !nn.isStandbyState() &&
        noStaleStorages) {
      return new FinalizeCommand(poolId);
    }

    return null;
  }
{code}

noStaleStorage is initialized to false, then in the for loop {{ for(StorageBlockReport r :
reports) {}} it could be changed to true. 

The problem is, since it's a loop of all reports, the value returned from the last blockreport
(the last iteration) will set it to the final value. Even if it was set to true in the middle
of a loop, the final iteration could overwrite it to false.





> Failure to create FinalizeCommand
> ---------------------------------
>
>                 Key: HDFS-6577
>                 URL: https://issues.apache.org/jira/browse/HDFS-6577
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 2.4.0
>            Reporter: Yongjun Zhang
>            Assignee: Yongjun Zhang
>
> I visually see a bug in NameNodeRpcServer.java, 
> {code}
>  @Override // DatanodeProtocol
>   public DatanodeCommand blockReport(DatanodeRegistration nodeReg,
>       String poolId, StorageBlockReport[] reports) throws IOException {
>     verifyRequest(nodeReg);
>     if(blockStateChangeLog.isDebugEnabled()) {
>       blockStateChangeLog.debug("*BLOCK* NameNode.blockReport: "
>            + "from " + nodeReg + ", reports.length=" + reports.length);
>     }
>     final BlockManager bm = namesystem.getBlockManager();
>     boolean noStaleStorages = false;
>     for(StorageBlockReport r : reports) {
>       final BlockListAsLongs blocks = new BlockListAsLongs(r.getBlocks());
>       noStaleStorages = bm.processReport(nodeReg, r.getStorage(), poolId, blocks);
>       metrics.incrStorageBlockReportOps();
>     }
>     if (nn.getFSImage().isUpgradeFinalized() &&
>         !nn.isStandbyState() &&
>         noStaleStorages) {
>       return new FinalizeCommand(poolId);
>     }
>     return null;
>   }
> {code}
> noStaleStorage is initialized to false, then in the for loop {{ for(StorageBlockReport
r : reports) {}} it could be changed to true. 
> The problem is, since it's a loop of all reports, the value returned from the last blockreport
(the last iteration) will set it to the final value. Even if it was set to true in the middle
of a loop, the final iteration could overwrite it to false. Similarly, if it was set to false
in the middle, and the final iter could overwrite it to true.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message