hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tomasz Nykiel" <tnyk...@cs.toronto.edu>
Subject Re: Review Request: HDFS-395 DFS Scalability: Incremental block reports
Date Wed, 24 Aug 2011 03:33:32 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/998/
-----------------------------------------------------------

(Updated 2011-08-24 03:33:32.850921)


Review request for hadoop-hdfs, Dhruba Borthakur and Hairong Kuang.


Summary
-------

This patch introduces explicit ACKs sent form datanodes to the namenode in order to minimize
the difference between NN blocks info and DN state.
This will allow for sending the full block reports less frequently, which in turn will minimize
the overhead of processing them, and blocking the namenode for extended time.


This addresses bug HDFS-395.
    https://issues.apache.org/jira/browse/HDFS-395


Diffs (updated)
-----

  trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
1160950 
  trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java 1160950

  trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java 1160950

  trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDatasetAsyncDiskService.java
1160950 
  trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java 1160950

  trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockCommand.java
1160950 
  trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java
1160950 
  trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/ReceivedDeletedBlockInfo.java
PRE-CREATION 
  trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java
1160950 
  trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java
1160950 

Diff: https://reviews.apache.org/r/998/diff


Testing
-------

Will do JUnit tests.


Thanks,

Tomasz


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message