hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tomasz Nykiel" <tom...@fb.com>
Subject Re: Review Request: HDFS-395 DFS Scalability: Incremental block reports
Date Fri, 01 Jul 2011 20:34:17 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/998/
-----------------------------------------------------------

(Updated 2011-07-01 20:34:17.116645)


Review request for hadoop-hdfs, Dhruba Borthakur and Hairong Kuang.


Summary
-------

This patch introduces explicit ACKs sent form datanodes to the namenode in order to minimize
the difference between NN blocks info and DN state.
This will allow for sending the full block reports less frequently, which in turn will minimize
the overhead of processing them, and blocking the namenode for extended time.


This addresses bug HDFS-395.
    https://issues.apache.org/jira/browse/HDFS-395


Diffs (updated)
-----

  trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java 1142085 
  trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java 1142085 
  trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java 1142085 
  trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java 1142085 
  trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java 1142085

  trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/ReceivedDeletedBlockInfo.java
PRE-CREATION 
  trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java
1142085 
  trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java 1142085


Diff: https://reviews.apache.org/r/998/diff


Testing
-------

Will do JUnit tests.


Thanks,

Tomasz


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message