hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tomasz Nykiel" <tom...@fb.com>
Subject Review Request: HDFS-395 DFS Scalability: Incremental block reports
Date Fri, 01 Jul 2011 20:32:12 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/998/
-----------------------------------------------------------

Review request for hadoop-hdfs, Dhruba Borthakur and Hairong Kuang.


Summary
-------

This patch introduces explicit ACKs sent form datanodes to the namenode in order to minimize
the difference between NN blocks info and DN state.
This will allow for sending the full block reports less frequently, which in turn will minimize
the overhead of processing them, and blocking the namenode for extended time.


This addresses bug HDFS-395.
    https://issues.apache.org/jira/browse/HDFS-395


Diffs
-----

  trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java 1142055 
  trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java 1142055 
  trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java 1142055 
  trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java 1142055 
  trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java 1142055

  trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java
1142055 
  trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java 1142055


Diff: https://reviews.apache.org/r/998/diff


Testing
-------

Will do JUnit tests.


Thanks,

Tomasz


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message