hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Merging some improvements to branch-2
Date Thu, 07 Jun 2012 21:51:26 GMT
Hi all,

I plan to merge the following JIRAs to branch-2 this week unless there
are any objections:

"Direct read" optimization:

6e51b33 HADOOP-8135. Add ByteBufferReadable interface to
FSDataInputStream. Contributed by Henry Robinson.
4418682 HADOOP-8244. Improve comments on ByteBufferReadable.read.
Contributed by Henry Robinson.
c66f982 HDFS-2834. Add a ByteBuffer-based read API to DFSInputStream.
Contributed by Henry Robinson.
467acd1 HDFS-3110. Use directRead API to reduce the number of buffer
copies in libhdfs. Contributed by Henry Robinson.

These improvements give a substantial savings in CPU to applications
using libhdfs, and have some potential for usage in apps like HBase
and MR as well.

bfa5c0a HDFS-3167. CLI-based driver for MiniDFSCluster. Contributed by
Henry Robinson.
21dfa6a HDFS-3235. MiniDFSClusterManager doesn't correctly support
-format option. Contributed by Henry Robinson.

These test-only improvements make it easier to construct system tests
against realistic pseudo-distributed clusters.

All of the above patches have been baking in trunk for quite some time.

Todd Lipcon
Software Engineer, Cloudera

View raw message