hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eli Collins <...@cloudera.com>
Subject Re: Merging some improvements to branch-2
Date Thu, 07 Jun 2012 23:25:22 GMT
+1

Will be great to have the direct read optimization (thanks Henry) in 2.x.

Thanks,
Eli

On Thu, Jun 7, 2012 at 2:51 PM, Todd Lipcon <todd@cloudera.com> wrote:
> Hi all,
>
> I plan to merge the following JIRAs to branch-2 this week unless there
> are any objections:
>
> "Direct read" optimization:
>
> 6e51b33 HADOOP-8135. Add ByteBufferReadable interface to
> FSDataInputStream. Contributed by Henry Robinson.
> 4418682 HADOOP-8244. Improve comments on ByteBufferReadable.read.
> Contributed by Henry Robinson.
> c66f982 HDFS-2834. Add a ByteBuffer-based read API to DFSInputStream.
> Contributed by Henry Robinson.
> 467acd1 HDFS-3110. Use directRead API to reduce the number of buffer
> copies in libhdfs. Contributed by Henry Robinson.
>
> These improvements give a substantial savings in CPU to applications
> using libhdfs, and have some potential for usage in apps like HBase
> and MR as well.
>
> MiniDFSClusterManager:
> bfa5c0a HDFS-3167. CLI-based driver for MiniDFSCluster. Contributed by
> Henry Robinson.
> 21dfa6a HDFS-3235. MiniDFSClusterManager doesn't correctly support
> -format option. Contributed by Henry Robinson.
>
> These test-only improvements make it easier to construct system tests
> against realistic pseudo-distributed clusters.
>
> All of the above patches have been baking in trunk for quite some time.
>
> Thanks
> -Todd
> --
> Todd Lipcon
> Software Engineer, Cloudera

Mime
View raw message