hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: HDFS-895 hflush speedup
Date Thu, 15 Apr 2010 18:54:20 GMT
On Thu, Apr 15, 2010 at 10:51 AM, Kannan Muthukkaruppan <Kannan@facebook.com
> wrote:

> Todd: Good stuff! Have you tried this out on a HBase test too?
>
>
Not yet - I basically had a burst of inspiration last night around 11pm,
coded til I couldn't stay awake, and threw up a patch :) I hope to give it a
try today (and also update patch to trunk so I can get some reviews there)


> > I think with it on, we can practically turn off group commit and get
> quite a bit speedup.
>
> Will batching now happen underneath the DFSClient automatically now that
> syncs/appends are not serialized.
>
>
It's not quite batching, but essentially I think we get the same benefit.
While one thread is waiting for *its* packet/write to be synced, another
thread can still append to the stream and call sync for its own data.

-Todd

-----Original Message-----
> From: Todd Lipcon [mailto:todd@cloudera.com]
> Sent: Thursday, April 15, 2010 1:59 AM
> To: hbase-dev@hadoop.apache.org
> Subject: HDFS-895 hflush speedup
>
> Any of you out there who are running the 0.20+200+etc hadoop with 0.20
> branch hbase want to try this patch? I think with it on, we can practically
> turn off group commit and get quite a bit speedup.
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>



-- 
Todd Lipcon
Software Engineer, Cloudera

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message