hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dhruba Borthakur <dhr...@gmail.com>
Subject Re: commit semantics
Date Wed, 13 Jan 2010 18:38:16 GMT
I will try to make a patch for it first. depending on the complexity of the
patch code, we can decide which release it can go in.

thanks,
dhruba

On Wed, Jan 13, 2010 at 9:56 AM, Jean-Daniel Cryans <jdcryans@apache.org>wrote:

> That's great dhruba, I guess the sooner it could go in is 0.21.1?
>
> J-D
>
> On Wed, Jan 13, 2010 at 8:51 AM, Dhruba Borthakur <dhruba@gmail.com>
> wrote:
> > I opened http://issues.apache.org/jira/browse/HDFS-895 for this one.
> >
> > thanks,
> > dhruba
> >
> > On Tue, Jan 12, 2010 at 9:41 PM, Joydeep Sarma <jsensarma@gmail.com>
> wrote:
> >
> >> this is internal to the dfsclient. this would explain why performance
> >> would suck with queue threshold of 1.
> >>
> >> leave it up to Dhruba to explain the details.
> >>
> >> On Tue, Jan 12, 2010 at 9:16 PM, stack <stack@duboce.net> wrote:
> >> > On Tue, Jan 12, 2010 at 9:12 PM, stack <stack@duboce.net> wrote:
> >> >
> >> >> > any IO to a HDFS-file (appends, writes, etc) ae actually blocked
on
> a
> >> >> > pending sync. "sync" in HDFS is a pretty heavyweight operation
as
> it
> >> >> stands.
> >> >>
> >> >> i think this is likely to explain limited throughput with the default
> >> >> write queue threshold of 1. if the appends cannot make progress while
> >> >> one is waiting for the sync - then the write pipeline is going to be
> >> >> idle most of the time (with queue threshold of 1).
> >> >>
> >> >> i think it would be good to have the sync not block other writers on
> >> >> the file/pipeline. logically - it's not clear why it needs to (since
> >> >> the sync is just a wait for the completion as of some write
> >> >> transaction id - allowing new ones to be queued up subsequently).
> >> >
> >> >
> >> > Are you talking about internal to DFSClient Joydeep?  Or some
> >> > synchronization block up in hlog?
> >> >
> >> > St.Ack
> >> >
> >>
> >
> >
> >
> > --
> > Connect to me at http://www.facebook.com/dhruba
> >
>



-- 
Connect to me at http://www.facebook.com/dhruba

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message