hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Varun Sharma <va...@pinterest.com>
Subject Re: 30% random performance in 0.95+
Date Sat, 29 Jun 2013 19:00:35 GMT
I did some tests yesterday, on this. I will send them in a separate thread.


On Sat, Jun 29, 2013 at 5:10 AM, lars hofhansl <larsh@apache.org> wrote:

> In my measurements 0.94 has been getting faster with each release in both
> read and write performance.
> I wonder how representative PE is after all; it only tests via the local
> FS layer (not HDFS), among other issues.
>
> -- Lars
>
>
>
> ----- Original Message -----
> From: Jean-Marc Spaggiari <jean-marc@spaggiari.org>
> To: dev@hbase.apache.org; lars hofhansl <larsh@apache.org>
> Cc:
> Sent: Friday, June 28, 2013 8:03 PM
> Subject: Re: 30% random performance in 0.95+
>
> I think we should do that on 0.94 as well. I don't see any good reason
> to not do it.
>
> JM
>
> 2013/6/28 lars hofhansl <larsh@apache.org>:
> > Yep.
> > Now the question is: Make these changes to 0.94 as well? Or just
> document these better.
> >
> > -- Lars
> >
> >
> >
> > ----- Original Message -----
> > From: Andrew Purtell <apurtell@apache.org>
> > To: "dev@hbase.apache.org" <dev@hbase.apache.org>; lars hofhansl <
> larsh@apache.org>
> > Cc:
> > Sent: Friday, June 28, 2013 2:08 PM
> > Subject: Re: 30% random performance in 0.95+
> >
> > I've been thinking about how to periodically search through some of our
> > parameter space to see what changes to defaults are better all the way
> > around. Probably will so something based on Bigtop.
> >
> >
> > On Friday, June 28, 2013, lars hofhansl wrote:
> >
> >> And indeed just this makes a tremendous difference. Unpatched 0.94 with
> >> 40% block cache configured is actually faster than 0.95 with the same
> block
> >> cache size.
> >>
> >> -- Lars
> >>
> >>
> >>
> >> ----- Original Message -----
> >> From: lars hofhansl <larsh@apache.org <javascript:;>>
> >> To: "dev@hbase.apache.org <javascript:;>" <dev@hbase.apache.org
> <javascript:;>
> >> >
> >> Cc:
> >> Sent: Friday, June 28, 2013 1:34 PM
> >> Subject: Re: 30% random performance in 0.95+
> >>
> >> Thanks JM,
> >>
> >> HBASE-8450 (r1485562) is interesting. It increases (among other things)
> >> the block cache percentage from 24 to 40%, which would lead to a higher
> >> probability of a future random read to hit an already cached block.
> >>
> >>
> >> -- Lars
> >>
> >>
> >>
> >> ----- Original Message -----
> >> From: Jean-Marc Spaggiari <jean-marc@spaggiari.org <javascript:;>>
> >> To: dev@hbase.apache.org <javascript:;>; lars hofhansl <
> larsh@apache.org<javascript:;>
> >> >
> >> Cc:
> >> Sent: Friday, June 28, 2013 1:18 PM
> >> Subject: Re: 30% random performance in 0.95+
> >>
> >> I have the script done to run over a list of "svn releases", so if
> >> required, just give me a bunch of them or a range and I can restart.
> >> Just keep me posted.
> >>
> >> JM
> >>
> >> 2013/6/28 lars hofhansl <larsh@apache.org <javascript:;>>:
> >> > I did a few more test (on my laptop, which is not quite
> representative),
> >> and found only a 2-3% improvement from HBASE-8001+HBASE-8012 in the end.
> >> > I'll look through the issues that you identified.
> >> >
> >> > -- Lars
> >> >
> >> >
> >> >
> >> > ----- Original Message -----
> >> > From: Jean-Marc Spaggiari <jean-marc@spaggiari.org <javascript:;>>
> >> > To: dev@hbase.apache.org <javascript:;>
> >> > Cc:
> >> > Sent: Friday, June 28, 2013 12:51 PM
> >> > Subject: Re: 30% random performance in 0.95+
> >> >
> >> > Sorry folks,
> >> >
> >> > I'm a bit late to run the tests... 0.94.8 and 0.94.9 are currently
> >> > running, but here is what I have been able to capture so far for 0.95
> >> > over the last year:
> >> > r1357480 1513196
> >> > r1367009 1440244.4
> >> > r1375812 1287143.5
> >> > r1381671 1287200.2
> >> > r1388620 1295262.6
> >> > r1394335 1022140.2
> >> > r1403898 884171.9
> >> > r1410631 804229.9
> >> > r1419787 846816.9
> >> > r1426557 853535.3
> >> > r1433514 873265.1
> >> > r1438972 840666.9
> >> > r1446106 877432.2
> >> > r1452661 883974.8
> >> > r1458421 882233.3
> >> > r1464267 847000.8
> >> > r1478964 877433.5
> >> > r1485868 744905.5
> >> > r1494869 765105.9
> >> >
> >> > So seems that there was some improvements between r1367009 and
> >> > r1403898 but they are old. Also another major improvement between
> >> > r1478964 and r1485868...
> >> >
> >> > Let me know if you want me to dig further and I will be very happy to
> do
> >> so.
> >> >
> >> > JM
> >> >
> >> > 2013/6/28 Stack <stack@duboce.net <javascript:;>>:
> >> >> On Fri, Jun 28, 2013 at 10:53 AM, lars hofhansl <larsh@apache.org
> <javascript:;>>
> >> wrote:
> >> >>
> >> >>> I partially tracked this down to HBASE-8001 and HBASE-8012 by
> looking
> >> at
> >> >>> the call stacks in a profiling session.
> >> >>> HBASE-8767 is a backport of both patched to 0.94.
> >> >>>
> >> >>
> >> >> Sounds like nice work by Raymond Liu...
> >> >> St.Ack
> >> >
> >>
> >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message