Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: error (nike.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: <1372454914.50775.YahooMailNeo@web140602.mail.bf1.yahoo.com>
References: <1371854791.70579.YahooMailNeo@web140605.mail.bf1.yahoo.com>
 <CAPQV63X1jB+McZOQaoJZQgU82uKGbvnHMJPY5EPsFNRfOibNJQ@mail.gmail.com>
 <CA+RK=_CkWVdjG362_KMZC4rqy3HHqiCTwEG25Jfe4H8cyvoSSw@mail.gmail.com>
 <CAPQV63WeQFYzr437r0HLZV4GWoNWr__6yG9nfgRqNMaEVS-kAQ@mail.gmail.com>
 <CAPQV63X-6a4Rojrujcj+P186FOiNVLN4GK_YQF_uB8VKHDANHA@mail.gmail.com>
 <1371907957.16360.YahooMailNeo@web140606.mail.bf1.yahoo.com>
 <CAPQV63UNGHrV+-SrfpsYXkiB1miieSGUM0PoU2dOotA0ahg-2w@mail.gmail.com>
 <1371987674.1185.YahooMailNeo@web140602.mail.bf1.yahoo.com>
 <1372441992.9879.YahooMailNeo@web140603.mail.bf1.yahoo.com>
 <CADcMMgHEAbKQM7C4xnRThhCXw=QF4bhoMRzm=ezV6FHqMxhVOg@mail.gmail.com>
 <CAPQV63W0spFToijQGyfQLFyUC05VU6Prh6MTwor3KdfR18DfvQ@mail.gmail.com>
 <1372450391.41740.YahooMailNeo@web140602.mail.bf1.yahoo.com>
 <CAPQV63U4-cWLU5B3oJKXzX=ChhqTRSUU_Xa4Bs40RAyEgWw1NA@mail.gmail.com>
 <1372451693.62784.YahooMailNeo@web140603.mail.bf1.yahoo.com>
 <1372452952.80435.YahooMailNeo@web140601.mail.bf1.yahoo.com>
 <CA+RK=_AafNy=niHSNZ6FV72ZsjGZB3=v+rbEogWR==hEzehFow@mail.gmail.com>
 <1372454914.50775.YahooMailNeo@web140602.mail.bf1.yahoo.com>
From: Jean-Marc Spaggiari <jean-marc@spaggiari.org>
Date: Fri, 28 Jun 2013 23:03:59 -0400
Message-ID: 
 <CAPQV63UKuSaFgRvKW3ErGmpwMk5F59hUp+f+H=YLn8DTNvJ16Q@mail.gmail.com>
Subject: Re: 30% random performance in 0.95+
To: dev@hbase.apache.org, lars hofhansl <larsh@apache.org>
Content-Type: text/plain; charset=UTF-8

I think we should do that on 0.94 as well. I don't see any good reason
to not do it.

JM

2013/6/28 lars hofhansl <larsh@apache.org>:
> Yep.
> Now the question is: Make these changes to 0.94 as well? Or just document these better.
>
> -- Lars
>
>
>
> ----- Original Message -----
> From: Andrew Purtell <apurtell@apache.org>
> To: "dev@hbase.apache.org" <dev@hbase.apache.org>; lars hofhansl <larsh@apache.org>
> Cc:
> Sent: Friday, June 28, 2013 2:08 PM
> Subject: Re: 30% random performance in 0.95+
>
> I've been thinking about how to periodically search through some of our
> parameter space to see what changes to defaults are better all the way
> around. Probably will so something based on Bigtop.
>
>
> On Friday, June 28, 2013, lars hofhansl wrote:
>
>> And indeed just this makes a tremendous difference. Unpatched 0.94 with
>> 40% block cache configured is actually faster than 0.95 with the same block
>> cache size.
>>
>> -- Lars
>>
>>
>>
>> ----- Original Message -----
>> From: lars hofhansl <larsh@apache.org <javascript:;>>
>> To: "dev@hbase.apache.org <javascript:;>" <dev@hbase.apache.org<javascript:;>
>> >
>> Cc:
>> Sent: Friday, June 28, 2013 1:34 PM
>> Subject: Re: 30% random performance in 0.95+
>>
>> Thanks JM,
>>
>> HBASE-8450 (r1485562) is interesting. It increases (among other things)
>> the block cache percentage from 24 to 40%, which would lead to a higher
>> probability of a future random read to hit an already cached block.
>>
>>
>> -- Lars
>>
>>
>>
>> ----- Original Message -----
>> From: Jean-Marc Spaggiari <jean-marc@spaggiari.org <javascript:;>>
>> To: dev@hbase.apache.org <javascript:;>; lars hofhansl <larsh@apache.org<javascript:;>
>> >
>> Cc:
>> Sent: Friday, June 28, 2013 1:18 PM
>> Subject: Re: 30% random performance in 0.95+
>>
>> I have the script done to run over a list of "svn releases", so if
>> required, just give me a bunch of them or a range and I can restart.
>> Just keep me posted.
>>
>> JM
>>
>> 2013/6/28 lars hofhansl <larsh@apache.org <javascript:;>>:
>> > I did a few more test (on my laptop, which is not quite representative),
>> and found only a 2-3% improvement from HBASE-8001+HBASE-8012 in the end.
>> > I'll look through the issues that you identified.
>> >
>> > -- Lars
>> >
>> >
>> >
>> > ----- Original Message -----
>> > From: Jean-Marc Spaggiari <jean-marc@spaggiari.org <javascript:;>>
>> > To: dev@hbase.apache.org <javascript:;>
>> > Cc:
>> > Sent: Friday, June 28, 2013 12:51 PM
>> > Subject: Re: 30% random performance in 0.95+
>> >
>> > Sorry folks,
>> >
>> > I'm a bit late to run the tests... 0.94.8 and 0.94.9 are currently
>> > running, but here is what I have been able to capture so far for 0.95
>> > over the last year:
>> > r1357480 1513196
>> > r1367009 1440244.4
>> > r1375812 1287143.5
>> > r1381671 1287200.2
>> > r1388620 1295262.6
>> > r1394335 1022140.2
>> > r1403898 884171.9
>> > r1410631 804229.9
>> > r1419787 846816.9
>> > r1426557 853535.3
>> > r1433514 873265.1
>> > r1438972 840666.9
>> > r1446106 877432.2
>> > r1452661 883974.8
>> > r1458421 882233.3
>> > r1464267 847000.8
>> > r1478964 877433.5
>> > r1485868 744905.5
>> > r1494869 765105.9
>> >
>> > So seems that there was some improvements between r1367009 and
>> > r1403898 but they are old. Also another major improvement between
>> > r1478964 and r1485868...
>> >
>> > Let me know if you want me to dig further and I will be very happy to do
>> so.
>> >
>> > JM
>> >
>> > 2013/6/28 Stack <stack@duboce.net <javascript:;>>:
>> >> On Fri, Jun 28, 2013 at 10:53 AM, lars hofhansl <larsh@apache.org<javascript:;>>
>> wrote:
>> >>
>> >>> I partially tracked this down to HBASE-8001 and HBASE-8012 by looking
>> at
>> >>> the call stacks in a profiling session.
>> >>> HBASE-8767 is a backport of both patched to 0.94.
>> >>>
>> >>
>> >> Sounds like nice work by Raymond Liu...
>> >> St.Ack
>> >
>>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>