Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: 
 <CA+RK=_CXqAu6HR_paGdXHupdzaci0QRfaVo9RxbMm-Dm1cpGKA@mail.gmail.com>
References: 
 <CAPQV63XHe2zvh9LbrYpJzC3b5SD9rF9x=DS+9Hzwnz+SQv-P4w@mail.gmail.com>
 <CAAT7MkoGa7ArdiMakps-2nrJzd_EM0GvkAu4LXocoBYBJBns0A@mail.gmail.com>
 <CA+RK=_DSAq_vpCGc9wdOmdZGFPRm-JrE0m85ozr9T-8u9Ax7BQ@mail.gmail.com>
 <CAPQV63VCOhcDNE8TWXCvDUMW-sDYpztZHDT3QHNUoNeXMd=nJg@mail.gmail.com>
 <CAMUu0w8Z30V=dd1RtbVt3CN9CG=ydMYBaVgN7WMk1VZihTO60g@mail.gmail.com>
 <CAPQV63WcPHb2mQYyQR4bEcjVBOOKHBSEwdvLB5KCQEDnyavZgg@mail.gmail.com>
 <1363490631.4437.YahooMailNeo@web140606.mail.bf1.yahoo.com>
 <CA+RK=_CXqAu6HR_paGdXHupdzaci0QRfaVo9RxbMm-Dm1cpGKA@mail.gmail.com>
From: Jean-Marc Spaggiari <jean-marc@spaggiari.org>
Date: Sun, 17 Mar 2013 12:47:42 -0400
Message-ID: 
 <CAPQV63VCpyCfUwazWOa4NXmBe3fJxSayKvaFwWfP4vbbibaD=g@mail.gmail.com>
Subject: Re: Performances Tests
To: dev@hbase.apache.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

@Lars: here is an example of what I use
for i in {1..10}; do echo; echo -n $i ; rm -rf /tmp/*;
bin/start-hbase.sh; sleep 60; bin/hbase
org.apache.hadoop.hbase.PerformanceEvaluation sequentialWrite 1;
bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=3D100
filterScan 1; bin/stop-hbase.sh; done &>> output.txt

Giving only 1 as the readers allow to not launch the test as a MR. I
have not try the same tests with a bigger number, but I can.

@Andy: LoadTestTool is also on my list, but for YCSB it's a but harder
since I will need more than one dedicated computer. But I will look at
it too. I might be able to remove 1 node from my cluster and dedicate
it to the tests when required...


I will be pretty buzy next week, but before next week-end I will try
to prepare the LoadTestTool scripts to run something similar. With
multiple scenarios, like with and whithout bloom, etc.

If yu have any recommandation/request, feel free! I will come back
soon with some additionnal numbers.

JM

2013/3/17 Andrew Purtell <apurtell@apache.org>:
> I'm not sure I trust the results of PerformanceEvauation.
>
> LoadTestTool and YCSB have their own issues but seem to produce more
> consistent results. I've been thinking on adding scanning and filtering
> tests to LoadTestTool.
>
> On Sunday, March 17, 2013, lars hofhansl wrote:
>
>> Cool. The 0.94.3 scanning improvements seems almost unbelievable
>> (especially since many of my improvements to reduce the internal frictio=
n
>> went into 0.94.4).
>> I would like to track down the random read regression.
>>
>> Can you send the commands you ran? Are you running this as M/R job or
>> standalone client?
>>
>> Thanks for doing this J-M.
>>
>>
>> -- Lars
>>
>>
>>
>> ________________________________
>>  From: Jean-Marc Spaggiari <jean-marc@spaggiari.org <javascript:;>>
>> To: dev@hbase.apache.org <javascript:;>
>> Sent: Saturday, March 16, 2013 7:03 PM
>> Subject: Re: Performances Tests
>>
>> Hi Enis,
>>
>> "interesting" in the positive way ;)
>>
>> Results are there:
>>
>> http://www.spaggiari.org/media/blogs/hbase/pictures/performances-1.pdf?m=
time=3D1363484477
>>
>> The improvment on scan are impressive. sequentialRead and randomScan wen=
t
>> down.
>>
>> In ran the 0.94.6 tests with RC2. If we have a RC3 I will rerun them.
>>
>> I will add HFilePerformanceEvaluation soon but I'm facinf some issues
>> with it on previous HBase version...
>>
>> JM
>>
>> 2013/3/12 Enis S=C3=B6ztutar <enis.soz@gmail.com>:
>> >> I just finished to run all the PerformanceEvaluation tests on a
>> > dedicated computer with all 0.9x.x HBase versions, and I found results
>> > interesting.
>> > Can you please provide your numbers if you can. What is interesting fr=
om
>> > your findings?
>> >
>> > Enis
>> >
>> >
>> >
>> > On Tue, Mar 12, 2013 at 5:41 PM, Jean-Marc Spaggiari <
>> > jean-marc@spaggiari.org> wrote:
>> >
>> >> If you run only 1 client with PerformanceEvaluation, it's not running
>> >> it over MapReduce, so you don't have this overhead. But you can still
>> >> run it if you want to have something more distributed. Might be usefu=
l
>> >> to have the 2 options. But at the end, LoadTestTool or
>> >> PerformanceEvaluation, any of the 2 is good as long as we are adding
>> >> those tests.
>> >>
>> >> I just finished to run all the PerformanceEvaluation tests on a
>> >> dedicated computer with all 0.9x.x HBase versions, and I found result=
s
>> >> interesting. That gives us a good baseline to see if new HBase
>> >> improvements are really improving performances.
>> >>
>> >> JM
>> >>
>> >> 2013/3/8 Andrew Purtell <apurtell@apache.org>:
>> >> > Tangentally: I think I prefer LoadTestTool over
>> PerformanceEvaluation, it
>> >> > doesn't depend on nor is influenced by MapReduce job startup.
>> >> >
>> >> >
>> >> > On Fri, Mar 8, 2013 at 10:05 PM, ramkrishna vasudevan <
>> >> > ramkrishna.s.vasudevan@gmail.com> wrote:
>> >> >
>> >> >> @JM
>> >> >> I agree with you.  Mainly the perf improvement changes needs some
>> >> >> testcases.
>> >> >> But sometimes the scenario on which the perf improvments happens a=
re
>> bit
>> >> >> difficult to generate and we will be able to do in a standalone ca=
se
>> >> only.
>> >> >>  May be overall if we need to get that perf improvment result we
>> need a
>> >> >> real cluster with suitable data.  That is what i have experienced.
>> Just
>> >> >> telling.
>> >> >>
>> >> >> Regards
>> >> >> Ram
>> >> >>
>> >> >> On Fri, Mar 8, 2013 at 7:28 PM, Jean-Marc Spaggiari <
>> >> >> jean-marc@spaggiari.org
>> >> >> > wrote:
>> >> >>
>> >> >> > Hi,
>> >> >> >
>> >> >> > In HBase we already have PerformanceEvaluation which gives us a
>> good
>> >> >> > way to validate that nothing broke HBase speed in the recent
>> updates.
>> >> >> >
>> >> >> > I can see in the JIRAs many improvements coming, like for the la=
zy
>> >> >> > seeks, the bloom filters, etc. however, there is no tests for th=
ose
>> >> >> > improvements.
>> >> >> >
>> >> >> > Will it not be good to ask people to add some new tests in
>> >> >> > PerformanceEvaluation when they are introducing an improvement
>> which
>> >> >> > is not covered there?
>> >> >> >
>> >> >> > We should not touch existing tests because we need to have a way=
 to
>> >> >> > compare the baseline between the different versions, but we can
>> still
>> >> >> > add some new. Like in addition to RandomSeekScanTest we can add
>> >> >> > RandomSeekScanBloomEnabledTest and so on. And even better if we =
can
>> >> >> > back port those new tests to previous version.
>> >> >> >
>> >> >> > The s
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)