hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Rawson <ryano...@gmail.com>
Subject Re: Question regarding PerformanceEvaluation.RandomSeekScanTest performance.
Date Wed, 24 Feb 2010 08:09:23 GMT
Wow that is brutal.

Are you able to make a patch?  How could we add a test to protect
ourselves from these kinds of things?

On Tue, Feb 23, 2010 at 11:56 PM, Dan Washusen <dan@reactive.org> wrote:
> It's an issue with WhileMatchFilter.  I've created issue HBASE-2258...
>
> On 24 February 2010 18:25, Ryan Rawson <ryanobjc@gmail.com> wrote:
>
>> That is very interesting, there are definitely some hidden performance
>> landmines in the filter code... you have clearly found one.
>>
>> strange in that while match filter is just a thin wrapper.
>>
>>
>>
>> On Tue, Feb 23, 2010 at 11:22 PM, Dan Washusen <dan@reactive.org> wrote:
>> > OK, a bit more info; removing the WhileMatchFilter drops the scans down
>> to
>> > 25ms each.
>> >
>> > Updated scan looks like:
>> >
>> >> Scan scan = new Scan(getRandomRow(this.rand, this.totalRows));
>> >>
>> >> scan.addColumn(FAMILY_NAME, QUALIFIER_NAME);
>> >>
>> >> scan.setFilter(new PageFilter(120));
>> >>
>> >>
>> > The slow scan uses:
>> >
>> >> Scan scan = new Scan(getRandomRow(this.rand, this.totalRows));
>> >>
>> >> scan.addColumn(FAMILY_NAME, QUALIFIER_NAME);
>> >>
>> >> scan.setFilter(new *WhileMatchFilter*(new PageFilter(120)));
>> >>
>> >>
>> >
>> >
>> > On 24 February 2010 18:17, Dan Washusen <dan@reactive.org> wrote:
>> >
>> >> Hey All,
>> >> While testing the recent MemStoreScanner slowness I noticed that each
>> scan
>> >> in the randomSeekScan test takes about 19 seconds to complete.  The scan
>> in
>> >> question provides a startRow and a WhileMatchFilter containing a
>> PageFilter
>> >> that asks for 120 rows.  I would have expected this scan to return in
>> >> roughly the same amount of time as a scan that specifies a startRow and
>> >> stopRow that spans a similar number of rows.
>> >>
>> >> Before I start investigating further I was wondering if anyone had any
>> >> thoughts?
>> >>
>> >> Cheers,
>> >> Dan
>> >>
>> >> p.s. Running the test on the latest 0.20 code...
>> >>
>> >
>>
>

Mime
View raw message