hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Rawson <ryano...@gmail.com>
Subject Re: Question regarding PerformanceEvaluation.RandomSeekScanTest performance.
Date Wed, 24 Feb 2010 07:25:38 GMT
That is very interesting, there are definitely some hidden performance
landmines in the filter code... you have clearly found one.

strange in that while match filter is just a thin wrapper.



On Tue, Feb 23, 2010 at 11:22 PM, Dan Washusen <dan@reactive.org> wrote:
> OK, a bit more info; removing the WhileMatchFilter drops the scans down to
> 25ms each.
>
> Updated scan looks like:
>
>> Scan scan = new Scan(getRandomRow(this.rand, this.totalRows));
>>
>> scan.addColumn(FAMILY_NAME, QUALIFIER_NAME);
>>
>> scan.setFilter(new PageFilter(120));
>>
>>
> The slow scan uses:
>
>> Scan scan = new Scan(getRandomRow(this.rand, this.totalRows));
>>
>> scan.addColumn(FAMILY_NAME, QUALIFIER_NAME);
>>
>> scan.setFilter(new *WhileMatchFilter*(new PageFilter(120)));
>>
>>
>
>
> On 24 February 2010 18:17, Dan Washusen <dan@reactive.org> wrote:
>
>> Hey All,
>> While testing the recent MemStoreScanner slowness I noticed that each scan
>> in the randomSeekScan test takes about 19 seconds to complete.  The scan in
>> question provides a startRow and a WhileMatchFilter containing a PageFilter
>> that asks for 120 rows.  I would have expected this scan to return in
>> roughly the same amount of time as a scan that specifies a startRow and
>> stopRow that spans a similar number of rows.
>>
>> Before I start investigating further I was wondering if anyone had any
>> thoughts?
>>
>> Cheers,
>> Dan
>>
>> p.s. Running the test on the latest 0.20 code...
>>
>

Mime
View raw message