hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dave Latham (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2248) New MemStoreScanner copies memstore for each scan, makes short scans slow
Date Tue, 23 Feb 2010 23:40:27 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12837519#action_12837519

Dave Latham commented on HBASE-2248:

@Dan: Took a read over the patch, though it seemed to be based in a different dir and didn't
want to apply nicely.  From what I can see the ScanTest still does getScanner in testSetup
before the timer is begun.  This may be fine, if the point of this test is to measure scan
performance per-row and not setup/teardown time.  It just explains why the ScanTest doesn't
exhibit this issue.  It does look like other tests, such as the RandomSeekScanTest and the
new RandomScanWithRangeTest do test setup/teardown time as part of each "testRow" and should
exhibit this issue, if run.

> New MemStoreScanner copies memstore for each scan, makes short scans slow
> -------------------------------------------------------------------------
>                 Key: HBASE-2248
>                 URL: https://issues.apache.org/jira/browse/HBASE-2248
>             Project: Hadoop HBase
>          Issue Type: Bug
>    Affects Versions: 0.20.3
>            Reporter: Dave Latham
>             Fix For: 0.20.4
>         Attachments: hbase-2248.gc, Screen shot 2010-02-23 at 10.33.38 AM.png, threads.txt
> HBASE-2037 introduced a new MemStoreScanner which triggers a ConcurrentSkipListMap.buildFromSorted
clone of the memstore and snapshot when starting a scan.
> After upgrading to 0.20.3, we noticed a big slowdown in our use of short scans.  Some
of our data repesent a time series.   The data is stored in time series order, MR jobs often
insert/update new data at the end of the series, and queries usually have to pick up some
or all of the series.  These are often scans of 0-100 rows at a time.  To load one page, we'll
observe about 20 such scans being triggered concurrently, and they take 2 seconds to complete.
 Doing a thread dump of a region server shows many threads in ConcurrentSkipListMap.biuldFromSorted
which traverses the entire map of key values to copy it.  

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message