Mailing-List: contact hbase-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-dev@hadoop.apache.org
Message-ID: <539972731.88811267760727283.JavaMail.jira@brutus.apache.org>
Date: Fri, 5 Mar 2010 03:45:27 +0000 (UTC)
From: "ryan rawson (JIRA)" <jira@apache.org>
To: hbase-dev@hadoop.apache.org
Subject: [jira] Commented: (HBASE-2248) Provide new non-copy mechanism to
 assure atomic reads in get and scan
In-Reply-To: <174857300.448551266881787938.JavaMail.jira@brutus.apache.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12841687#action_12841687 ] 

ryan rawson commented on HBASE-2248:
------------------------------------

I think your suggestion is a good one, the race condition is really small, and holding up a client for just a few more microseconds should be reasonable.  Once we restructure to not put logs appends between memstore puts, we are literally talking about the speed of adding a few dozen entries in an array.  There is no data copy involved, since KeyValue was already read in during RPC time, and we are talking inserting small objects into a data structure.

I originally thought of being speedy about returning, but read your own writes does make this be an issue.  I'll add in your suggestions and put this test in as well.

Thanks for the great test!

> Provide new non-copy mechanism to assure atomic reads in get and scan
> ---------------------------------------------------------------------
>
>                 Key: HBASE-2248
>                 URL: https://issues.apache.org/jira/browse/HBASE-2248
>             Project: Hadoop HBase
>          Issue Type: Bug
>    Affects Versions: 0.20.3
>            Reporter: Dave Latham
>             Fix For: 0.20.4
>
>         Attachments: HBASE-2248-demonstrate-previous-impl-bugs.patch, HBASE-2248-ryan.patch, hbase-2248.gc, HBASE-2248.patch, readownwrites-lost.2.patch, readownwrites-lost.patch, Screen shot 2010-02-23 at 10.33.38 AM.png, threads.txt
>
>
> HBASE-2037 introduced a new MemStoreScanner which triggers a ConcurrentSkipListMap.buildFromSorted clone of the memstore and snapshot when starting a scan.
> After upgrading to 0.20.3, we noticed a big slowdown in our use of short scans.  Some of our data repesent a time series.   The data is stored in time series order, MR jobs often insert/update new data at the end of the series, and queries usually have to pick up some or all of the series.  These are often scans of 0-100 rows at a time.  To load one page, we'll observe about 20 such scans being triggered concurrently, and they take 2 seconds to complete.  Doing a thread dump of a region server shows many threads in ConcurrentSkipListMap.biuldFromSorted which traverses the entire map of key values to copy it.  

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.