Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 5 Nov 2015 01:12:27 +0000 (UTC)
From: "Enis Soztutar (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12776620.1424500574000.158394.1446685947709@Atlassian.JIRA>
In-Reply-To: <JIRA.12776620.1424500574000@Atlassian.JIRA>
References: <JIRA.12776620.1424500574000@Atlassian.JIRA>
 <JIRA.12776620.1424500574747@arcas>
Subject: [jira] [Commented] (HBASE-13082) Coarsen StoreScanner locks to
 RegionScanner
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14990872#comment-14990872 ] 

Enis Soztutar commented on HBASE-13082:
---------------------------------------

bq. HDFS does not allow to rename if there are any active readers. 
I don't think this is how hdfs works. The FileLink.FileLinkInputStream is especially for this particular reason where an open file may get moved to a different directory. The way region replicas work is that they rely on having a high TTL for the hfiles so that even if primary replica compacts the file away and moves the file to the archive directory, the secondary replica will be able to still read from the archive location. 

 
> Coarsen StoreScanner locks to RegionScanner
> -------------------------------------------
>
>                 Key: HBASE-13082
>                 URL: https://issues.apache.org/jira/browse/HBASE-13082
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Lars Hofhansl
>            Assignee: ramkrishna.s.vasudevan
>         Attachments: 13082-test.txt, 13082-v2.txt, 13082-v3.txt, 13082-v4.txt, 13082.txt, 13082.txt, HBASE-13082.pdf, HBASE-13082_1.pdf, HBASE-13082_1_WIP.patch, HBASE-13082_2_WIP.patch, HBASE-13082_3.patch, HBASE-13082_4.patch, HBASE-13082_9.patch, HBASE-13082_9.patch, gc.png, gc.png, gc.png, hits.png, next.png, next.png
>
>
> Continuing where HBASE-10015 left of.
> We can avoid locking (and memory fencing) inside StoreScanner by deferring to the lock already held by the RegionScanner.
> In tests this shows quite a scan improvement and reduced CPU (the fences make the cores wait for memory fetches).
> There are some drawbacks too:
> * All calls to RegionScanner need to be remain synchronized
> * Implementors of coprocessors need to be diligent in following the locking contract. For example Phoenix does not lock RegionScanner.nextRaw() and required in the documentation (not picking on Phoenix, this one is my fault as I told them it's OK)
> * possible starving of flushes and compaction with heavy read load. RegionScanner operations would keep getting the locks and the flushes/compactions would not be able finalize the set of files.
> I'll have a patch soon.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)