Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Fri, 16 Jun 2017 12:29:00 +0000 (UTC)
From: "Duo Zhang (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.13079980.1497509259000.33344.1497616140045@Atlassian.JIRA>
In-Reply-To: <JIRA.13079980.1497509259000@Atlassian.JIRA>
References: <JIRA.13079980.1497509259000@Atlassian.JIRA> <JIRA.13079980.1497509259612@jira-lw-us.apache.org>
Subject: [jira] [Commented] (HBASE-18221) Switch from pread to stream should
 happen under HStore's reentrant lock
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Fri, 16 Jun 2017 12:29:05 -0000


    [ https://issues.apache.org/jira/browse/HBASE-18221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051830#comment-16051830 ] 

Duo Zhang commented on HBASE-18221:
-----------------------------------

So add a new method to HStore which will return both storefiles and compactedfiles at once? Or push some logic in the trySwitchToStreamRead method down to HStore? I mean, just collect the file paths in trySwitchToStreamRead, and pass the paths to HStore, HStore will give us the store files for these paths.

Thanks.

> Switch from pread to stream should happen under HStore's reentrant lock
> -----------------------------------------------------------------------
>
>                 Key: HBASE-18221
>                 URL: https://issues.apache.org/jira/browse/HBASE-18221
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Scanners
>    Affects Versions: 2.0.0, 3.0.0, 2.0.0-alpha-1
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 2.0.0, 3.0.0, 2.0.0-alpha-2
>
>
> Found this while debugging HBASE-18186. When we try to reopen the scanners on the storefiles while trying to switch over from pread to stream, we do not use the HStore's reentrant lock to get the current Storefiles from the StoreFileManager. All the scan APIs are guarded under that and we must do it here also other wise the CompactedHfileDischarger may cause race issues with the HStore's datastructures like here
> {code}
> 2017-06-14 18:16:17,223 WARN  [RpcServer.default.FPBQ.Fifo.handler=23,queue=1,port=16020] regionserver.StoreScanner: failed to switch to stream read
> java.lang.NullPointerException
>         at org.apache.hadoop.hbase.regionserver.StoreFileScanner.getScannersForStoreFiles(StoreFileScanner.java:133)
>         at org.apache.hadoop.hbase.regionserver.HStore.getScanners(HStore.java:1221)
>         at org.apache.hadoop.hbase.regionserver.StoreScanner.trySwitchToStreamRead(StoreScanner.java:997)
>         at org.apache.hadoop.hbase.regionserver.StoreScanner.shipped(StoreScanner.java:1134)
>         at org.apache.hadoop.hbase.regionserver.KeyValueHeap.shipped(KeyValueHeap.java:445)
>         at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.shipped(HRegion.java:6459)
>         at org.apache.hadoop.hbase.regionserver.RSRpcServices$RegionScannerShippedCallBack.run(RSRpcServices.java:339)
>         at org.apache.hadoop.hbase.ipc.ServerCall.setResponse(ServerCall.java:252)
>         at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:166)
>         at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:278)
>         at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:258)
> {code}
> I have a working patch fixing this problem. Will do some more testing and try to upload the patch after I write a test case for this. 


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)