hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-18221) Switch from pread to stream should happen under HStore's reentrant lock
Date Fri, 16 Jun 2017 11:27:01 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051781#comment-16051781

ramkrishna.s.vasudevan commented on HBASE-18221:

I verified with a test case. Ya the problem is not directly by CompacitonHFileDischarger but
it is due to the fact that a compaction happens when we try to switch to pread and recreate
But [~Apache9] - even if we try to add the compacted files also - still there is a gap between
the time the compacted files gets updated in StorefileManager#addCompactionResults() and the
time we get the compactedFile list. 
Am saying this because I even tried that but still there is  gap so the best way is to protect
the trySwitchToPread under HStore's reentrant lock - then we are fine. And I don't find anything
wrong there because every scanner is created under HStore's lock only. 
What is your opinion? Thanks.

> Switch from pread to stream should happen under HStore's reentrant lock
> -----------------------------------------------------------------------
>                 Key: HBASE-18221
>                 URL: https://issues.apache.org/jira/browse/HBASE-18221
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Scanners
>    Affects Versions: 2.0.0, 3.0.0, 2.0.0-alpha-1
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 2.0.0, 3.0.0, 2.0.0-alpha-2
> Found this while debugging HBASE-18186. When we try to reopen the scanners on the storefiles
while trying to switch over from pread to stream, we do not use the HStore's reentrant lock
to get the current Storefiles from the StoreFileManager. All the scan APIs are guarded under
that and we must do it here also other wise the CompactedHfileDischarger may cause race issues
with the HStore's datastructures like here
> {code}
> 2017-06-14 18:16:17,223 WARN  [RpcServer.default.FPBQ.Fifo.handler=23,queue=1,port=16020]
regionserver.StoreScanner: failed to switch to stream read
> java.lang.NullPointerException
>         at org.apache.hadoop.hbase.regionserver.StoreFileScanner.getScannersForStoreFiles(StoreFileScanner.java:133)
>         at org.apache.hadoop.hbase.regionserver.HStore.getScanners(HStore.java:1221)
>         at org.apache.hadoop.hbase.regionserver.StoreScanner.trySwitchToStreamRead(StoreScanner.java:997)
>         at org.apache.hadoop.hbase.regionserver.StoreScanner.shipped(StoreScanner.java:1134)
>         at org.apache.hadoop.hbase.regionserver.KeyValueHeap.shipped(KeyValueHeap.java:445)
>         at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.shipped(HRegion.java:6459)
>         at org.apache.hadoop.hbase.regionserver.RSRpcServices$RegionScannerShippedCallBack.run(RSRpcServices.java:339)
>         at org.apache.hadoop.hbase.ipc.ServerCall.setResponse(ServerCall.java:252)
>         at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:166)
>         at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:278)
>         at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:258)
> {code}
> I have a working patch fixing this problem. Will do some more testing and try to upload
the patch after I write a test case for this. 

This message was sent by Atlassian JIRA

View raw message