hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-18055) HBASE-17917 closes the scanners while a scan is in progess for switching over to stream reads
Date Tue, 16 May 2017 06:58:04 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011847#comment-16011847
] 

ramkrishna.s.vasudevan edited comment on HBASE-18055 at 5/16/17 6:57 AM:
-------------------------------------------------------------------------

Let me try to explain. 
For simplicity sake say we have one row per block. We have totally 4 rows (4 blocks). All
are cached in L2.
The scan is reading 3 blocks (3 rows).  Tries with the next next() call. Sees that the preadMaxBytes
is reached and creates new scanners and seeks to the 3rd row (since row 3 was latest). And
then closes all the older scanners.
So the block 1 and block 2 (of row 1 and row2) are now decremented and ref count is 0 since
older scanners refers to these blocks. block 3 would have been incremented to 2 (while seeking)
and then while closing it would have become 1. 
But since block 1 or block 2 are already decremented to 0, it means any other new blocks that
gets cached during this time (by other scan) can occupy the L2 cache and so the cells that
are added in the Result are now corrupted because the actual blocks in the L2 cache are overwritten.





was (Author: ram_krish):
Let me try to explain. 
For simplicity sake say we have one row per block. We have totally 4 rows (4 blocks). All
are cached in L2.
The scan is reading 3 blocks (3 rows).  Tries with the next next() call. Sees that the preadMaxBytes
is reached and creates new scanners and seeks to the 3rd row (since row 3 was latest). And
then closes all the older scanners.
So the block 1 and block 2 (of row 1 and row2) are now decremented and ref count is 0 since
older scanners refers to these blocks. block 3 would have been incremented to 2 and then while
closing it would have become 1. 
But since block 1 or block 2 are already decremented to 0, it means any other new blocks that
gets cached during this time (by other scan) can occupy the L2 cache and so the cells that
are added in the Result are now corrupted because the actual blocks in the L2 cache are overwritten.




> HBASE-17917 closes the scanners while a scan is in progess for switching over to stream
reads
> ---------------------------------------------------------------------------------------------
>
>                 Key: HBASE-18055
>                 URL: https://issues.apache.org/jira/browse/HBASE-18055
>             Project: HBase
>          Issue Type: Bug
>          Components: regionserver, Scanners
>    Affects Versions: 2.0.0
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 2.0.0
>
>
> In HBASE-17917 tries to switch from pread to stream read when a specific size of bytes
are read. So in order to switch over, it closes the existing scanners and creates a new scanners
with pread=false.
> When we close the exisitng scanners - if the blocks are served from offheap cache we
will decrement the ref count on those blocks and if it becomes zero we make the block ready
for eviction. Then there is a chance that the result could be corrupted if new blocks occupy
the cache. So the expectation was that till the RPC call completes the response we will hold
on to the blocks that are referred by the scan. (except the last one). So trying to switch
over to stream read will break this expectation and hence TestBlockEvictionfromclient fails.




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message