hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17655) Removing MemStoreScanner and SnapshotScanner
Date Mon, 20 Feb 2017 08:05:44 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15874174#comment-15874174

ramkrishna.s.vasudevan commented on HBASE-17655:

Once we have composite memstore as default then even the snapshotScanner that is created for
composite memstore is nothing but a MemstoreScanner. In that case SnapshotScanner is redundant.
SnapshotScanner was infact added for that case where when the snapshot was made of a pipeline
there was a instance created for snapshotScanner and that had to explicity closed so that
the segments are  not returned back. Will check RB once.

> Removing MemStoreScanner and SnapshotScanner
> --------------------------------------------
>                 Key: HBASE-17655
>                 URL: https://issues.apache.org/jira/browse/HBASE-17655
>             Project: HBase
>          Issue Type: Improvement
>          Components: Scanners
>    Affects Versions: 2.0.0
>            Reporter: Eshcar Hillel
>            Assignee: Eshcar Hillel
>         Attachments: HBASE-17655-V01.patch
> With CompactingMemstore becoming the new default, a store comprises multiple memory segments
and not just 1-2. MemStoreScanner encapsulates the scanning of segments in the memory part
of the store. SnapshotScanner is used to scan the snapshot segment upon flush to disk.
> Having the logic of scanners scattered in multiple classes (StoreScanner, SegmentScanner,
MemStoreScanner, SnapshotScanner) makes maintainance and debugging challenging tasks, not
always for a good reason.
> For example, MemStoreScanner has a KeyValueHeap (KVH). When creating the store scanner
which also has a KVH, this makes a KVH inside a KVH. Reasoning about the correctness of the
methods supported by the scanner (seek, next, hasNext, peek, etc.) is hard and debugging 
them is cumbersome. 
> In addition, by removing the MemStoreScanner layer we allow store scanner to filter out
each one of the memory scanners instead of either taking them all (in most cases) or discarding
them all (rarely).
> SnapshotScanner is a simplified version of SegmentScanner as it is used only in a specific
context. However it is an additional implementation of the same logic with no real advantage
of improved performance.
> Therefore, I suggest removing both MemStoreScanner and SnapshotScanner. The code is adjusted
to handle the list of segment scanners they encapsulate.
> This fits well with the current code since in most cases at some point a list of scanner
is expected, so passing the actual list of segment scanners is more natural than wrapping
a single (high level) scanner with Collections.singeltonList(...).

This message was sent by Atlassian JIRA

View raw message