hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14636) Clear HFileScannerImpl#prevBlocks in between Compaction flow
Date Tue, 20 Oct 2015 03:55:27 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14964472#comment-14964472

Anoop Sam John commented on HBASE-14636:

bq.What about closeCheckInterval? We are already on an interval checking for a close. Could
we do the shipped in here on this same interval? Or at least unite the two time-based checks?
closeCheckInterval -> This is not really time based check. This is size written based check.
Default is 10 MB. After this much sized cells written, we check whether store is still writable.
 Use this same for shipped call also?  2 concerns
1. This config can be 0 and then we won't be doing the shipped call at all. We don't want
this to happen.
2. The def 10 MB seems too high.

I first tried with a size based check only.  Then just changed to time based as it looked
simple wrt #ops also.
If we go with size based I would suggest the size limit for check to be some thing like
HConstants.DEFAULT_BLOCKSIZE * conf.get(min_files_for_compact) -> So by def 64 KB * 3 
so that there is chance for at least one block will get cleared from prevBlocks list.
This make sense?
We will need another long variable and add cell sizes to that taking some more ops to be done.
Not a big deal any way. I can do this if all agree.  And size based check is the best way

> Clear HFileScannerImpl#prevBlocks in between Compaction flow
> ------------------------------------------------------------
>                 Key: HBASE-14636
>                 URL: https://issues.apache.org/jira/browse/HBASE-14636
>             Project: HBase
>          Issue Type: Sub-task
>          Components: regionserver, Scanners
>            Reporter: Anoop Sam John
>            Assignee: Anoop Sam John
>            Priority: Blocker
>             Fix For: 2.0.0
>         Attachments: HBASE-14636.patch, HBASE-14636.patch

This message was sent by Atlassian JIRA

View raw message