hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billy Pearson (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-834) Upper bound on files we compact at any one time
Date Fri, 29 Aug 2008 07:26:44 GMT

    [ https://issues.apache.org/jira/browse/HBASE-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12626888#action_12626888
] 

Billy Pearson commented on HBASE-834:
-------------------------------------

Might check the javadocs etc not sure what I need to do there I made some changes but please
review them for me and let me know if I am missing anything.

second thought on the ttl on minor compaction a deleted record will be removed in minor compactions
if ttl is expired but the record will 
remain until the major compaction or a compaction that includes the cell to be deleted and
it will be deleted then also sense the cell its 
self will have a expired ttl we should not get it in a get/scanner. so I thank it is still
ok to leave the ttl code to do its work on minor compaction's.


> Upper bound on files we compact at any one time
> -----------------------------------------------
>
>                 Key: HBASE-834
>                 URL: https://issues.apache.org/jira/browse/HBASE-834
>             Project: Hadoop HBase
>          Issue Type: Improvement
>    Affects Versions: 0.2.1, 0.18.0
>            Reporter: stack
>            Assignee: Billy Pearson
>            Priority: Blocker
>             Fix For: 0.2.1, 0.18.0
>
>         Attachments: 834-0.2.1-patch.txt, 834-0.2.1-patchv2.txt, 834-0.2.1-patchv3.txt,
834-patch.txt
>
>
> From Billy in HBASE-64, which we closed because it got pulled all over the place:
> {code}
> Currently we do compaction on a region when the hbase.hstore.compactionThreshold is reached
- default 3
> I thank we should configure a max number of mapfiles to compact at one time simulator
to doing a minor compaction in bigtable. This keep compaction's form getting tied up in one
region to long letting other regions get way to many memcache flushes making compaction take
longer and longer for each region
> If we did that when a regions updates start to slack off the max number will eventuly
include all mapfiles causeing a major compaction on that region. Unlike big table this would
leave the master out of the process and letting the region server handle the major compaction
when it has time.
> When doing a minor compaction on a few files I thank we should compact the newest mapfiles
first leave the larger/older ones for when we have low updates to a region.
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message