hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15055) Major compaction is not triggered when both of TTL and hbase.hstore.compaction.max.size are set
Date Mon, 11 Jan 2016 07:53:39 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15091565#comment-15091565
] 

Anoop Sam John commented on HBASE-15055:
----------------------------------------

The main concern was that because of this size limit, even major compaction is skipping these
large files. (ie. irrespective of TTL is there or not I believe)..    
The fix avoids the size check based on TTL and expired ratio. I agree that later on there
are still some more logic to exclude files.   Still things look too much complicated now...
   My Qs
1.  When this max size is specified, at least the major compaction should select all files
and do a real major compaction right?  Else when those will get compacted? Possible tomb stone
removal!   This should happen irrespective of the TTL thing no?
2.  When this size limit is specified, that should be respected for at least minor compaction.
 Here there is chance that it can get violated..  (With even not a single cell TTL expired
within that big file)

Still I feel some thing problematic in the whole..  We might need better refactoring of these
things rather than adding new configs again and again.

> Major compaction is not triggered when both of TTL and hbase.hstore.compaction.max.size
are set
> -----------------------------------------------------------------------------------------------
>
>                 Key: HBASE-15055
>                 URL: https://issues.apache.org/jira/browse/HBASE-15055
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Eungsop Yoo
>            Assignee: Eungsop Yoo
>            Priority: Minor
>         Attachments: HBASE-15055-v1.patch, HBASE-15055-v2.patch, HBASE-15055-v3.patch,
HBASE-15055-v4.patch, HBASE-15055-v5.patch, HBASE-15055-v6.patch, HBASE-15055-v7.patch, HBASE-15055-v8.patch,
HBASE-15055-v9.patch, HBASE-15055.patch
>
>
> Some large files may be skipped by hbase.hstore.compaction.max.size in candidate selection.
It causes skipping of major compaction. So the TTL expired records are still remained in the
disks and keep consuming disks.
> To resolve this issue, I suggest that to skip large files only if there is no TTL expired
record.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message