hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hsieh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11644) External MOB compaction tools
Date Fri, 19 Sep 2014 01:20:34 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14139839#comment-14139839
] 

Jonathan Hsieh commented on HBASE-11644:
----------------------------------------

hey [~jingchengdu], this latest patch has problems when applied to the hbase-11339 branch
-- significantly more than HBASE-11646 had. Most of them are around TableName vs String in
the admin flush argument.  (this might also be due to patches being applied in a different
order).

I took a stab at working through most of them, but now several of the mob tests fail. (mvn
clean test -Dtest=TestMob*)  Can you rebase and please post patches that actually apply against
the target hbase-11339 branch properly?

I've attached the quick version I put together as hbase-11644-0918-bad.patch if you want to
work from that.

> External MOB compaction tools
> -----------------------------
>
>                 Key: HBASE-11644
>                 URL: https://issues.apache.org/jira/browse/HBASE-11644
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Compaction, master
>    Affects Versions: hbase-11339
>            Reporter: Jingcheng Du
>            Assignee: Jingcheng Du
>             Fix For: hbase-11339
>
>         Attachments: HBASE-11644-Sep-15.diff, HBASE-11644-Sep-16.diff, HBASE-11644-Sep-16.diff,
HBASE-11644-Sep-18.diff, HBASE-11644.diff
>
>
> From the design doc,  mob files are not involved in the normal HBase compaction process.
 This means deleted mobs would still take up space and that we never really merge mob files
that accrue over time.   Currently, MOBs depend on two external tools:
> 1) A TTL cleaner that removes mobs that have passed their TTL or exceeded minVersions.
> 2) A 'sweep tool' cleaner that remove mobs that have had their references deleted and
merges small files into larger ones.  
> Today the tools are triggered by admins.  The longer term goal would be to integrate
them into hbase such that by default mobs are cleaned.  The tools will be preserved however
so that advanced admins can disable automatic cleanups and manually trigger these compaction
like operaitons.  #1 would likely be a chore in the master while #2 requires some design work
to integrate into hbase.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message