hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7253) Compaction Tool
Date Sat, 01 Dec 2012 23:35:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508101#comment-13508101
] 

stack commented on HBASE-7253:
------------------------------

It looks great Matteo.  You have actually used this tool in production already, is that not
true: i.e. it works (smile).

Would suggest you add to the usage explanation of -compactOnce and -mapred args.  I'd image
-mapred means run it as a mapreduce job and if this flag is not passed, then we will run it
locally.  Not sure of -compactOnce.  Would suggest you also add the usage as the release note
on this issue so folks can learn about this new facility and that its easy to use.

Good stuff Matteo.
                
> Compaction Tool
> ---------------
>
>                 Key: HBASE-7253
>                 URL: https://issues.apache.org/jira/browse/HBASE-7253
>             Project: HBase
>          Issue Type: New Feature
>          Components: Compaction
>    Affects Versions: 0.96.0
>            Reporter: Matteo Bertozzi
>            Assignee: Matteo Bertozzi
>            Priority: Minor
>             Fix For: 0.96.0
>
>         Attachments: HBASE-7253-v0.patch
>
>
> In HBASE-5616, as part of the compaction code refactor, a CompactionTool was added.
> but there are some issues:
> * The tool is under test/
> * mockito is required, so the "test" scope should be removed from the pom.xml, otherwise
the tool doesn't start
> * The mock, used by the tool, is mocking HRegion.getRegionInfo() but some code (Store)
uses HRegion.regionInfo directly HStore.java#L2021,  HStore.java#L1389, HStore.java#L1402
and you end up with a NPE in the tool.
> * The Mocked Store uses a dummy family and the compacted files doesn't get the same family
properties specified (compression, encoding, ...)
> * at the end of compaction CompactionTool.java#L155, on by default, the compaction file
is removed (note that the compacted one are already removed inside the store.compact()...
and you end up with an empty dir, if you compact everything.
> I've fixed some stuff and added support to:
>  * Run the compaction as a MR Job
>  * Specify a Table (compact each region/family)
>  * Specify a Region (compact each family)
>  * Specify a Family (as before)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message