hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jimmy Xiang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7967) implement compactor for stripe compactions
Date Fri, 29 Mar 2013 21:41:15 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617754#comment-13617754
] 

Jimmy Xiang commented on HBASE-7967:
------------------------------------

Some initial comments:
1. Inside one stripe, can we reuse some logic in the default compaction policy? The logic
should be similar, right?  There are many new configuration parameters, can we re-use some
from the default policy, such as max files, min files, etc?  Especially, they can be tuned
per table/column family.
2. There is a configuration assumeOrdering.  When should it be used?
3. Will we support any stripe type other than count based/size based?  If so, probably we
need to change how stripe type is configured, since it seems that we can support only two
types now .
4. For count based, do we have to always have that many stripes?  Is it ok to have a size
limit or something so that we don't have many small stripes?
5. Based on the performance test you did, the write performance is not better. You mentioned
it could be because of write amplification. Do we have some number to prove it?  If we have
more IO, should the read performance be affected too?
6. Can we have some doc to walk through the algorithm you implemented for the count/size based
compaction policy? I was wondering how some L0 files end up in a specific stripe, how each
stripe is created and maintained. Some flow-chart may be very helpful.
                
> implement compactor for stripe compactions
> ------------------------------------------
>
>                 Key: HBASE-7967
>                 URL: https://issues.apache.org/jira/browse/HBASE-7967
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Compaction
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>         Attachments: HBASE-7967-v0.patch, HBASE-7967-v0-with-stuff.patch, HBASE-7967-v1.patch,
HBASE-7967-v1-with-7679-7680.patch, HBASE-7967-v2.patch, HBASE-7967-v2-with-7679-7680.patch
>
>
> Compactor needs to be implemented. See details in parent and blocking jira.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message