hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hsieh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11339) HBase MOB
Date Tue, 02 Sep 2014 22:58:53 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14118911#comment-14118911
] 

Jonathan Hsieh commented on HBASE-11339:
----------------------------------------

bq. Related, the MOB design also attempts to avoid write amplification of large cells during
compaction, by segregating large values into separate files set outside the normal compaction
process. Rather than normal compaction, an external MapReduce based tool is used for compacting
MOB files. HBase has never required MapReduce before and we should really think hard before
introducing such a change. Are we sure the desired objectives cannot be met with a pluggable
compaction policy?

Removing the external processes that perform "mob compaction" is one of the follow up goals
and is noted at HBASE-11861.  We want to get rid of the MR dependencies because it introduces
a new piece of operational complexity and I don't want that.  I don't consider the MOB feature
to be production ready if it still requires the external process to manage this.  

The mob feature like other features that require external tooling be experimental until simplified
operationally.  We've done this before -- for example,, I call favored nodes HBASE-7932 experimental
becuase it is not "set-it-and-forget"; it requires extra processes such as an external balancer.
 For MOB, after we get the other blockers in (snapshot support, metrics) we'll revamp the
mob compaction and then remove the experimental tag.  My goal would be to get this all in
by the end of the year.

> HBase MOB
> ---------
>
>                 Key: HBASE-11339
>                 URL: https://issues.apache.org/jira/browse/HBASE-11339
>             Project: HBase
>          Issue Type: Umbrella
>          Components: regionserver, Scanners
>            Reporter: Jingcheng Du
>            Assignee: Jingcheng Du
>         Attachments: HBase MOB Design-v2.pdf, HBase MOB Design-v3.pdf, HBase MOB Design-v4.pdf,
HBase MOB Design.pdf, MOB user guide.docx, MOB user guide_v2.docx, hbase-11339-in-dev.patch
>
>
>   It's quite useful to save the medium binary data like images, documents into Apache
HBase. Unfortunately directly saving the binary MOB(medium object) to HBase leads to a worse
performance since the frequent split and compaction.
>   In this design, the MOB data are stored in an more efficient way, which keeps a high
write/read performance and guarantees the data consistency in Apache HBase.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message