hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2655) 2-pass compression support
Date Wed, 02 Jun 2010 15:49:41 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12874622#action_12874622
] 

stack commented on HBASE-2655:
------------------------------

@Michele Understood.  Fellas have complained about the way broke lzo manifests itself.  HBase
will actually take on writes.  Its only when it goes to flush that it drops the edits and
in a way that is essentially hidden to the client -- exceptions are thrown in the regionserver
log.  So, i'd say, make another issue if you don't mind but its not for you to fix, not unless
you are inclined.  It'd be about better user experience around choosing a compression that
is not supported or not properly installed. 

You know of this page in wiki? http://wiki.apache.org/hadoop/UsingLzoCompression? You might
want to add a note on end pointing at your new fancy stuff.  You might even change the pointer
over in the wiki home page to include BMdiff.

Good stuff.

> 2-pass compression support
> --------------------------
>
>                 Key: HBASE-2655
>                 URL: https://issues.apache.org/jira/browse/HBASE-2655
>             Project: HBase
>          Issue Type: New Feature
>          Components: io
>            Reporter: Michele (@pirroh) Catasta
>            Priority: Minor
>             Fix For: 0.21.0
>
>         Attachments: HBASE-2655.diff
>
>
> Quoting from BigTable paper: "Many clients use a two-pass custom compression scheme.
The first pass uses Bentley and McIlroy's scheme, which compresses long common strings across
a large window. The second pass uses a fast compression algorithm that looks for repetitions
in a small 16 KB window of the data. Both compression passes are very fast—they encode at
100-200 MB/s, and decode at 400-1000 MB/s on modern machines."
> The goal of this patch is to integrate a similar compression scheme in HBase.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message