hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4608) HLog Compression
Date Wed, 14 Mar 2012 06:39:40 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13229022#comment-13229022

stack commented on HBASE-4608:

@Lars Generalizing the compression done here is out of scope for this issue.  The patch was
not written that way from the get go.  The reviews done up to like v22odd made no mention
of supporting other compression types.  I'd suggest we do it in another issue if and when
its wanted.

Let me put v27 up on rb.

bq. I forget, do we also SNAPPY/LZO/GZ compress the HLogs?

We don't do this because these compression algorithms work in blocks of 32k or so.  If not
tied off probably on the end we could lose up to 32k of edits.
> HLog Compression
> ----------------
>                 Key: HBASE-4608
>                 URL: https://issues.apache.org/jira/browse/HBASE-4608
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Li Pi
>            Assignee: stack
>             Fix For: 0.94.0
>         Attachments: 4608-v19.txt, 4608-v20.txt, 4608-v22.txt, 4608v1.txt, 4608v13.txt,
4608v13.txt, 4608v14.txt, 4608v15.txt, 4608v16.txt, 4608v17.txt, 4608v18.txt, 4608v23.txt,
4608v24.txt, 4608v25.txt, 4608v27.txt, 4608v5.txt, 4608v6.txt, 4608v7.txt, 4608v8fixed.txt
> The current bottleneck to HBase write speed is replicating the WAL appends across different
datanodes. We can speed up this process by compressing the HLog. Current plan involves using
a dictionary to compress table name, region id, cf name, and possibly other bits of repeated
data. Also, HLog format may be changed in other ways to produce a smaller HLog.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message