hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-15881) Allow BZIP2 compression
Date Wed, 25 May 2016 01:45:12 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15299306#comment-15299306
] 

Andrew Purtell edited comment on HBASE-15881 at 5/25/16 1:44 AM:
-----------------------------------------------------------------

+1

I tried this once with LZMA, see HBASE-2987. We had trouble with LZMA specifically because
it is a super slow algorithm, so HBASE-2988 was an attempt to mitigate that by only doing
the final and expensive compression in major compaction. The major compaction was assumed
to be the final archival step for the data. This was a use case where the left portion of
the key was a time based component.

Bzip2 should compress significantly faster in comparison. Configuring more major compaction
threads would mitigate throughput issues (at expense of CPU) or the 2988 strategy could be
employed instead.


was (Author: apurtell):
+1

I tried this once with LZMA, see HBASE-2987. We had trouble with LZMA specifically because
it is a super slow algorithm, HBASE-2988 was an attempt to address it. Bzip2 should compress
significantly faster in comparison. Even so 2988 would mitigate. 

> Allow BZIP2 compression
> -----------------------
>
>                 Key: HBASE-15881
>                 URL: https://issues.apache.org/jira/browse/HBASE-15881
>             Project: HBase
>          Issue Type: New Feature
>          Components: HFile
>            Reporter: Lars Hofhansl
>         Attachments: 15881-0.98.txt
>
>
> BZIP2 is a very efficient compressor in terms of compression rate.
> Compression speed is very slow, de-compression is equivalent or faster than GZIP.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message