hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Compression.Algorithm extensibility?
Date Mon, 28 Nov 2011 22:58:19 GMT
Seems like we're doing some over-optimization here using ordinals. Why
not just write out the full enum name into the file? A few extra bytes
in the header of a multi-MB HFile hardly seems expensive.

Could probably switch this over in a backward-compatible manner pretty
easily if someone has time to do the patch.


On Mon, Nov 28, 2011 at 9:57 AM, Stack <stack@duboce.net> wrote:
> On Mon, Nov 28, 2011 at 8:26 AM, Doug Meil
> <doug.meil@explorysmedical.com> wrote:
>> Hi folks-
>> In Compression.java there is this comment…
>>  /**
>>   * Compression algorithms. The ordinal of these cannot change or else you
>>   * risk breaking all existing HFiles out there.  Even the ones that are
>>   * not compressed! (They use the NONE algorithm)
>>   */
>>  public static enum Algorithm {
>> … which makes the prospect of adding new instances slightly terrifying.  Has anybody
considered changing the constructor to supply an ordinal value, and that way if anybody adds
a custom one they could supply a value like 9999999 or something hopefully never used by HBase?
> The comment is just to warn against damage if ordinals are removed or
> reordered.  Do whatever it is you need to make it work Doug.  It
> should be possible to insert other compressors w/o gymnastics.
> St.Ack

Todd Lipcon
Software Engineer, Cloudera

View raw message