hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Rawson <ryano...@gmail.com>
Subject Re: prefix compression implementation
Date Tue, 20 Sep 2011 05:41:33 GMT
So if the HCell or whatever ends up returning ByteBuffers, then that
plays straight in to scatter/gather NIO calls, and if some of them are
DBB, then so much the merrier.

For example, the thrift stuff takes ByteBuffers when its calling for a
byte sequence.


On Mon, Sep 19, 2011 at 10:39 PM, Stack <stack@duboce.net> wrote:
> One other thought is that exposing ByteRange, ByteBuffer, and v1 array
> stuff in Interface seems like you are exposing 'implementation'
> details that perhaps shouldn't show through.  I'm guessing its
> unavoidable though if the Interface is to be used in a few different
> contexts: i.e. "v1" has to work if we are to get this new stuff in,
> some srcs will be DBBs, etc.
> St.Ack
> On Mon, Sep 19, 2011 at 10:33 PM, Stack <stack@duboce.net> wrote:
>> On Mon, Sep 19, 2011 at 3:26 PM, Matt Corgan <mcorgan@hotpads.com> wrote:
>>> I don't think the name is all that important, though i thought HCell was
>>> less clumsy than KeyValue or KeyValueInterface.  Take a look at this
>>> interface on github:
>>> https://github.com/hotpads/hbase-prefix-trie/blob/master/src/org/apache/hadoop/hbase/model/HCell.java
>>> Seems like it should be trivially easy to get KeyValue to implement that.
>>>  Then it provides the right methods to make compareTo methods that will work
>>> across different implementations.  The implementations of those methods
>>> might have an if-statement to determine the class of the "other" HCell, and
>>> choose the fastest byte comparison method behind the scenes.
>> I'd say call it Cell rather than HCell.
>> You have getRowArray rather than getRow which we currently have but I
>> suppose it makes sense since you can then group by suffix.
>> There is a patch lying around that adds a version to KV by using top
>> two bytes of the type byte.  If you need me to dig it up, just say
>> (then you might not have to have v1 stuff in your Interface).
>> You might need to add some equals for stuff like same row, cf, and
>> qualifier... but they can come later.
>> The comparator stuff is currently horrid because it depends on
>> context; i.e. whether the KVs are from -ROOT- or .META. or from a
>> userspace table.  There are some ideas for having it so only one
>> comparator for all types but thats another issue.
>> St.Ack

View raw message