hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8496) Implement tags and the internals of how a tag should look like
Date Fri, 28 Jun 2013 08:09:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695314#comment-13695314

ramkrishna.s.vasudevan commented on HBASE-8496:

bq.Add author, date, and JIRA you are referring to.
Will do.
bq.Should you make a TaggedKeyValueCodec or you think it fine just making KeyValueCodec handle
tags if present?
I think KeyValueCodec will server the purpose because KVCodec does not mind what is the internal
byte structure so with or without tag is should be able to handle.  What you feel?
bq.If tags as attribute, maybe have a special key, something like, tags.visibility.
A special constant you mean? Should be fine. Mine just showed an example we can make that
key something like a keyword.
bq. Besides, hfile is broke at the moment in that it only knows one kind of kv serialization
and it is baked in everwhere. How you propose we fix this? Can we move hfile to be Cell based?
HFile to cell based should be the best choice but converting the current hfile blocks to cell
based blocks  i need to check the changes.  Let me give a try at it.
bq.We can do an alternate serialization later, one that keeps tags close so we can spin through
them fast w/o having to read value data.
Making alternate serialization later may involve changes to the existing comparators i feel,
may be with Cell based this change should be simple i think.
bq.HFileBlockEncoders also are broke in that they presume the KV serialization format. HFileBlockEncoders
should be like the new Cell Codecs.
Yes.  Everywhere the code just tries to go with KV format.  
bq.When you say additional 'read', do you mean a new seek?
No i meant the read issued on the byte buffers.
bq.Could hfile blocks be cellblocks? And in meta data for hfile say what decoder to use?
bq.If Cells, you'll get your tags carried along for you. Your CPs could then exploit them
when present. The serialization could be your proposed KV-with-tags-on-the-end but when we
touch hfile, can we pass cellblocks and read cellblocks (encoding and/or compression could
be metadata in hfile – I suppose this means one encoding/compression per file which is probably
fine. Compressed/encoded blocks in memory would have to be accessed w/ CellScanner... would
need resettable CellScanner though I suppose.
As said above will check this also.  May be i can comment on this based on the analysis that
i do in next couple of days and get back on this.  

If we try to use KeyValueCodec and try to use it per Cellblock then the KeyValueCodec.decoder()
should work on the Cellblock but rather KeyValueCodec does not help much in the decoding time.

The performance results with KVCodec seemed to be performing less than the other prototypes
that i tried out.
> Implement tags and the internals of how a tag should look like
> --------------------------------------------------------------
>                 Key: HBASE-8496
>                 URL: https://issues.apache.org/jira/browse/HBASE-8496
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 0.98.0
>         Attachments: Tag design.pdf
> The intent of this JIRA comes from HBASE-7897.
> This would help us to decide on the structure and format of how the tags should look

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message