hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7233) Remove Writable Interface from KeyValue
Date Mon, 03 Dec 2012 00:33:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508416#comment-13508416

stack commented on HBASE-7233:

bq.  In Result we have pbs for Mutation, which serializes columns.

I don't see that.  I see:

 24 /**
 23  * For performance reason, we don't use KeyValue
 22  * here. We use the actual KeyValue bytes.
 21  */
 20 message Result {
 19   repeated bytes keyValueBytes = 1;
 18 }

... which looks like it would be a bit tough to evolve.  We should change this (Am I looking
in wrong place?)

bq. In M/R jobs there will be massive amounts of KVs that are streamed from the mapper to
the reducer, I do not think we want to introduce serialization that needs to copy each KV
in its entirety before it can serialized.

Yeah.  Could do something particular for MR but can't be something that would stop our evolving
KV/Cell over MR or over RPC.

Not sure how we'd do that though currently.  Maybe a Result has a CellInputStream into which
we write the Cells (or whatever [~mcorgan] called it -- what was it Matt?  I don't seem to
see it in committed hbase-common) and we give this blob to pb to serialize.... then then on
other end we do the CellOutputStream reading them.....

bq. I.e. we'd have allow the pb to somehow know about the row array, row offset, and rowlength,
as well as CF array/offset/length, qual arrary/offset/length, and value array/offset/length
and be able to serialize the subportions of these arrays.

IIUC, not sure this possible. I think you are implying custom pb builder?  I may have you
> Remove Writable Interface from KeyValue
> ---------------------------------------
>                 Key: HBASE-7233
>                 URL: https://issues.apache.org/jira/browse/HBASE-7233
>             Project: HBase
>          Issue Type: Bug
>            Reporter: stack
>            Assignee: stack
>            Priority: Blocker
>         Attachments: 7233.txt, 7233-v2.txt
> Undo KeyValue being a Writable.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message