hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7413) Convert WAL to pb
Date Thu, 25 Apr 2013 20:00:16 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13642134#comment-13642134

stack commented on HBASE-7413:

bq. If CellBlock is only a mechanism to write this stuff to RPC, then why do we need to construct
it in memory. We can just dump it into some stream.

An implementation could do that yeah.  Other implementations will want to do encoding or compression
and will need to run with buffers to store up intermediary forms in between flushes.

Over in RPC, our RPC as is prefixes all messages with a total length.  While this constraint
is in place, we can only know the cellblock size after its creation.

bq. We can put these on the wire directly too, though.

Writing WAL, yeah (unless you need to know size in advance), on the wire, not currently (After
this refactor of rpc is done, the next refactor should be to do chunking/stream.. undo the
need of the prefatory length).

bq. The count, yes. For the certain types, perhaps cellblock should be cellsink, and it can
do whatever internally - construct in memory, write to stream, etc.?

Is it not now?  Or, is there something we need to do to make it a "CellSink"?

bq. That will depend on what operations client performs. WAL currently is already compressed
using dictionary compression (if enabled).

Not the values though?

bq. newCodedOutput

We could, yeah, or not put data into pb at all (which is what rpc is trying to do)
> Convert WAL to pb
> -----------------
>                 Key: HBASE-7413
>                 URL: https://issues.apache.org/jira/browse/HBASE-7413
>             Project: HBase
>          Issue Type: Sub-task
>          Components: wal
>            Reporter: stack
>            Assignee: Sergey Shelukhin
>            Priority: Critical
>             Fix For: 0.95.1
>         Attachments: HBASE-7413-v0.patch, HBASE-7413-v1.patch, HBASE-7413-v2.patch, HBASE-7413-v3.patch
> From HBASE-7201

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message