hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7413) Convert WAL to pb
Date Tue, 23 Apr 2013 22:29:16 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13639725#comment-13639725

Sergey Shelukhin commented on HBASE-7413:

Some responses.

bq. Because we build the cell block in memory? Yeah. I don't see any way around it. You have
any suggestions? 
This is where my lack of understanding kicks in :)
If CellBlock is only a mechanism to write this stuff to RPC, then why do we need to construct
it in memory. We can just dump it into some stream.
bq. Hopefully w/ a good prefix encoding and maybe a compression (more copying?) it'll be worth
the composition in memory before we put it on the wire.
We can put these on the wire directly too, though.

bq. Not sure how to do the above. Certain cellblock types could do this – i.e. a write through
of Cells – but cellblocks that do prefix encoding will want to store up and flush at well
defined junctures. The count of Cells you should have before you start no?
The count, yes. For the certain types, perhaps cellblock should be cellsink, and it can do
whatever internally - construct in memory, write to stream, etc.?

bq. If not enough Cells, probably no need to make a CellBlock. If values area large – >
a couple of kilobytes – probably worth compressing the WAL values.
That will depend on what operations client performs. WAL currently is already compressed using
dictionary compression (if enabled).

> Convert WAL to pb
> -----------------
>                 Key: HBASE-7413
>                 URL: https://issues.apache.org/jira/browse/HBASE-7413
>             Project: HBase
>          Issue Type: Sub-task
>          Components: wal
>            Reporter: stack
>            Assignee: Sergey Shelukhin
>            Priority: Critical
>             Fix For: 0.95.1
>         Attachments: HBASE-7413-v0.patch, HBASE-7413-v1.patch, HBASE-7413-v2.patch
> From HBASE-7201

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message