hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7413) Convert WAL to pb
Date Thu, 25 Apr 2013 20:22:18 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13642167#comment-13642167
] 

Sergey Shelukhin commented on HBASE-7413:
-----------------------------------------

Instead of size one can store record count. That impedes partial recovery in case of corrupt
buffer (i.e. with size you can just skip over size bytes to whatever is next), but is otherwise
equivalent and known in advance (in most cases if the buffer is corrupt we don't know if we
can trust the size, so the recovery benefit is also speculative).

For CellSink, I'd say the default implementation of  the whole compressor-encoder-... pipeline
should just put bytes into OutputStream, and buildCellBlock can use the pipeline and supply
its baos. So that the default mode of writing cells would be stream based.
I guess you could say it is now, via the encoder (and this is what I used in the new patch),
but then cellblock would be 2nd class citizen, IPC specific :)

As for value compression, yeah, values in WAL are not compressed.
                
> Convert WAL to pb
> -----------------
>
>                 Key: HBASE-7413
>                 URL: https://issues.apache.org/jira/browse/HBASE-7413
>             Project: HBase
>          Issue Type: Sub-task
>          Components: wal
>            Reporter: stack
>            Assignee: Sergey Shelukhin
>            Priority: Critical
>             Fix For: 0.95.1
>
>         Attachments: HBASE-7413-v0.patch, HBASE-7413-v1.patch, HBASE-7413-v2.patch, HBASE-7413-v3.patch
>
>
> From HBASE-7201

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message