hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Liyin Tang (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5403) Checkpoint the compressed HLog
Date Wed, 15 Feb 2012 22:00:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13208831#comment-13208831

Liyin Tang commented on HBASE-5403:

@Nicolas, The block size in the DFS usually will be set quite large, let's say 256M. And it
is inefficient to write small log file which is less than one dfs block. I asume this is the
main benefit of checkpointing vs rolling the log.

> Checkpoint the compressed HLog
> ------------------------------
>                 Key: HBASE-5403
>                 URL: https://issues.apache.org/jira/browse/HBASE-5403
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Liyin Tang
>            Assignee: Liyin Tang
> Let's assume that HBase replication can be based on replaying the HLog in the replica
> The replica process could be crash during the replay. Obviously, the replica process
need a way to start from the lastest check point in the HLog, even the HLog is compressed.
> So the proposal is to write a series of checkpoints within the HLog. 
> Each each checkpoint will have a header with some special sequence of bytes.
> And between each checkpoints, HLog should use new dictionaries to compress.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message