hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4844) Coprocessor hooks for log rolling
Date Wed, 07 Dec 2011 00:56:41 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164022#comment-13164022

Lars Hofhansl commented on HBASE-4844:

bq. The logroller could signal new log to copy?

Right, and it could trigger a coprocessor hook to do the actual work of archiving. The coprocessor
would get the path to the old file and then copy it somewhere else.
Looking at the code, there're races, though. Until the HLogs.writer is set to the new writer,
all writes would still go to the old file. So if the coprocessor post hook is before that
and it makes a copy of the file some edit might be missed (that go into the old file after
it was copied, but before the writer was switched over).
Wouldn't it be nice if we had hardlinks in HDFS? :)

So I think the coprocessor post hook should be called after the HLog.writer assignment. If
it did the copy synchronously it only needs to finish before the next log for the same regionserver
is rolled (still a race, though).

I'll attach a very simple patch tonight or tomorrow morning and then folks can poke holes
in it.

> Coprocessor hooks for log rolling
> ---------------------------------
>                 Key: HBASE-4844
>                 URL: https://issues.apache.org/jira/browse/HBASE-4844
>             Project: HBase
>          Issue Type: New Feature
>    Affects Versions: 0.94.0
>            Reporter: Lars Hofhansl
>            Priority: Minor
> In order to eventually do point in time recovery we need a way to reliably back up the
logs. Rather than adding some hard coded changes, we can provide coprocessor hooks and folks
can implement their own policies.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message