hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jesse Yates (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6055) Snapshots in HBase 0.96
Date Wed, 26 Sep 2012 23:23:09 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464276#comment-13464276

Jesse Yates commented on HBASE-6055:

bq. When you're talking about hfiles, you are referring to the log files right? I've a bit
a of confusion reading your comment, bacause the log files are sequence files. anyway...

Oops, typing tired. Yeah, I mean hlogs the entire time.

The logs in /hbase/.logs are splitted (new files are created in region/recover.edits) and
if you look at HRegion.replayRecoveredEditsIfAny(), the content of recover.edits is removed
as soon as the edits are applied. Removed, not archived. And this means that as soon as the
table goes online, the snapshot doesn't have a way to read those files.

but as you've said, the original (full) log is still available during split, but moved to
the archive (.oldlogs) as soon as the split is done.

This means that if you see files in recover.edits, you should have the full logs in /hbase/.logs
folder. And you can keep a reference to them, as you do for the online snapshot

Keeping all the logs in .oldlogs as well as .logs will cover a LOT more hlogs than are necessary
to restore the table. Better would be just just reference all the files in the recovered.edits
directory, but I worry that there will probably be some race conditions (especially in cases
where a server is brought up and down multiple times). Easier just seems to be to remove the
log file when when all the recovered.edits are finished. For instance, we could use the FileLink
stuff Matteo is working on to ref-count that hlog and only delete it when the last 'reference'
(or file derived from that hlog) is gone from the recovered.edits directory
> Snapshots in HBase 0.96
> -----------------------
>                 Key: HBASE-6055
>                 URL: https://issues.apache.org/jira/browse/HBASE-6055
>             Project: HBase
>          Issue Type: New Feature
>          Components: Client, master, regionserver, snapshots, Zookeeper
>            Reporter: Jesse Yates
>            Assignee: Jesse Yates
>             Fix For: hbase-6055, 0.96.0
>         Attachments: Snapshots in HBase.docx
> Continuation of HBASE-50 for the current trunk. Since the implementation has drastically
changed, opening as a new ticket.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message