hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6233) [brainstorm] snapshots: hardlink alternatives
Date Mon, 09 Jul 2012 07:57:35 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409257#comment-13409257

stack commented on HBASE-6233:

It'd be a radical change Matteo.  It feels like a hbase 2.0 kinda thing rather than a 0.96-type
change (But this is a 'brainstorm' issue so we have license to talk hypotheticals).

I think we could auto-migrate from the one format to the new; new hfiles would be written
into new location in hdfs while we'd read the old unmigrated hfiles from the old layout ("Policy"
for compatibility up to this is that versions go forward perhaps w/ a "migration step" but
preferably not and we do not have to support reverting an upgrade... thats "policy" so far).

Would we need x-row transactions updating files in .META.?  I don't think so.  Read/write
locks might be enough.

We might need to let .META. split now that it can grow largish fast.

We've had "interesting" issues updating .META. in the past: e.g. socket timeout on client
side but the edit went through anyways.... that kinda thing.  Now the repercussions of failed
or false positive fail will be larger?

Yeah, instead of looking inside hdfs, hbck will have to read .META.  In hdfs, we'd still have
tables and regions, or not?

> [brainstorm] snapshots: hardlink alternatives
> ---------------------------------------------
>                 Key: HBASE-6233
>                 URL: https://issues.apache.org/jira/browse/HBASE-6233
>             Project: HBase
>          Issue Type: Brainstorming
>            Reporter: Matteo Bertozzi
>            Assignee: Matteo Bertozzi
>         Attachments: Restore-Snapshot-Hardlink-alternatives.pdf
> Discussion ticket around snapshots and hardlink alternatives.
> (See the HDFS-3370 discussion about hardlink and implementation problems)
> (taking for a moment WAL out of the discussion and focusing on hfiles)
> With hardlinks available taking snapshot will be fairly easy:
> * (hfiles are immutable)
> * hardlink to .snapshot/name to take snapshot
> * hardlink from .snapshot/name to restore the snapshot
> * No code change needed (on fs.delete() only one reference is deleted)
> but we don't have hardlinks, what are the alternatives?

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message