incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josep Blanquer <blanq...@rightscale.com>
Subject Re: Backup/Restore: Coordinating Cassandra Nodetool Snapshots with Amazon EBS Snapshots?
Date Thu, 23 Jun 2011 15:30:26 GMT
On Thu, Jun 23, 2011 at 7:30 AM, Peter Schuller <peter.schuller@infidyne.com
> wrote:

> > EBS volume atomicity is good. We've had tons of experience since EBS came
> > out almost 4 years ago,  to back all kinds of things, including large
> DBs.
> > One important thing to have in mind though, is that EBS snapshots are
> done
> > at the block level, not at the filesystem level. So depending on the
> > filesystem you have on top of the drives you might need to tell the
> > filesystem to "make sure this is consistent or recoverable now". For
> > example, if you use the log-based XFS, you might need to do xfs_freeze,
> > snapshot disc/s, xfs_unfreeze. To make sure that the restored filesystem
> > data (and not only the low level disk blocks) is recoverable when you
> > restore them.
>
> No. That is only require if you're doing multi-volume EBS snapshots
> (e.g. XFS on LVM). The entire point of an atomic snapshot is that
> atomicity gives a consistent snapshot; a modern file system which is
> already crash-consistent will be consistent in an atomic snapshot
> without additional action taken.
>
>

> That said, of course exercising those code paths regularly, rather
> than just on crashes, may mean that you have an elevated chance of
> triggering a bug that you would normally see very rarely. In that way,
> xfs_freeze might actually help probabilistically; however strictly
> speaking, discounting bugs, a crash-consistent fs will be "consistent
> snapshot consistent" as well (it is logically implied).
>
>
Actually, I'm afraid that's not true (unless I'm missing something). Even if
you have only 1 drive, you still need to stop writes to the disk for the
short time it takes the low level "drivers" to snapshot it (i.e., marking
all blocks as clean so you can do CopyOnWrite later). I.e., you need to give
a chance to LVM, or the EBS low level 'modules' in the hypervisor ( whatever
you use underneath...), to have exclusive control of the drive for a moment.
Now, that being said, some systems (like LVM)  will do a freeze themselves,
so technically speaking you don't need to explicitly do a freeze
yourself...but that's not to say that a freeze is not required for
snapshotting.

But this all assumes the entire stack is correct, and that e.g. an
> fsync() propagates correctly (i.e., not eaten by some LVM or mount
> option to the fs) in order to bring that consistency up to the
> application level.
>
> --
> / Peter Schuller
>

Mime
View raw message