couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Van Couvering <david.vancouver...@gmail.com>
Subject Re: Time machine backup while CouchDB running?
Date Fri, 29 Jan 2010 06:57:06 GMT
Like, yeah, I get it, thanks!  I love all the spiral notebook slides :)

I knew it was append only, but I missed the fact that it's always
consistent, and lock-free.  Having worked on lots of database and with db
teams, and all the issues around concurrency, consistency, deadlocks, etc.,
this is a Big Deal.

Great work, Damien, et. al.!

David

On Thu, Jan 28, 2010 at 10:40 PM, Kevin Ferguson <kevin@meebo-inc.com>wrote:

> David, check out this preso-- it should explain how the CouchDB storage
> engine works in a bit more detail.
>
> http://jchrisa.net/drl/_design/sofa/_show/post/How-CouchDB-Treats-the-Disks
>
> Kevin
>
> On Thu, Jan 28, 2010 at 10:37 PM, David Van Couvering <
> david.vancouvering@gmail.com> wrote:
>
> > Thinking further on this: I think one difference is with Derby each table
> > is
> > its own file.  Since you need to maintain relational consistency in a SQL
> > DB, this is a problem if you're copying lots of table files while
> > transactions are in flight.  Sybase could be a single file (or raw
> device),
> > or it could be split across multiple devices for better throughput, so
> > again
> > you have potential consistency problems (plus I think you tended to back
> up
> > through a special API, as standard file copy doesn't work on raw
> devices).
> >
> > I guess in CouchDB since it's a single file, *perhaps* you don't have the
> > same issues around maintaining consistency...
> >
> > On Thu, Jan 28, 2010 at 10:33 PM, David Van Couvering <
> > david.vancouvering@gmail.com> wrote:
> >
> > > Well, that's true of any database, but for most databases if you try to
> > > take a "snapshot" of a db file while it's running, chances are pretty
> > good
> > > you're not going to like what you get.  More often than not a
> transaction
> > > will have been half-written to the file when you "grab" it and back it
> > up.
> > >  And as we all know, the only thing worse than finding a worm in an
> apple
> > is
> > > finding half a worm...
> > >
> > > That's why "online backup" is a key feature requirement for any
> database
> > > worth it's salt.  We had to implement it for Sybase, and we had to
> > implement
> > > it for Apache Derby.
> > >
> > > What I'm hearing here is because the way CouchDB works (I'm not sure I
> > > fully grok it), at any point you take a copy of the db file, it's in a
> > > consistent state.  I guess what this means is each document is written
> as
> > a
> > > single atomic write, so you can't end up with half a document in your
> > > backup.
> > >
> > > I know you've told me this works, but call me paranoid - is that really
> > > true?  What if the document is umpteen gajillibytes long?  Is it still
> > > written as a *single atomic write* to the disk?  Do you all lock the
> file
> > > each time you do a write?  Does Time Machine lock the file from writes
> > the
> > > whole time it's reading it/taking a snapshot of it?  Just want to make
> > > really sure we're all on the same page here.
> > >
> > > Thanks,
> > >
> > > David
> > >
> > >
> > > On Thu, Jan 28, 2010 at 9:51 PM, de Saint Martin C├ędric <
> > > cedric@desaintmartin.fr> wrote:
> > >
> > >>        Tell me if I'm wrong, but CouchDB's databases are stored in
> > files.
> > >> So if you backup all these files, there is no risk of corrupting your
> > data.
> > >>        Conclusion : TimeMachine will work.
> > >>
> > >>
> > >> On 29 janv. 2010, at 00:41, David Van Couvering wrote:
> > >>
> > >> > Anyone know if a TimeMachine backup of a running CouchDB will work,
> or
> > >> is
> > >> > there a likelihood of corruption?
> > >> >
> > >> > --
> > >> > David W. Van Couvering
> > >> >
> > >> > http://www.linkedin.com/in/davidvc
> > >> > http://davidvancouvering.blogspot.com
> > >> > http://twitter.com/dcouvering
> > >>
> > >>
> > >>
> > >
> > >
> > > --
> > > David W. Van Couvering
> > >
> > > http://www.linkedin.com/in/davidvc
> > > http://davidvancouvering.blogspot.com
> > > http://twitter.com/dcouvering
> > >
> >
> >
> >
> > --
> > David W. Van Couvering
> >
> > http://www.linkedin.com/in/davidvc
> > http://davidvancouvering.blogspot.com
> > http://twitter.com/dcouvering
> >
>



-- 
David W. Van Couvering

http://www.linkedin.com/in/davidvc
http://davidvancouvering.blogspot.com
http://twitter.com/dcouvering

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message