couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Miller <alexmil...@apple.com.INVALID>
Subject Re: [DISCUSS] soft-deletion
Date Wed, 18 Mar 2020 23:39:18 GMT
> On Mar 18, 2020, at 14:03, Joan Touzet <wohali@apache.org> wrote:
>> And this would all rely on having a continuous backup configured and running that
would hold a minimum of 48 hours of changes.
> 
> People aren't in the habit of doing this in the field. I don't know if we'd be able to
impress upon people the proper hygiene for this, especially if their current backup solution
is "I zip/tar up the /opt/couchdb/data directory daily."
> 
> If CouchDB somehow shipped out of the box with the FDB continuous backup configured and
running as you say, the system load for this was minimal, and the storage required not extreme...we
might be able to pass it off.

I think there’s an argument that backup to (local) file could be made automated and
managed as part of the CouchDB installation process, but it definitely would introduce
much more complexity to the install.  I was honestly expecting a bit more of an objection
to the idea from Joan “CouchDB should be simple and friendly for small deployments” Touzet.
;)


> On Mar 18, 2020, at 14:06, Paul Davis <paul.joseph.davis@gmail.com> wrote:
> 
> Alex,
> 
> The first con I see for that approach is that its not soft-deletion.
> Its actual deletion with an API for restoration. Which, fair enough,
> is probably a feature we should consider supporting for CouchDB
> installations that are based on FoundationDB.
> 
> The second major con is that it relies on CouchDB being based on
> FoundationDB. Part of CouchDB's design philosophy is that the internet
> may or may not exist, and if it does exist that it may or may not be
> reliable. There are lots of deployments of CouchDB that are part of a
> desktop application or POS installation that may see internet only
> periodically if at all so an S3 backup solution is out.
> 
> Then the last major con I see is the time-to-restore disparity. With
> soft-deletion restoration is a few milliseconds. Streaming from S3
> will obviously depend on the size of the database and obviously be
> orders of magnitude longer.
> 
> On the pro side for the soft-delete on FoundationDB is that the first
> draft of the RFC is 108 lines [1]. We obviously can't say for sure how
> big or involved the fdbrestore approach would be but I think we'd all
> agree it'd be bigger.
> 
> Paul
> 
> [1] https://github.com/apache/couchdb/pull/2666


Thanks for the explanation.  Time-to-restore disparity would be significant,
although I’m not sure that I follow that there’s really a semantic difference
to a user between soft-delete and an instantaneous restore.  Backing up to
local files could also theoretically work to avoid an internet dependency, but
you’d be getting operational complexity for not a tremendous gain, and there’s
probably other complications I’m not thinking of.  I appreciate the CouchDB
design philosophy education.

And oh wow, you meant the code is 108 lines and not the document.
That's even smaller than I was expecting. Y’all keep writing surprisingly
succinct erlang. :O


> There also may come a time that there's a flavor of CouchDB that uses
> LevelDB or SQLite or FDBLite (I just made that up, any idea how hard it'd be?)
> for these sorts of embedded deployments where fdbrestore/fdbbackup
> wouldn't be feasible.

Yes!  I have snipped away this thought, as I’ve actually been waiting for
someone from CouchDB to come asking how hard would it be to make
libfdb_c.so-but-actually-it’s-just-rocksdb/lmdb/sqlite.  This is a project that I
would love to discuss in far more depth, but probably something we should
do outside the scope of a soft deletion RFC.  I do think it’d be overall feasible
and perhaps be a nice option for embedded/mobile deployments of CouchDB. 


Thanks,
Alex
Mime
View raw message