incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benoit Chesneau <bchesn...@gmail.com>
Subject Re: couchdb attachments vs storing files on fs or s3 like service
Date Mon, 27 Apr 2009 17:08:37 GMT
2009/4/27 Paul Davis <paul.joseph.davis@gmail.com>:
> Benoit,
>
> I think that it'd be hard to tell a priori what the better of the two
> schemes would be. I think the big trade off is that CouchDB doesn't have
> access to sendfile, vs your webserver doesn't have the ease of replicating
> around attachments with your records.
>
> In other words, if this is only ever going to be served from a single host,
> it could be faster to not use files in the main database, but if you ever
> envision using any sharding or distributedness then it may end up being
> faster in the long run to be able to replicate all your data without
> thinking about syncing the external filesystem.
>
> In the end, I think your best bet would be to measure twice and cut once
> though.
>
> HTH,
> Paul Davis
>
Paul,

My first intention was allowing each user to replicate results, edit
on their own, then replicate back to the main server. So attachements
seem the perfect fit. But I'm a little worry about the size it could
take on the disk (and somehow the cost behind) . The problem is not
upload on the main server where I could control file size, but during
replication and compaction.

I see some "solutions" :
-  limit attachment size on main server. But  will it stop replication
if one doc has an attachment with bigger size?
- one node / user with user quotas on main server storage
- a proxy that limit traffich and do check on db size

Other solution (maybe the best) would be indeed to have a way on
couchdb side to save attachements as s3 bucket. Is this something
possible today without making a lot of change in CouchDB core ?

Other problem is of course compaction, since it will require some
storage to be done, but that's something manageable.

Anyway I will continue to investigate on this stuff, thanks for your answer :)

- benoƮt

Mime
View raw message