couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "c.Kleinhuis">
Subject Re: large attachments/huge databases ?`
Date Fri, 14 May 2010 09:09:36 GMT
hi, thank you for your description, and the project is only storing 
metadata :D
the files will be updated via ftp syncronize :D

i will post another question right now ...
> CK,
> My $0.02 on the storage of the videos is not to use CouchDB for that.  Use Couch to store
metadata on the files.  Stuff like filesystem path, server holding the file, date, time-stamp,
video info, etc.  The actual files are better stored in a filesystem somewhere else.  It's
kind of the idea of Facebook's Haystack project (
 That's Facebook's system to store photos.
> The problem with storing large attachments in any database is you're creating a filesystem
on a filesystem. You ask Couch for a file, Couch has to go the filesystem to get the files,
and then you go back and forth between Couch and the filesystem getting information that then
gets sent back to you.  Removing the middle-man, Couch in this case, should give you better
performance and a smaller DB.  As Couch, "where should I find the file" and the your application
just goes out and gets the file directly.
> There used to be a great article explaining why putting large files in a database (the
article was about MySQL) was a bad idea but I can't seen to find it anymore.
> Anyway, good luck!
> -Cesar  
> ________________________________
> From: c.Kleinhuis <>
> To:
> Sent: Thu, May 13, 2010 9:35:24 AM
> Subject: large attachments/huge databases ?`
> i need to convience my project manager ;)
> -he read that indexing is significantly higher than e.g. mysql -
> my answer was that indexing is not affecting performance because it is a
> one time action ....
> -another point is general performance of about e.g. 200.000 documents in a single
> database ... how is disk usage when maintaining versioning of each document ?
> -can the versioning be deactivated or deleted ?!
> and finally it is about handling huge movie files ( production 720p files ) which
> must be handled somehow, what happens if an upload fails, or should a proxy like
> php be used to receive those files, and just store a reference in the couchdb ?
> thx in advance
> ck

View raw message