incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Randall Leeds <randall.le...@gmail.com>
Subject Re: large attachments/huge databases ?`
Date Thu, 13 May 2010 20:10:29 GMT
If you want to be super couch-y and keep it HTTP based and keep couch
at the top of your server stack you could write your own http handler
that just streams files off the disk.

http://mycouch:5984/_files/path_to_file

Look at the other handlers in the couch.ini files and then look at the
corresponding .erl files in src/couchdb to get a feel for how to plug
your own in.

This might help you too:
http://vmx.cx/couchdb/tutorial/indexer.html

-Randall

On Thu, May 13, 2010 at 20:52, Cesar Delgado <beettlle@ymail.com> wrote:
> CK,
>
> My $0.02 on the storage of the videos is not to use CouchDB for that.  Use Couch to
store metadata on the files.  Stuff like filesystem path, server holding the file, date,
time-stamp, video info, etc.  The actual files are better stored in a filesystem somewhere
else.  It's kind of the idea of Facebook's Haystack project (http://www.facebook.com/note.php?note_id=76191543919).
 That's Facebook's system to store photos.
>
> The problem with storing large attachments in any database is you're creating a filesystem
on a filesystem. You ask Couch for a file, Couch has to go the filesystem to get the files,
and then you go back and forth between Couch and the filesystem getting information that then
gets sent back to you.  Removing the middle-man, Couch in this case, should give you better
performance and a smaller DB.  As Couch, "where should I find the file" and the your application
just goes out and gets the file directly.
>
> There used to be a great article explaining why putting large files in a database (the
article was about MySQL) was a bad idea but I can't seen to find it anymore.
>
> Anyway, good luck!
>
> -Cesar
>
>
>
> ________________________________
> From: c.Kleinhuis <ck@digitalgott.de>
> To: user@couchdb.apache.org
> Sent: Thu, May 13, 2010 9:35:24 AM
> Subject: large attachments/huge databases ?`
>
> i need to convience my project manager ;)
>
> -he read that indexing is significantly higher than e.g. mysql -
> my answer was that indexing is not affecting performance because it is a
> one time action ....
>
> -another point is general performance of about e.g. 200.000 documents in a single
> database ... how is disk usage when maintaining versioning of each document ?
> -can the versioning be deactivated or deleted ?!
>
> and finally it is about handling huge movie files ( production 720p files ) which
> must be handled somehow, what happens if an upload fails, or should a proxy like
> php be used to receive those files, and just store a reference in the couchdb ?
>
> thx in advance
> ck

Mime
View raw message