couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ethan <>
Subject Re: huge attachments - experience?
Date Thu, 28 Mar 2013 14:34:13 GMT
On Thu, Mar 28, 2013 at 7:23 AM, svilen <> wrote:

> yeah, checked all those. They are post-factum, file synchronizers - can
> not manage directories (rename a dir => copy all the dir again).
> The only thing that manages directories is bazaar but it chokes after
> 8-9Gb of data (out of memory). Anyway seems i need to separate
> filesystem metadata (filenames, directories etc) from actual raw
> datafiles. e.g. fancy hierarchy of hardlinks, pointing to flat-dir
> files. And apply separate syncs to those two. But should think it out
> first.

Hi, you might look at git-annex. I use it to store large media files.
Basically, it keeps a directory tree of symlinks checked in to git, but the
targets of those symlinks get copied around on your say so. It makes it
really easy to answer "which hard drive is this file on" as well as letting
you rename files even if the files aren't themselves there.


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message