couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Smith <jason.h.sm...@gmail.com>
Subject Re: Using CouchDB as a file system server side
Date Wed, 22 Jun 2016 08:27:20 GMT
And a quick answer to the OP:

Right, if you plan to store 20TB of attachments, and *especially* if you
will replicate entire databases around, then I expect that CouchDB
attachments will be inappropriate.

Of course, now you must take responsibility for replicating 20TB around
your site, potentially tracking revisions and deletions. That will add
complexity to your project. But if you are undertaking a distributed
filesystem, I imagine that you planned for a bit of complexity :)

On Wed, Jun 22, 2016 at 3:20 PM, Jason Smith <jason.h.smith@gmail.com>
wrote:

> On Wed, Jun 22, 2016 at 3:29 AM, Alexander Harm <contact@aharm.de> wrote:
>
>> I especially love the quote of Laurie Voss:
>>
>> "One of the big things that everybody who's spent a lot of time with
>> databases knows is that you should never put your binaries in the database.
>> It's a terrible idea. It always goes wrong. I have never met a database in
>> 15 years of which it is not true, and it's definitely not true of CouchDB.
>> You are taking this thing which is meant to sort and organize data, and
>> you're giving it binary data, which it can neither sort nor organize. It
>> can't do anything with that data, other than get really fat.”
>>
>
> This is a pet peeve of mine.
>
> "Do not store files in the database."
>
> That simplistic mantra is up there with "[about regular expressions], now
> you have two problems." It is so simple, so naïve. It can be over-done, or
> under-analyzed.
>
> Some thoughts.
>
> 1. All large systems experience growing pains from off-the-shelf tools.
> All large projects must become custom. Twitter's growth is not an
> indictment of Ruby on Rails. Google's growth is not an indictment of the
> ext filesystem.
>
> 2. Attachments in CouchDB are very simple. Simple is easy to ship. Simple
> is easy to maintain. Simple projects allow you to focus on other problems.
>
> 3. Attachments in CouchDB make less sense as a project matures. Yes, I
> said it. But just be careful not to prematurely optimize.
>
> Once you really grow, you will be investigating CDNs, and caching tools,
> and all sorts of alternatives. But how did you become that successful? How
> did you come to this "good problem"? Because you shipped, and you
> delivered, and you scaled. And at long last, you have earned the privilege
> of building something large and complex.
>
> Laurie Voss joined npm and made that quote some years after npm came
> online. It had been growing exponentially for a few years. I remember
> clearly: npm was already processing 1,000 requests per second, all on
> plain-Jane CouchDB, with a very simple design.
>
> "You didn't build that," as the man says. In my opinion, attachments
> played a part in npm's early success, and moving away from them played a
> part in npm's latter success. Both were wise decisions.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message