incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jens Alfke <j...@couchbase.com>
Subject Re: Database too large for partition
Date Fri, 12 Jul 2013 19:03:15 GMT

On Jul 11, 2013, at 12:20 PM, Matthias Eck <matthias.eck@gmail.com> wrote:

> The database file itself has about 500GB, the views have another 500GB.

Is the database 500GB of pure JSON, or are there attachments? (The idea of 500GB of JSON boggles
my mind, but then, I’m not a big-data guy.)

I ask because if a lot of that size is attachments, you can save space by storing those elsewhere,
like in S3, and leaving just URLs or other IDs in the docs. This will also speed up compaction
because the attachment data doesn’t have to be copied. The drawback is that you’ll have
to manually delete attachments after their corresponding doc[s] are deleted or updated.

—Jens
Mime
View raw message