incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Newson <rnew...@apache.org>
Subject Re: Database too large for partition
Date Thu, 11 Jul 2013 19:23:54 GMT
Yes, you can delete the view files then compact the database, just
don't query the views otherwise they'll be rebuilt. Were you also
compacting your views? It's a separate operation, one per design
document, and sometimes folks don't realize that.

The real answer here is to shard this huge database. BigCouch is the
current option and we're in the middle of integrating that into a
future CouchDB release.

B.


On 11 July 2013 20:20, Matthias Eck <matthias.eck@gmail.com> wrote:
> I did occasional compaction, but not very frequently.
> The database file itself has about 500GB, the views have another 500GB.
> If I delete the views can I then try compaction without generating the
> views at first?
>
> Matthias
>
>
> On Thu, Jul 11, 2013 at 12:14 PM, Robert Newson <rnewson@apache.org> wrote:
>
>> Hit send to soon. I meant the 'data_size' field of course.
>>
>> b.
>>
>>
>> On 11 July 2013 20:13, Robert Newson <rnewson@apache.org> wrote:
>> > " but you need space equal to the content of the database to do a
>> compaction."
>> >
>> > That's not true. You need space equal to the "disk_size" value from
>> > GET /dbname. The only time your statement would be true is if you
>> > compacted a database that you had just compacted and had made no
>> > changes to. Compacting it again will just write the whole thing out
>> > again. Obviously this worse case is also the case where there's no
>> > point compacting anyway.
>> >
>> > B.
>> >
>> >
>> >
>> >
>> > On 11 July 2013 20:00, Tim Tisdall <tisdall@gmail.com> wrote:
>> >> You could delete the generated views in /var/lib/couchdb/.your_database/
>> >> and recover a little extra space.  There could also be a partially done
>> >> compaction in /var/lib/couchdb/
>> >>
>> >> The longer term solution is to do compactions... but you need space
>> equal
>> >> to the content of the database to do a compaction.
>> >>
>> >>
>> >> On Thu, Jul 11, 2013 at 2:56 PM, Matthias Eck <matthias.eck@gmail.com
>> >wrote:
>> >>
>> >>> Hello,
>> >>>
>> >>> I have a CouchDB database that is on a 1TB partition on Amazon EC2.
>> >>> Unfortunately the database now filled the whole partition and 1TB is
>> the
>> >>> size limit on Amazon EC2, so I cannot copy it to a larger partition.
>> >>>
>> >>> 1. I would like to get the database running again. Are there any files
>> that
>> >>> I can safely delete to save some immediate space so CouchDB will run
>> again?
>> >>>
>> >>> 2. Any suggestions to solve this problem longer-term? Is it possible
to
>> >>> have a database spanning multiple partitions?
>> >>>
>> >>> Thanks,
>> >>> Matthias
>> >>>
>>

Mime
View raw message