incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dave Cottlehuber <d...@muse.net.nz>
Subject Re: couchdb space usage issue
Date Fri, 30 Mar 2012 11:17:07 GMT
On 30 March 2012 11:01, Mehdi El Fadil <mehdi.elfadil@mango-is.com> wrote:
> Hi,
>
> I have left an application populating couchdb with tweets on a dev server
> running for a few days.
>
> At one point there were 1.7 million documents, and the database was using
> approx. 8GB. One day later there were 1.71 million documents, but the DB
> had filled the whole disk (20 GB). This looks strange, only 100 000 new
> docs, using more space than the 1.7 million that were previously there. All
> the documents were created by the same script.
>
> What happened during this day was insertion of documents, one replication
> (before the disk was filled), few get _all_docs queries, and I doubt that
> any view has been refreshed.
>
> There has not been any update running on the docs. Logs show errors related
> to the no space left on device, but nothing before that.
>
> I have copied the 20GB couchdb database file to a larger disk to get able
> to trigger a replication, and the replicated version only uses a bit more
> than 8 GB.
>
> Any explanation of the cause of the huge jump in space usage by the
> database?
>
> couchdb version is 1.1.0 is I remember well.
>
> Thanks.
> --
> *
> Mehdi El Fadil <http://be.linkedin.com/in/elfadme>
> <http://mango-is.com>
> *

Hi Mehdi,

1.2.0 will have support for auto-compaction when released.

Depending on what _id you choose, adding documents may require
re-writing internal b~tree nodes. This can increase space which is
then reclaimed post compaction or replication.

Ideally your ids are ascending to minimise this, mail back a few
and we'll see.

A+
Dave

Mime
View raw message