couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Leddy <m...@loop.com.br>
Subject Re: View group compactions shutting down in trunk
Date Wed, 14 Sep 2011 13:38:46 GMT

Thanks for the patch Filipe, but I am pretty sure that the compaction
daemon is not involved in this.

I removed all item in the 'compactions' config and restarted couchdb.
As ?CONFIG_ETS is empty that compact_loop does nothing and 
maybe_compact_db is never called.

Next I started a view compaction in Futon whilst doing the inserts
that cause the constant lru recovery. As expected the result was the
the same:

"Shutting down view group server, monitored db is closing" in the
log at the same time the view group compaction was shut down.

I had never seen this before as I had no reason to compact these
large views whilst having so many concurrent accesses on the 
smaller databases.

Regards,

Mike

On Tue, 2011-09-13 at 20:53 -0700, Filipe David Manana wrote:
> Mike,
> 
> Can you try the following patch?
> 
> http://friendpaste.com/2fT5Ne4NAQYoS9CWBsOKUW
> 
> Basically it keeps the respective database open until view compaction finishes.
> 
> Thanks for sharing this
> 
> On Tue, Sep 13, 2011 at 2:39 PM, Mike Leddy <mike@loop.com.br> wrote:
> > Hello,
> >
> > I've been experimenting with trunk and the new compaction daemon,
> > and discovered that some of my larger views were never being fully
> > compacted.
> >
> > Basically I am encountering a large number of these situations:
> >
> > [Tue, 13 Sep 2011 13:24:46 GMT] [info] [<0.173.0>] Shutting down view group
server, monitored db is closing.
> > [Tue, 13 Sep 2011 13:24:46 GMT] [error] [<0.155.0>] Compaction daemon - an
error ocurred while compacting the view group `base` from database `XXXX-1301529600-1303948799`:
shutdown
> >
> > I suspect that since I have a large number of other smaller databases
> > (approx 2500) that are constantly being updated there is a constant
> > stream of databases being closed as a result of least recently used
> > recovery - I currently have max_open_dbs at 500.
> >
> > This does not affect database compaction as a database is never
> > considered idle when compacting.
> >
> > Consequently the larger views are shutdown before completion. Also,
> > when a new attempt to re-compact is processed the partially complete
> > *.compact.view file is truncated. As a result these large views
> > never get fully compacted.
> >
> > If the view group compaction continued where it left off I guess
> > the impact would be minimal.
> >
> > Has anyone else seen this ?
> >
> > Regards,
> >
> > Mike
> >
> >
> >
> >
> >
> >
> 
> 
> 



Mime
View raw message