couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Davis <paul.joseph.da...@gmail.com>
Subject Re: View group compactions shutting down in trunk
Date Wed, 14 Sep 2011 04:12:25 GMT
That seems like a reasonable patch but I'm confused why we never had
reports of this before. Is it just that we didn't have people cycling
through db's fast enough that noticed view compaction dying or has
something else changed? I'd consider it unlikely that no one would've
noticed by now, but not unsurprising.

On Wed, Sep 14, 2011 at 3:53 AM, Filipe David Manana
<fdmanana@apache.org> wrote:
> Mike,
>
> Can you try the following patch?
>
> http://friendpaste.com/2fT5Ne4NAQYoS9CWBsOKUW
>
> Basically it keeps the respective database open until view compaction finishes.
>
> Thanks for sharing this
>
> On Tue, Sep 13, 2011 at 2:39 PM, Mike Leddy <mike@loop.com.br> wrote:
>> Hello,
>>
>> I've been experimenting with trunk and the new compaction daemon,
>> and discovered that some of my larger views were never being fully
>> compacted.
>>
>> Basically I am encountering a large number of these situations:
>>
>> [Tue, 13 Sep 2011 13:24:46 GMT] [info] [<0.173.0>] Shutting down view group
server, monitored db is closing.
>> [Tue, 13 Sep 2011 13:24:46 GMT] [error] [<0.155.0>] Compaction daemon - an
error ocurred while compacting the view group `base` from database `XXXX-1301529600-1303948799`:
shutdown
>>
>> I suspect that since I have a large number of other smaller databases
>> (approx 2500) that are constantly being updated there is a constant
>> stream of databases being closed as a result of least recently used
>> recovery - I currently have max_open_dbs at 500.
>>
>> This does not affect database compaction as a database is never
>> considered idle when compacting.
>>
>> Consequently the larger views are shutdown before completion. Also,
>> when a new attempt to re-compact is processed the partially complete
>> *.compact.view file is truncated. As a result these large views
>> never get fully compacted.
>>
>> If the view group compaction continued where it left off I guess
>> the impact would be minimal.
>>
>> Has anyone else seen this ?
>>
>> Regards,
>>
>> Mike
>>
>>
>>
>>
>>
>>
>
>
>
> --
> Filipe David Manana,
>
> "Reasonable men adapt themselves to the world.
>  Unreasonable men adapt the world to themselves.
>  That's why all progress depends on unreasonable men."
>

Mime
View raw message