couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jaakko Sipari (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (COUCHDB-994) Crash after compacting large views
Date Mon, 16 May 2011 08:20:47 GMT

    [ https://issues.apache.org/jira/browse/COUCHDB-994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033913#comment-13033913
] 

Jaakko Sipari commented on COUCHDB-994:
---------------------------------------

We are seeing the same problem with CouchDB release 1.0.2 on top of Erlang R14B and 64bit
RHEL 5.x. This is how it happened:

1) We installed a new version of a design doc, which of course forces a full view index update.
This took 30h to complete.

2) Since the brand new view was 200GB (should be 1/4 of that or less), a couple of days later
we triggered view compaction. This also took 30h to complete. It might also be worth mentioning
here that we have a constant flow of updates/inserts to the database during the compaction,
but it's only something like 1000 updates/inserts per *minute*.

3) Right after the view compaction was complete, couchdb restarted itself and started the
full view index update from scratch.

To me our logs from the incident look very similar to the ones posted by Bob Clary. Although
I cannot post you the full logs from our client's production system, find below the beginning
of the two consecutive (but clearly separate) log messages:

First:

[Tue, 10 May 2011 03:50:28 GMT] [error] [<0.153.0>] ** Generic server <0.153.0>
terminating 
** Last message in was {'$gen_cast',
                        {compact_done,
                         {group,
                          <<66,241,166,10,26,2,78,192,180,77,204,69,249,98,70,
                            23>>,


Second:

[Tue, 10 May 2011 03:50:30 GMT] [error] [<0.153.0>] {error_report,<0.31.0>,
    {<0.153.0>,crash_report,
     [[{initial_call,{couch_view_group,init,['Argument__1']}},
       {pid,<0.153.0>},
       {registered_name,[]},
       {error_info,
           {exit,
               {timeout,{gen_server,call,[<0.156.0>,{drop,<0.153.0>}]}},
               [{gen_server,terminate,6},{proc_lib,init_p_do_apply,3}]}},
       {ancestors,
           [couch_view,couch_secondary_services,couch_server_sup,<0.32.0>]},
       {messages,
           [{'EXIT',<0.32379.11>,


> Crash after compacting large views
> ----------------------------------
>
>                 Key: COUCHDB-994
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-994
>             Project: CouchDB
>          Issue Type: Bug
>    Affects Versions: 1.0.2
>         Environment: Centos5 64bit vm with 2CPU and 4G RAM running Erlang R14B and configured
to use the 64bit js-devel libraries.
> URL: http://svn.apache.org/repos/asf/couchdb/branches/1.0.x
> Repository Root: http://svn.apache.org/repos/asf
> Repository UUID: 13f79535-47bb-0310-9956-ffa450edef68
> Revision: 1050680
>            Reporter: Bob Clary
>             Fix For: 1.2
>
>         Attachments: couch_errors.txt, couch_errors_2.txt
>
>
> The database has over 9 million records. Several of the views are relatively dense in
that they emit a key for most documents. The views are successfully created initially but
with relatively large sizes from 20 to 95G. When attempting to compact them, the server will
crash upon completion of the compaction.
> This does not occur with the released 1.0.1 version but does with the 1.0.x svn version.
I'll attach example logs. Unfortunately they are level error and may not have enough information.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message