incubator-couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Filipe Manana (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (COUCHDB-1187) Server terminated if timeout reached during reopen_db after compaction
Date Mon, 06 Jun 2011 12:23:59 GMT

    [ https://issues.apache.org/jira/browse/COUCHDB-1187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044813#comment-13044813
] 

Filipe Manana commented on COUCHDB-1187:
----------------------------------------

Hi Alexey,

I've observed that behaviour as well in the past several times with large databases. I commited
a simple fix to trunk a while ago:

https://github.com/apache/couchdb/commit/d928e012035c16927772a8cff2802a34dd0d6535

I've verified that so far it fixes those cases. Do you have a chance to apply this patch to
whatever release/branch you use and confirm it works for your case?

thanks

> Server terminated if timeout reached during reopen_db after compaction
> ----------------------------------------------------------------------
>
>                 Key: COUCHDB-1187
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-1187
>             Project: CouchDB
>          Issue Type: Bug
>          Components: Database Core
>    Affects Versions: 1.0.2
>         Environment: Gentoo Linux
> Erlang 14.2.1
>            Reporter: Alexey Loshkarev
>              Labels: compaction
>
> I have installed couchdb database onto ReiserFS partition. Before databases becomes large,
all was fine.
> But now, I have some databases with very much revisions in it (but very few documents).
So, it grows from 10M to 1G in couple of days. Next, i compact it and it become 10M again.
> But, if i have forgot to compact it in a few days, it becomes 10GB+. Deleting such big
files is very time-consumption on reiserfs (i know, reiser4, ext3/4 or xfs has no such problem,
but reiserfs3 has). It may take up to couple of minutes to finish deletion.
> Now, after compaction, database terminated, due timeout on reopen_db action.
> Here are log:
> ========================
> [Sun, 05 Jun 2011 14:47:16 GMT] [info] [<0.128.0>] Starting compaction for db "exhaust"
> [Sun, 05 Jun 2011 14:47:16 GMT] [info] [<0.2451.0>] 10.20.21.1 - - 'POST' /exhaust/_compact
202
> [Sun, 05 Jun 2011 14:47:21 GMT] [info] [<0.2452.0>] 10.20.21.1 - - 'GET' /_active_tasks
200
> [Sun, 05 Jun 2011 14:47:21 GMT] [info] [<0.2463.0>] 10.20.21.1 - - 'GET' / 200
> [Sun, 05 Jun 2011 14:47:24 GMT] [info] [<0.128.0>] Compaction for db "exhaust"
completed.
> [Sun, 05 Jun 2011 14:47:24 GMT] [info] [<0.2464.0>] 127.0.0.1 - - 'GET' / 200
> [Sun, 05 Jun 2011 14:47:26 GMT] [info] [<0.2473.0>] 10.20.21.1 - - 'GET' /_active_tasks
200
> [Sun, 05 Jun 2011 14:47:26 GMT] [info] [<0.2476.0>] 10.20.21.1 - - 'GET' / 200
> [Sun, 05 Jun 2011 14:47:29 GMT] [error] [<0.507.0>] ** Generic server <0.507.0>
terminating 
> ** Last message in was {'$gen_cast',reopen_db}
> ** When Server state == {group_state,undefined,<<"exhaust">>,
>                          {"/var/lib/couchdb",<<"exhaust">>,
>                           {group,
>                            <<26,211,78,5,6,56,56,6,195,233,132,7,12,137,41,86>>,
>                            nil,nil,<<"_design/conflicts">>,<<"javascript">>,
>                            [],
>                            [{view,0,
>                              [<<"list">>],
>                              <<"function(doc) {\n    if(doc._conflicts) {\n   
    emit(doc._conflicts, null);\n    }\n}\n">>,
>                              nil,[],[]}],
>                            nil,0,0,nil,nil}},
>                          {group,
>                           <<26,211,78,5,6,56,56,6,195,233,132,7,12,137,41,86>>,
>                           {db,<0.125.0>,<0.128.0>,nil,<<"1307284558924035">>,
>                            <0.118.0>,<0.137.0>,
>                            {db_header,5,407871,0,
>                             {14473890608,{670,14}},
>                             {14473892185,684},
>                             {14473796209,[]},
>                             0,nil,nil,1000},
>                            407871,
>                            {btree,<0.118.0>,
>                             {14473890608,{670,14}},
>                             #Fun<couch_db_updater.7.10053969>,
>                             #Fun<couch_db_updater.8.35220795>,
>                             #Fun<couch_btree.5.124754102>,
>                             #Fun<couch_db_updater.9.107593676>},
>                            {btree,<0.118.0>,
>                             {14473892185,684},
>                             #Fun<couch_db_updater.10.30996817>,
>                             #Fun<couch_db_updater.11.96515267>,
>                             #Fun<couch_btree.5.124754102>,
>                             #Fun<couch_db_updater.12.117826253>},
>                            {btree,<0.118.0>,
>                             {14473796209,[]},
>                             #Fun<couch_btree.0.83553141>,
>                             #Fun<couch_btree.1.30790806>,
>                             #Fun<couch_btree.2.124754102>,nil},
>                            407871,<<"exhaust">>,
>                            "/var/lib/couchdb/exhaust.couch",[],[],nil,
>                            {user_ctx,null,[],undefined},
>                            nil,1000,
>                            [before_header,after_header,on_file_open],
>                            false},
>                           <0.2256.0>,<<"_design/conflicts">>,<<"javascript">>,
>                           [],
>                           [{view,0,
>                             [<<"list">>],
>                             <<"function(doc) {\n    if(doc._conflicts) {\n    
   emit(doc._conflicts, null);\n    }\n}\n">>,
>                             {btree,<0.2256.0>,
>                              {157,{1,[]}},
>                              #Fun<couch_btree.3.83553141>,
>                              #Fun<couch_btree.4.30790806>,
>                              #Fun<couch_view.less_json_ids.2>,
>                              #Fun<couch_view_group.10.132983779>},
>                             [],[]}],
>                           {btree,<0.2256.0>,
>                            {51,[]},
>                            #Fun<couch_btree.0.83553141>,
>                            #Fun<couch_btree.1.30790806>,
>                            #Fun<couch_btree.2.124754102>,nil},
>                           407871,0,nil,nil},
>                          nil,nil,false,[],<0.2261.0>,<0.511.0>}
> ** Reason for termination == 
> ** {timeout,{gen_server,call,[<0.137.0>,{drop,<0.507.0>}]}}
> [Sun, 05 Jun 2011 14:47:29 GMT] [error] [<0.507.0>] {error_report,<0.31.0>,
>     {<0.507.0>,crash_report,
>      [[{initial_call,{couch_view_group,init,['Argument__1']}},
>        {pid,<0.507.0>},
>        {registered_name,[]},
>        {error_info,
>            {exit,
>                {timeout,{gen_server,call,[<0.137.0>,{drop,<0.507.0>}]}},
>                [{gen_server,terminate,6},{proc_lib,init_p_do_apply,3}]}},
>        {ancestors,
>            [couch_view,couch_secondary_services,couch_server_sup,<0.32.0>]},
>        {messages,[]},
>        {links,[<0.511.0>,<0.2256.0>,<0.101.0>]},
>        {dictionary,[]},
>        {trap_exit,true},
>        {status,running},
>        {heap_size,987},
>        {stack_size,24},
>        {reductions,1902}],
>       [{neighbour,
>            [{pid,<0.511.0>},
>             {registered_name,[]},
>             {initial_call,{couch_event_sup,init,['Argument__1']}},
>             {current_function,{gen_server,loop,6}},
>             {ancestors,
>                 [<0.507.0>,couch_view,couch_secondary_services,
>                  couch_server_sup,<0.32.0>]},
>             {messages,[{'$gen_cast',stop}]},
>             {links,[<0.507.0>,<0.85.0>]},
>             {dictionary,[]},
>             {trap_exit,false},
>             {status,runnable},
>             {heap_size,233},
>             {stack_size,9},
>             {reductions,32}]}]]}}
> [Sun, 05 Jun 2011 14:47:35 GMT] [error] [<0.2256.0>] ** Generic server <0.2256.0>
terminating 
> ** Last message in was {'EXIT',<0.507.0>,
>                            {timeout,
>                                {gen_server,call,
>                                    [<0.137.0>,{drop,<0.507.0>}]}}}
> ** When Server state == {file,{file_descriptor,prim_file,{#Port<0.3297>,22}},
>                               0,4185}
> ** Reason for termination == 
> ** {timeout,{gen_server,call,[<0.137.0>,{drop,<0.507.0>}]}}
> [Sun, 05 Jun 2011 14:47:35 GMT] [error] [<0.2256.0>] {error_report,<0.31.0>,
>     {<0.2256.0>,crash_report,
>      [[{initial_call,{couch_file,init,['Argument__1']}},
>        {pid,<0.2256.0>},
>        {registered_name,[]},
>        {error_info,
>            {exit,
>                {timeout,{gen_server,call,[<0.137.0>,{drop,<0.507.0>}]}},
>                [{gen_server,terminate,6},{proc_lib,init_p_do_apply,3}]}},
>        {ancestors,
>            [<0.507.0>,couch_view,couch_secondary_services,couch_server_sup,
>             <0.32.0>]},
>        {messages,[]},
>        {links,[#Port<0.3297>,<0.2261.0>]},
>        {dictionary,[]},
>        {trap_exit,true},
>        {status,running},
>        {heap_size,377},
>        {stack_size,24},
>        {reductions,2223}],
>       [{neighbour,
>            [{pid,<0.2261.0>},
>             {registered_name,[]},
>             {initial_call,{couch_ref_counter,init,['Argument__1']}},
>             {current_function,{gen_server,loop,6}},
>             {ancestors,
>                 [<0.507.0>,couch_view,couch_secondary_services,
>                  couch_server_sup,<0.32.0>]},
>             {messages,
>                 [{'DOWN',#Ref<0.0.0.44077>,process,<0.507.0>,
>                      {timeout,
>                          {gen_server,call,[<0.137.0>,{drop,<0.507.0>}]}}}]},
>             {links,[<0.2256.0>]},
>             {dictionary,[]},
>             {trap_exit,false},
>             {status,runnable},
>             {heap_size,377},
>             {stack_size,9},
>             {reductions,114}]}]]}}
> ========================

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message