I've seen this message before when using mnesia, the default for ets tables is 1400. You might
try setting the environment variable:
ERL_MAX_ETS_TABLES
to something higher
On Jul 15, 2010, at 7:18 PM, Chris Stockton wrote:
> Hello,
>
> I have gone through a good amount of research and effort trying to get
> a large amount of databases replicating, with a good amount of help
> from this list. I have hit a road block and am unable to get more then
> 2000+ databases replicating.
>
> Something tells me [1] might be part of my issue. I have downloaded
> and installed couchdb-0.11.1, I made sure that patch was present in
> the source. I included the limits in the INI file but they do not seem
> to be respected, or I am hitting a different kind of limit. Here [2]
> is the errors that couch starts throwing when I get to roughly 2048~
> (the default number of allowed connections).
>
> These exist in my couchdb init script:
> ulimit -n 1000000
> export ERL_MAX_PORTS=100000
> export ERL_MAX_ETS_TABLES=10000
>
> This is the init script start command:
> daemon --user couchdb "$exec -b -p
> /<dir>/couchdb-0.11.1-test/couch.pid -a
> /<dir>/couchdb-0.11.1-test/local.ini +P 65536"
>
> I am certain the following local.ini is being parsed, as I run on a
> non-standard port, in addition I am certain the path is correct:
> [couchdb]
> os_process_timeout=60000
> max_dbs_open = 100000
>
> [httpd]
> max_connections = 9000
> port = 59840
> bind_address = 0.0.0.0
>
> At this point I am looking for some expertise from other people who
> have dealt with large-scale replication.
>
> Kind Regards,
>
> -Chris
>
> [1] https://issues.apache.org/jira/browse/COUCHDB-705
> [2] ---
>
> [Thu, 15 Jul 2010 23:03:53 GMT] [error] [emulator] ** Too many db tables **
>
>
>
> [Thu, 15 Jul 2010 23:03:53 GMT] [error] [<0.21959.3>] {error_report,<0.28.0>,
> {<0.21959.3>,crash_report,
> [[{pid,<0.21959.3>},
> {registered_name,[]},
> {error_info,
> {exit,
> {system_limit,
> [{ets,new,[replication_stats,[set,private]]},
> {couch_rep,do_init,1},
> {couch_rep,init,1},
> {gen_server,init_it,6},
> {proc_lib,init_p_do_apply,3}]},
> [{gen_server,init_it,6},{proc_lib,init_p_do_apply,3}]}},
> {initial_call,{couch_rep,init,['Argument__1']}},
> {ancestors,
> [couch_rep_sup,couch_primary_services,couch_server_sup,<0.29.0>]},
> {messages,[]},
> {links,[<0.22081.3>,<0.22182.3>,<0.22268.3>,<0.22166.3>,<0.78.0>]},
> {dictionary,[]},
> {trap_exit,true},
> {status,running},
> {heap_size,2584},
> {stack_size,23},
> {reductions,2097}],
> [{neighbour,
> [{pid,<0.22268.3>},
> {registered_name,[]},
> {initial_call,{erlang,apply,2}},
> {current_function,{erlang,apply,2}},
> {ancestors,[]},
> {messages,[]},
> {links,[<0.21959.3>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,runnable},
> {heap_size,377},
> {stack_size,0},
> {reductions,0}]}]]}}
>
> [Thu, 15 Jul 2010 23:03:54 GMT] [error] [<0.22166.3>] ** Generic
> server <0.22166.3> terminating
> ** Last message in was {'EXIT',<0.21959.3>,
> {system_limit,
> [{ets,new,[replication_stats,[set,private]]},
> {couch_rep,do_init,1},
> {couch_rep,init,1},
> {gen_server,init_it,6},
> {proc_lib,init_p_do_apply,3}]}}
> ** When Server state == {state,<0.22167.3>,nil,
> {db,<0.21978.3>,<0.22062.3>,nil,
> <<"1279235033964957">>,<0.21967.3>,
> <0.22063.3>,
> {db_header,5,0,0,nil,nil,nil,0,nil,nil,1000},
> 0,
> {btree,<0.21967.3>,nil,
> #Fun<couch_db_updater.7.82129660>,
> #Fun<couch_db_updater.8.42953822>,
> #Fun<couch_btree.5.124754102>,
> #Fun<couch_db_updater.9.115326703>},
> {btree,<0.21967.3>,nil,
> #Fun<couch_db_updater.10.103072508>,
> #Fun<couch_db_updater.11.104248294>,
> #Fun<couch_btree.5.124754102>,
> #Fun<couch_db_updater.12.125559248>},
> {btree,<0.21967.3>,nil,
> #Fun<couch_btree.0.83553141>,
> #Fun<couch_btree.1.30790806>,
> #Fun<couch_btree.2.124754102>,nil},
> 0,<<"db_2213">>,
>
> "/var/lib/mysql_data/couchdb-0.11.1-test/var/lib/couchdb/db_2213.couch",
> [],[],nil,
> {user_ctx,<<"network">>,
> [<<"_admin">>],
> <<"{couch_httpd_auth,
> default_authentication_handler}">>},
> nil,1000,
> [before_header,after_header,on_file_open]},
> <0.21959.3>,false,0,
> {<0.22184.3>,#Ref<0.0.2.116318>},
> {[],[]},
> 0,0,0}
> ** Reason for termination ==
> ** {system_limit,[{ets,new,[replication_stats,[set,private]]},
> {couch_rep,do_init,1},
> {couch_rep,init,1},
> {gen_server,init_it,6},
> {proc_lib,init_p_do_apply,3}]}
>
>
> [Thu, 15 Jul 2010 23:03:54 GMT] [error] [<0.22166.3>] {error_report,<0.28.0>,
> {<0.22166.3>,crash_report,
> [[{pid,<0.22166.3>},
> {registered_name,[]},
> {error_info,
> {exit,
> {system_limit,
> [{ets,new,[replication_stats,[set,private]]},
> {couch_rep,do_init,1},
> {couch_rep,init,1},
> {gen_server,init_it,6},
> {proc_lib,init_p_do_apply,3}]},
> [{gen_server,terminate,6},{proc_lib,init_p_do_apply,3}]}},
> {initial_call,{couch_rep_missing_revs,init,['Argument__1']}},
> {ancestors,
> [<0.21959.3>,couch_rep_sup,couch_primary_services,couch_server_sup,
> <0.29.0>]},
> {messages,[]},
> {links,[<0.22167.3>]},
> {dictionary,[]},
> {trap_exit,true},
> {status,running},
> {heap_size,233},
> {stack_size,23},
> {reductions,144}],
> []]}}
>
> [Thu, 15 Jul 2010 23:03:54 GMT] [error] [<0.22182.3>] ** Generic
> server <0.22182.3> terminating
> ** Last message in was {'EXIT',<0.21959.3>,
> {system_limit,
> [{ets,new,[replication_stats,[set,private]]},
> {couch_rep,do_init,1},
> {couch_rep,init,1},
> {gen_server,init_it,6},
> {proc_lib,init_p_do_apply,3}]}}
> ** When Server state == {state,<0.21959.3>,
> {http_db,
>
> "http://network:savetheSQL@g1dlsdbdb002.dev.glbt1.gdg:59845/db_462/",
> [],[],
> [{"User-Agent","CouchDB/0.11.1"},
> {"Accept","application/json"},
> {"Accept-Encoding","gzip"}],
> [],get,nil,
> [{response_format,binary},
> {inactivity_timeout,30000}],
> 10,500,nil},
> <0.22166.3>,<0.22184.3>,[],0,
> {[],[]},
> nil,false,0,nil,[],[]}
> ** Reason for termination ==
> ** {system_limit,[{ets,new,[replication_stats,[set,private]]},
> {couch_rep,do_init,1},
> {couch_rep,init,1},
> {gen_server,init_it,6},
> {proc_lib,init_p_do_apply,3}]}
>
>
> [Thu, 15 Jul 2010 23:03:54 GMT] [error] [<0.22182.3>] {error_report,<0.28.0>,
> {<0.22182.3>,crash_report,
> [[{pid,<0.22182.3>},
> {registered_name,[]},
> {error_info,
> {exit,
> {system_limit,
> [{ets,new,[replication_stats,[set,private]]},
> {couch_rep,do_init,1},
> {couch_rep,init,1},
> {gen_server,init_it,6},
> {proc_lib,init_p_do_apply,3}]},
> [{gen_server,terminate,6},{proc_lib,init_p_do_apply,3}]}},
> {initial_call,{couch_rep_reader,init,['Argument__1']}},
> {ancestors,
> [<0.21959.3>,couch_rep_sup,couch_primary_services,couch_server_sup,
> <0.29.0>]},
> {messages,[]},
> {links,[<0.22184.3>]},
> {dictionary,[]},
> {trap_exit,true},
> {status,running},
> {heap_size,987},
> {stack_size,23},
> {reductions,282}],
> [{neighbour,
> [{pid,<0.22184.3>},
> {registered_name,[]},
> {initial_call,{erlang,apply,2}},
> {current_function,{gen,wait_resp_mon,3}},
> {ancestors,[]},
> {messages,
> [{'DOWN',#Ref<0.0.2.116318>,process,<0.22166.3>,
> {system_limit,
> [{ets,new,[replication_stats,[set,private]]},
> {couch_rep,do_init,1},
> {couch_rep,init,1},
> {gen_server,init_it,6},
> {proc_lib,init_p_do_apply,3}]}}]},
> {links,[<0.22182.3>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,runnable},
> {heap_size,377},
> {stack_size,15},
> {reductions,14}]}]]}}
>
> [Thu, 15 Jul 2010 23:03:54 GMT] [error] [<0.31207.0>] Uncaught error
> in HTTP request: {error,
> {case_clause,
> {error,
> {{system_limit,
> [{ets,new,
> [replication_stats,[set,private]]},
> {couch_rep,do_init,1},
> {couch_rep,init,1},
> {gen_server,init_it,6},
> {proc_lib,init_p_do_apply,3}]},
> {child,undefined,
>
> "a0bd902c5803e5fb8e59918c8805d776+continuous+create_target",
> {gen_server,start_link,
> [couch_rep,
> ["a0bd902c5803e5fb8e59918c8805d776",
> {[{<<"source">>,
>
> <<"http://network:savetheSQL@g1dlsdbdb002.dev.glbt1.gdg:59845/db_462">>},
> {<<"target">>,<<"db_2213">>},
> {<<"continuous">>,true},
> {<<"create_target">>,true},
> {<<"cancel">>,false}]},
> {user_ctx,<<"network">>,
> [<<"_admin">>],
> <<"{couch_httpd_auth,
> default_authentication_handler}">>}],
> []]},
> temporary,1,worker,
> [couch_rep]}}}}}
>
> [Thu, 15 Jul 2010 23:03:54 GMT] [error] [<0.22081.3>] ** Generic
> server <0.22081.3> terminating
> ** Last message in was {'EXIT',<0.21959.3>,
> {system_limit,
> [{ets,new,[replication_stats,[set,private]]},
> {couch_rep,do_init,1},
> {couch_rep,init,1},
> {gen_server,init_it,6},
> {proc_lib,init_p_do_apply,3}]}}
> ** When Server state == {state,nil,nil,
> [<0.21959.3>,
> {http_db,
>
> "http://network:savetheSQL@g1dlsdbdb002.dev.glbt1.gdg:59845/db_462/",
> [],[],
> [{"User-Agent","CouchDB/0.11.1"},
> {"Accept","application/json"},
> {"Accept-Encoding","gzip"}],
> [],get,nil,
> [{response_format,binary},
> {inactivity_timeout,30000}],
> 10,500,nil},
> 0,
> [{<<"source">>,
>
> <<"http://network:savetheSQL@g1dlsdbdb002.dev.glbt1.gdg:59845/db_462">>},
> {<<"target">>,<<"db_2213">>},
> {<<"continuous">>,true},
> {<<"create_target">>,true},
> {<<"cancel">>,false}]],
> 0,<0.22165.3>,
> {1279,235033,989017},
> false,0,<<>>,
> {<0.22167.3>,#Ref<0.0.2.116315>},
> {[],[]}}
> ** Reason for termination ==
> ** {system_limit,[{ets,new,[replication_stats,[set,private]]},
> {couch_rep,do_init,1},
> {couch_rep,init,1},
> {gen_server,init_it,6},
> {proc_lib,init_p_do_apply,3}]}
>
>
> [Thu, 15 Jul 2010 23:03:54 GMT] [error] [<0.22081.3>] {error_report,<0.28.0>,
> {<0.22081.3>,crash_report,
> [[{pid,<0.22081.3>},
> {registered_name,[]},
> {error_info,
> {exit,
> {system_limit,
> [{ets,new,[replication_stats,[set,private]]},
> {couch_rep,do_init,1},
> {couch_rep,init,1},
> {gen_server,init_it,6},
> {proc_lib,init_p_do_apply,3}]},
> [{gen_server,terminate,6},{proc_lib,init_p_do_apply,3}]}},
> {initial_call,{couch_rep_changes_feed,init,['Argument__1']}},
> {ancestors,
> [<0.21959.3>,couch_rep_sup,couch_primary_services,couch_server_sup,
> <0.29.0>]},
> {messages,
> [{ibrowse_async_response,
> {1279,235033,989017},
> {error,closing_on_request}},
> {'EXIT',<0.22165.3>,normal}]},
> {links,[]},
> {dictionary,[]},
> {trap_exit,true},
> {status,running},
> {heap_size,987},
> {stack_size,23},
> {reductions,903}],
> []]}}
>
> [Thu, 15 Jul 2010 23:03:54 GMT] [info] [<0.31207.0>] Stacktrace:
> [{couch_rep,start_replication_server,1},
> {couch_rep,replicate,2},
> {couch_httpd_misc_handlers,handle_replicate_req,1},
> {couch_httpd,handle_request_int,5},
> {mochiweb_http,headers,5},
> {proc_lib,init_p_do_apply,3}]
>
> [Thu, 15 Jul 2010 23:03:54 GMT] [info] [<0.31207.0>] 172.19.68.122 - -
> 'POST' /_replicate 500
>
> [Thu, 15 Jul 2010 23:03:54 GMT] [debug] [<0.31207.0>] httpd 500 error response:
> {"error":"case_clause","reason":"{error,\n {{system_limit,\n
> [{ets,new,[replication_stats,[set,private]]},\n
> {couch_rep,do_init,1},\n {couch_rep,init,1},\n
> {gen_server,init_it,6},\n {proc_lib,init_p_do_apply,3}]},\n
> {child,undefined,\n
> \"a0bd902c5803e5fb8e59918c8805d776+continuous+create_target\",\n
> {gen_server,start_link,\n [couch_rep,\n
> [\"a0bd902c5803e5fb8e59918c8805d776\",\n
> {[{<<\"source\">>,\n
> <<\"http://network:savetheSQL@g1dlsdbdb002.dev.glbt1.gdg:59845/db_462\">>},\n
> {<<\"target\">>,<<\"db_2213\">>},\n
> {<<\"continuous\">>,true},\n
> {<<\"create_target\">>,true},\n
> {<<\"cancel\">>,false}]},\n {user_ctx,<<\"network\">>,\n
> [<<\"_admin\">>],\n
> <<\"{couch_httpd_auth, default_authentication_handler}\">>}],\n
> []]},\n temporary,1,worker,\n [couch_rep]}}}"}
>
> ---
|