couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arif Khan <>
Subject Couch 2.x cluster returning inconsistent _all_docs
Date Thu, 16 Aug 2018 18:25:57 GMT

We have a three node couchDB 2.x (one 2.1.1 and two 2.1.2) cluster which is
returning inconsistent number of _all_docs when we hit _all_docs endpoint
on any db that has more documents in addition to two _design docs (i.e
logDocs and permissions). Following are some error messages from the log

[error] 2018-08-16T18:16:15.150627Z couchdb@trinary.couchcluster.internal
<0.16547.1789> -------- Error opening view group `permissions` from
database `shards/55555554-71c71c6f/shadow_4495.1514409098`:

[error] 2018-08-16T18:15:00.068553Z couchdb@secondary.couchcluster.internal
<0.26642.4837> -------- CRASH REPORT Process couch_index_server
(<0.26642.4837>) with 1 neighbors exited with reason: no match of right
hand value {error,emfile} at couch_file:init/1(line:398) <=
gen_server:init_it/6(line:304) <= proc_lib:init_p_do_apply/3(line:239) at
gen_server:terminate/6(line:744) <= proc_lib:init_p_do_apply/3(line:239);
initial_call: {couch_index_server,init,['Argument__1']}, ancestors:
[couch_secondary_services,couch_sup,<0.204.0>], messages: [], links:
[<0.4278.4837>,<0.19436.4831>], dictionary: [], trap_exit: true, status:
running, heap_size: 6772, stack_size: 27, reductions: 1050564

Please note, we recently replaced two nodes in the cluster. The nodes got
replaced were named like couchdb@<ip-address> whereas the nodes replaced
them are named as couchdb@<fqdn> . Following is a high level process we

- Stop couchDB process in the outgoing node
- Add the incoming node to the cluster
- Update cluster metadata for all databases replacing the outgoing node
with the incoming one (in both by_node and by_range dictionaries)
- Wait until all shards have arrived at the new node
- Delete the old node from the cluster

Please help.


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message