incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Kocoloski <kocol...@apache.org>
Subject Re: Compact not completing
Date Tue, 09 Aug 2011 20:10:22 GMT
Hi Simon, CouchDB 1.1.0 includes a recent optimization to _changes?include_docs=true which
allows it to skip a lookup in the id tree and instead load the document body from the pointer
in the sequence tree.  In that case you wouldn't notice any missing entry in the id tree.
 You would notice it, however, if you did direct lookups for each document. Apologies for
the outdated instructions.  Can you try looking up the documents in a separate request and
see if the results change?

Adam

On Aug 9, 2011, at 5:49 AM, Simon Eisenmann wrote:

> Hi Adam,
> 
> i just checked the whole _changes feed (since=0) and could not find any
> document "missing" when using "include_docs=true".
> 
> The database in itself has only around 30 documents, so should be quite
> small. Though there are lots of creations and deletions happening all
> the time. Thus it its daily purged and compacted. 
> 
> So - the changes feed is of no help. Any other idea?
> 
> Thank you and best regards
> Simon
> 
> Am Montag, den 08.08.2011, 13:25 -0400 schrieb Adam Kocoloski:
>> Hi Simon, I think my amended instructions to Mike are still a sensible way to debug/workaround
the problem.  Reiterating (96282148 was the last seq Mike observed in the Futon status for
the compaction):
>> 
>>> 1) What you really want are the last 1000 Ids in the seq_tree prior to the compactor
crash.  So maybe something like
>>> 
>>> GET /iris/_changes?descending=true&limit=1000&since=96282148
>> 
>>> 2) Figure out which of those entries are missing from the id tree, e.g. lookup
the document and see if the response is {"not_found":"missing"}.  You could also try using
include_docs=true on the _changes feed to accomplish the same.
>> 
>>> 3) Once you've identified the problematic IDs, try creating them again.  You
might end up introducing duplicates in the _changes feed, but if you do there's a procedure
to fix that.
>> 
>> Regards, Adam
>> 
>> On Aug 8, 2011, at 12:31 PM, Simon Eisenmann wrote:
>> 
>>> Hi Guys,
>>> 
>>> i have a couple of CouchDB instances which started to be come
>>> unpackable. It shows this error:
>>> 
>>> [Mon, 08 Aug 2011 16:16:27 GMT] [info] [<0.10808.123>] Starting
>>> compaction for db "database1"
>>> [Mon, 08 Aug 2011 16:16:45 GMT] [error] [<0.10808.123>] ** Generic
>>> server <0.10808.123> terminating 
>>> ** Last message in was {'EXIT',<0.30396.143>,
>>>                       {function_clause,
>>>                        [{couch_db_updater,'-copy_docs/4-fun-2-',
>>>                          [not_found,
>>>                           {db,<0.10807.123>,<0.10808.123>,nil,
>>>                            <<"1312767347007568">>,<0.10805.123>,
>>>                            <0.10809.123>,
>>>                            {db_header,5,6340261,0,
>>>                             {7198006895,{65952,10145}},
>>>                             {7198010315,76813},
>>>                             {7198051016,[]},
>>>                             364,7050915618,nil,1000},
>>>                            6340261,
>>>                            {btree,<0.10805.123>,
>>>                             {7198006895,{65952,10145}},
>>>                             #Fun<couch_db_updater.10.19222179>,
>>>                             #Fun<couch_db_updater.11.21515767>,
>>>                             #Fun<couch_btree.5.124754102>,
>>>                             #Fun<couch_db_updater.12.93888648>},
>>>                            {btree,<0.10805.123>,
>>>                             {7198010315,76813},
>>>                             #Fun<couch_db_updater.13.40165027>,
>>>                             #Fun<couch_db_updater.14.82810239>,
>>>                             #Fun<couch_btree.5.124754102>,
>>>                             #Fun<couch_db_updater.15.104121193>},
>>>                            {btree,<0.10805.123>,
>>>                             {7198051016,[]},
>>>                             #Fun<couch_btree.0.83553141>,
>>>                             #Fun<couch_btree.1.30790806>,
>>>                             #Fun<couch_btree.2.124754102>,nil},
>>>                            6340261,<<"spreedcom_accounts_1">>,
>>>                            "/var/lib/couchdb/database1.couch",[],
>>>                            [],nil,
>>>                            {user_ctx,null,[],undefined},
>>>                            nil,1000,
>>>                            [before_header,after_header,on_file_open],
>>>                            false},
>>>                           <0.30397.143>]},
>>>                         {lists,map,2},
>>>                         {lists,map,2},
>>>                         {couch_db_updater,copy_docs,4},
>>>                         {couch_db_updater,'-copy_compact/3-fun-0-',6},
>>>                         {couch_btree,stream_kv_node2,8},
>>>                         {couch_btree,stream_kp_node,7},
>>>                         {couch_btree,fold,4}]}}
>>> 
>>> 
>>> ... lots of more similar errors following.
>>> 
>>> In this mailing list i have found a similar issue from the beginning of
>>> this year (Fri, 31 Dec 2010 12:38:18), though without a solution.
>>> 
>>> This database got purged a lot and has constant changes. So pretty
>>> similar from the older topic. So i assume there is something wrong in
>>> the database file related to previous purges. So looks like some bug
>>> here.
>>> 
>>> So - any hints how to fix this? The database is getting pretty large and
>>> has to be packed from time to time.
>>> 
>>> CouchDB is running with Version 1.1.0 on Linux 64bit. The database has
>>> initially been created with CouchDB 1.0.1 - though the issue appeared a
>>> couple of weeks ago (packing has been working with 1.1.0 before.
>>> 
>>> 
>>> Thank you and best regards
>>> Simon
>>> 
>>> 
>>> -- 
>>> Simon Eisenmann
>>> 
>>> [ mailto:simon@struktur.de ]
>>> 
>>> [ struktur AG | Kronenstra├če 22a | D-70173 Stuttgart ]
>>> [ T. +49.711.896656.0 | F.+49.711.89665610 ]
>>> [ http://www.struktur.de | mailto:info@struktur.de ]
>> 
> 
> -- 
> Simon Eisenmann
> 
> [ mailto:simon@struktur.de ]
> 
> [ struktur AG | Kronenstra├če 22a | D-70173 Stuttgart ]
> [ T. +49.711.896656.68 | F.+49.711.89665610 ]
> [ http://www.struktur.de | mailto:info@struktur.de ]


Mime
View raw message