couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Igor Klimer <i.kli...@getbacksa.pl>
Subject ODP: Error during compaction
Date Tue, 21 Jan 2014 09:08:23 GMT
Hi,
I'm extremely sorry for not replying sooner, however I was on sick leave last week.
I've tried your suggestion with an empty .compact file, however the results seem to be the
same...
Log: http://pastebin.com/MJCgGM8C

Started with an empty ecrepo.couch.compact file (touch ecrepo.couch.compact), then after about
3 hours, the error was printed in the logs and the compaction failed:
-rw-r--r-- 1 couchdb couchdb 137502523517 Jan 21 09:51 ecrepo.couch
-rw-r--r-- 1 couchdb couchdb  51692612367 Jan 21 02:07 ecrepo.couch.compact

There's over 100GB free space available on the disk.

At least I think I know what the number 51692471440 in log means ;) But I don't know if there's
a way to check which document resides at that position in file.

Best regards,
Igor Klimer

________________________________________
Od: Robert Samuel Newson [rnewson@apache.org]
Wysłano: 10 stycznia 2014 18:45
Do: user
Temat: Re: Error during compaction

Yes, I understood. The empty .compact file will trigger more checking in the compaction process,
I’m hoping it gets us past the problem.

B.

On 10 Jan 2014, at 13:34, Igor Klimer <i.klimer@getbacksa.pl> wrote:

> :)
> Just to clarify - the .compact file is getting created and then the compaction fails
after some time (an hour or more):
> 1) attempt on Windows with Couchdb 1.2.0 it failed because insufficient disk space. The
.compact file had at least 10GB, unfortunately, I don't remember how much (and whether it
was bigger then the one produced in the later attempts). There was no free disk space when
it failed, so I'm assuming that was the cause.
> 2) attempt on Windows with Couchdb 1.2.0 it failed with the error mentioned below. The
.compact file had around 50GB, there was plenty of free space left on the disk.
> 3) attempt on Ubuntu with Couchdb 1.5.0 it failed with the error mentioned below. The
.compact file had around 50GB, there was plenty of free space left on the disk, and judging
from the numbers present in the log (ids? node numbers?) it failed at the same moment as attempt
#2.
>
> Just wanted to make sure we're on the same page :) Do you still want me to try it with
an empty .compact file? (I can do this only during night hours, since I don't want to put
too much load on the server during working hours)
>
> Best regards,
> Igor Klimer
> ________________________________________
> Od: Robert Samuel Newson [rnewson@apache.org]
> Wysłano: 10 stycznia 2014 14:03
> Do: user
> Temat: Re: Error during compaction
>
> Hrm, strike one. Ok. Next thing to try is subtly different. stop couchdb, delete the
.compact file, but then make a new, empty .compact file (so ’touch /path/to/dbname.compact’),
start couchdb and compact.
>
> B.
>
> On 10 Jan 2014, at 12:42, Igor Klimer <i.klimer@getbacksa.pl> wrote:
>
>> Yes, I've already done that after the very fist attempt at compaction (the one that
failed because of lack of disk space). And it resulted in the second fail (on Windows), then
the same on Linux - I always deleted the incomplete (about 50% of the database, around 50GB)
.compact file before running the compaction again. So I was always doing compaction from scratch.
>>
>> Best regards,
>> Igor Klimer
>> ________________________________________
>> Od: Robert Samuel Newson [rnewson@apache.org]
>> Wysłano: 10 stycznia 2014 13:08
>> Do: user
>> Temat: Re: Error during compaction
>>
>> Thanks! that’s very useful. Hitting end of disk certainly feels like a cause here.
Since the compaction has never completed, I suggest we redo compaction from scratch.
>>
>> 1) stop couchdb
>> 2) delete (or move aside) the dbname.compact file for this database
>> 3) start couchdb
>> 4) compact the db
>>
>> Whether it works or not, please let us know.
>>
>> B.
>>
>> On 10 Jan 2014, at 08:25, Igor Klimer <i.klimer@getbacksa.pl> wrote:
>>
>>>> Given that you’re at 100Gb and compacting for the first time, can you tell
us if you were running on older couchdb versions that 1.2.0 between db creation and today?
>>>
>>> No, we've been running 1.2.0 from the start (around Oct 2012), then switched
to Ubuntu and 1.5.0.
>>>
>>>> Do you have free disk space?
>>> Yes, there's about 150% of the DB's size worth of free space :) I forgot to mention
("OK, here we go, the user will confess to some sin he committed and is ashamed of and is
most likely the reason for this failure") that we've run the compaction once before the error
on Windows I mentioned below, but it failed because of insufficient disk space - so I double
checked before running the compaction again if there's enough space. Here's the log, if it's
any helpful: http://pastebin.com/S1URXN0p
>>> Do you think it could have left the database in some corrupted state? It seems
it failed at a different part then the two next attempts (and, as far as I understand, compaction
is just copying over the database while pruning the old revisions and deleted documents).
>>>
>>> Thank you for your time and help and best regards,
>>> Igor Klimer
>>> ________________________________________
>>> Od: Robert Samuel Newson [rnewson@apache.org]
>>> Wysłano: 9 stycznia 2014 17:13
>>> Do: user
>>> Temat: Re: Error during compaction
>>>
>>> Do you have free disk space?
>>>
>>> On 9 Jan 2014, at 15:25, Robert Samuel Newson <rnewson@apache.org> wrote:
>>>
>>>>
>>>> Given that you’re at 100Gb and compacting for the first time, can you tell
us if you were running on older couchdb versions that 1.2.0 between db creation and today?
>>>>
>>>> B.
>>>>
>>>> On 9 Jan 2014, at 14:39, Igor Klimer <i.klimer@getbacksa.pl> wrote:
>>>>
>>>>> Hi all,
>>>>> I've stumbled upon a peculiar problem while trying to compact (for the
first time) a large(-ish) database (~100GB at that time). At about 50% it failed with this
error: http://pastebin.com/qeaZNHMj
>>>>> This is from Windows Server 2008 R2 Enterprise with Couchdb 1.2.0.
>>>>> I figured that it might be a bug in the Windows build (Erlang on Windows?
C'mon, that can't be good ;)) or already fixed in a newer version. Some time later we migrated
the server to a Linux box running Ubuntu 12.04.3 LTS (GNU/Linux 3.8.0-33-generic x86_64) and
update Couchdb to 1.5.0.
>>>>> Unfortunately, the same error occurred: http://pastebin.com/feJWu7bN
>>>>>
>>>>> I've tried wrapping my head around that error, googling it, checking
this mail list but to no avail :) So if anyone can give me any pointers as to what might be
causing this problem, I'd be very grateful.
>>>>>
>>>>> Best regards,
>>>>> Igor Klimer
>>>>>
>>>>> (sorry for the footer that will probably follow, unfortunately it's added
for all outgoing external mail...)
>>>>>
>>>>>
>>>>>
>>>>>




-------------------------------

getBACK S.A., ul. Powstańców Śląskich 2-4, 53-333 Wrocław
Sad rejestrowy: Sąd Rejonowy dla Wrocławia - Fabrycznej, VI Wydział Gospodarczy KRS.
Numer KRS: 0000413997
NIP: 8992733884
REGON: 021829989

Wysokość kapitału zakładowego opłaconego w całości: 4 000 000,00 zł

Zamieszczenie powyższych danych identyfikujących getBACK S.A. stosownie do art. 374 par.1
Kodeksu spółek handlowych nie jest równoznaczne z handlowym charakterem dostarczonej do
Państwa wiadomości e-mailowej i pozostaje bez wpływu na interpretacje zawartych w niej
oświadczeń.


Niniejszy e-mail oraz wszelkie załączone do niego pliki są poufne i mogą podlegać ochronie
prawnej. Jeżeli nie jest Pan/Pani zamierzonym adresatem powyższej wiadomości, nie może
jej Pan/Pani ujawniać, kopiować, dystrybuować, ani tez w żaden inny sposób udostępniać
lub wykorzystywać. O błędnym zaadresowaniu wiadomości prosimy niezwłocznie poinformować
nadawcę i usunąć wiadomość.


This e-mail message may contain confidential and/or privileged information. If you are not
the intended recipient (or have received this e-mail in error) please notify the sender immediately
and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material
in this e-mail is strictly forbidden.

Mime
View raw message