couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "[mRg]" <emar...@gmail.com>
Subject Re: Couch not releasing deleted files
Date Mon, 27 Sep 2010 10:14:56 GMT
Anyone still following this on here, I asked this question on the dev list
and Filipe has committed a potential fix for this issue here:

http://svn.apache.org/viewvc?view=revision&revision=1001196

<http://svn.apache.org/viewvc?view=revision&revision=1001196>if people are
still having this issue they could try applying this, we are now doing
internal testing to see if this has fixed it for our use case.

Regards

Stephen

On 19 September 2010 00:36, Robert Newson <robert.newson@gmail.com> wrote:

> It's been discussed by dev's at CouchCamp but a mailing list reminder
> couldn't hurt. There's some discussion about 1.1, so perhaps that's a
> good thread to raise this on.
>
> B.
>
> On Sun, Sep 19, 2010 at 12:24 AM, [mRg] <emargee@gmail.com> wrote:
> > Has this been raised with the dev list or is there a bug raised for this
> > issue ? Seems a few people are having this and more are probably not
> seeing
> > yet.
> >
> > On 15 September 2010 08:49, [mRg] <emargee@gmail.com> wrote:
> >
> >> Yes there are 6 views (2 design docs)
> >>
> >>
> >> On 14 September 2010 21:51, Adam Kocoloski <kocolosk@apache.org> wrote:
> >>
> >>> Were there any view indexes on that DB?  My suspicion is that
> >>> couch_view_group misses a step somewhere in its dance and keeps a
> reference
> >>> to an old #db{}
> >>>
> >>> Adam
> >>>
> >>> On Sep 14, 2010, at 4:46 PM, [mRg] wrote:
> >>>
> >>> > It was just on a DB with a lot of writes (documents having lots of
> >>> > revisions, rather that lots of new docs), the database grows as
> expected
> >>> but
> >>> > after just doing a simple compact on the db the old files still seem
> to
> >>> be
> >>> > on disk as reported previously.
> >>> >
> >>> > It seems its been around a while now as (as far as we can see) 0.11
/
> >>> 1.0
> >>> > and 1.0.1 all have the same issue, when we replicated the same data
> over
> >>> to
> >>> > the newer versions the same identical problem happened.
> >>> >
> >>> > On 14 September 2010 14:49, Robert Newson <robert.newson@gmail.com>
> >>> wrote:
> >>> >
> >>> >> This was mentioned at CouchCamp, several developers suspect there's
> a
> >>> >> subtle bug in the code where file descriptors are not being closed
> >>> >> under some conditions.
> >>> >>
> >>> >> Did the OP identify a particular sequence that led here?
> >>> >>
> >>> >> B.
> >>> >>
> >>> >> On Mon, Sep 13, 2010 at 6:25 PM, Juan Jose Comellas <
> >>> juanjo@comellas.org>
> >>> >> wrote:
> >>> >>> I'm pretty certain it's a bug in CouchDB. I haven't checked
the
> code,
> >>> but
> >>> >>> what is happening is that CouchDB is keeping file descriptors
> >>> >> corresponding
> >>> >>> to deleted files open, probably after compacting a database.
I'm
> >>> >> inferring
> >>> >>> this based on the fact that with lsof I saw several cases of
two
> files
> >>> >> with
> >>> >>> the same name (one deleted and one not) opened by CouchDB.
> >>> >>>
> >>> >>>
> >>> >>> On Mon, Sep 13, 2010 at 7:28 PM, [mRg] <emargee@gmail.com>
wrote:
> >>> >>>
> >>> >>>> Is this an issue with the OS or with Couch/Beam itself
?
> >>> >>>>
> >>> >>>> On 10 September 2010 16:57, Juan Jose Comellas <
> juanjo@comellas.org>
> >>> >>>> wrote:
> >>> >>>>
> >>> >>>>> It also happened to me with Ubuntu 8.04 and CouchDB
0.10.x. The
> only
> >>> >> way
> >>> >>>> I
> >>> >>>>> found to solve the problem was by restarting CouchDB.
> >>> >>>>>
> >>> >>>>>
> >>> >>>>> On Thu, Sep 9, 2010 at 10:25 AM, [mRg] <emargee@gmail.com>
> wrote:
> >>> >>>>>
> >>> >>>>>> [CouchDB Version : 0.11]
> >>> >>>>>> [OS: RHEL 5.5]
> >>> >>>>>>
> >>> >>>>>> Hi all,
> >>> >>>>>>
> >>> >>>>>> We are seeing some odd behaviour with CouchDb not
releasing
> files
> >>> >> after
> >>> >>>>> it
> >>> >>>>>> has compacted them meaning we are seeing disk space
continue to
> >>> >>>> increase
> >>> >>>>>> until the service is restarted even though we have
regular
> >>> >> compaction /
> >>> >>>>>> cleanup.
> >>> >>>>>>
> >>> >>>>>> Typical scenario : File system alert for disk space
..
> >>> >>>>>>
> >>> >>>>>> df says 5.5.GB used ..
> >>> >>>>>>
> >>> >>>>>> df -h | grep -P 'Filesystem|db'
> >>> >>>>>>
> >>> >>>>>> Filesystem            Size  Used Avail Use% Mounted
on
> >>> >>>>>>
> >>> >>>>>>                     8.0G  5.5G  2.2G  72% /var/lib/db
> >>> >>>>>>
> >>> >>>>>>
> >>> >>>>>> .. but the files on the disk (and in Futon) say
only 793MB is
> being
> >>> >>>> used
> >>> >>>>> ..
> >>> >>>>>>
> >>> >>>>>>
> >>> >>>>>> du -sh /var/lib/db/*
> >>> >>>>>>
> >>> >>>>>> 793M     /var/lib/db/couchdb
> >>> >>>>>>
> >>> >>>>>>
> >>> >>>>>> .. running lsof shows the files marked as deleted
but still
> there
> >>> >> (I'm
> >>> >>>>>> assuming beam still has some kind of handle still
open to them)
> ..
> >>> >>>>>>
> >>> >>>>>>
> >>> >>>>>> lsof | grep -P 'COMMAND|/var/lib/db/couchdb/'
> >>> >>>>>>
> >>> >>>>>> COMMAND     PID      USER   FD      TYPE      
      DEVICE
> >>> >> SIZE
> >>> >>>>>> NODE NAME
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   12u      REG      
      253,10
> >>> >> 4183
> >>> >>>>>> 1210374
> >>> >>>>>>
> >>> >>>>>>
> >>> >>>>>
> >>> >>>>
> >>> >>
> >>>
> /var/lib/db/couchdb/.active_context_design/8efe9584a944f4b24b97f134f541d80f.view
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   16u      REG      
      253,10
> >>> >> 612039080
> >>> >>>>>> 1204227 /var/lib/db/couchdb/active_context.couch
(deleted)
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   20u      REG      
      253,10
> >>> >> 180699242
> >>> >>>>>> 1204237 /var/lib/db/couchdb/active_context.couch
(deleted)
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   21u      REG      
      253,10
> >>> >> 3133540
> >>> >>>>>> 2023429 /var/lib/db/couchdb/thesaurus.couch (deleted)
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   25u      REG      
      253,10
> >>> >> 71422052
> >>> >>>>>> 2023428 /var/lib/db/couchdb/active_context_schedule.couch
> (deleted)
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   26u      REG      
      253,10
> >>> >> 71372900
> >>> >>>>>> 2056193 /var/lib/db/couchdb/active_context_schedule.couch
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   28u      REG      
      253,10
> >>> >> 621252717
> >>> >>>>>> 1204238 /var/lib/db/couchdb/active_context.couch
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   29u      REG      
      253,10
> >>> >> 1740903
> >>> >>>>>> 90113 /var/lib/db/couchdb/mapping_cache.couch
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   30u      REG      
      253,10
> >>> >> 3133540
> >>> >>>>>> 1204253 /var/lib/db/couchdb/thesaurus.couch (deleted)
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   31u      REG      
      253,10
> >>> >> 2318439
> >>> >>>>>> 1204240 /var/lib/db/couchdb/mappings.couch (deleted)
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   32u      REG      
      253,10
> >>> >> 983150
> >>> >>>>>> 2004994
> >>> >>>>>>
> >>> >>>>>
> >>> >>>>
> >>> >>
> >>>
> /var/lib/db/couchdb/.mappings_design/f9964c50b95c167c8ac3d8d3b4c76a4b.view
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   33u      REG      
      253,10
> >>> >> 3133540
> >>> >>>>>> 2056194 /var/lib/db/couchdb/thesaurus.couch
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   34u      REG      
      253,10
> >>> >> 70742116
> >>> >>>>>> 1204231 /var/lib/db/couchdb/active_context_schedule.couch
> (deleted)
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   44u      REG      
      253,10
> >>> >> 1740903
> >>> >>>>>> 2023427 /var/lib/db/couchdb/mapping_cache.couch
(deleted)
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   45u      REG      
      253,10
> >>> >> 65396836
> >>> >>>>>> 1994756 /var/lib/db/couchdb/active_context_schedule.couch
> (deleted)
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   47u      REG      
      253,10
> >>> >> 71229540
> >>> >>>>>> 1204243 /var/lib/db/couchdb/active_context_schedule.couch
> (deleted)
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   48u      REG      
      253,10
> >>> >> 240254078
> >>> >>>>>>  1204226 /var/lib/db/couchdb/active_context.couch
(deleted)
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   49u      REG      
      253,10
> >>> >> 2232423
> >>> >>>>>> 1204241 /var/lib/db/couchdb/mappings.couch
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   51u      REG      
      253,10
> >>> >> 96903353
> >>> >>>>>> 1210371
> >>> >>>>>>
> >>> >>>>>>
> >>> >>>>>
> >>> >>>>
> >>> >>
> >>>
> /var/lib/db/couchdb/.active_context_design/b406f7bb6d01b6d50324f14cde588324.view
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   52u      REG      
      253,10
> >>> >> 1192034
> >>> >>>>>> 1249282
> >>> >>>>>>
> >>> >>>>>>
> >>> >>>>>
> >>> >>>>
> >>> >>
> >>>
> /var/lib/db/couchdb/.mapping_cache_design/8bbf80a6e1ca8f2c71ebc90c730e7190.view
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   53u      REG      
      253,10
> >>> >> 1622135
> >>> >>>>>> 2004993
> >>> >>>>>>
> >>> >>>>>
> >>> >>>>
> >>> >>
> >>>
> /var/lib/db/couchdb/.mappings_design/1ff69004b575837aae9d4147162197d4.view
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   54u      REG      
      253,10
> >>> >> 4172
> >>> >>>>>> 1208327
> >>> >>>>>>
> >>> >>>>>
> >>> >>>>
> >>> >>
> >>>
> /var/lib/db/couchdb/.thesaurus_design/73fe79b20ff06e341257fa78231ca87e.view
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   55u      REG      
      253,10
> >>> >> 13496483
> >>> >>>>>> 1208325
> >>> >>>>>>
> >>> >>>>>
> >>> >>>>
> >>> >>
> >>>
> /var/lib/db/couchdb/.thesaurus_design/fcc08d6e41c54c0d470c5984ea9eb6e3.view
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   56u      REG      
      253,10
> >>> >> 12959902
> >>> >>>>>> 1208326
> >>> >>>>>>
> >>> >>>>>
> >>> >>>>
> >>> >>
> >>>
> /var/lib/db/couchdb/.thesaurus_design/c06195da19f34ced37fa1ab91e663c73.view
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   57u      REG      
      253,10
> >>> >> 70254692
> >>> >>>>>> 1204236 /var/lib/db/couchdb/active_context_schedule.couch
> (deleted)
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   58u      REG      
      253,10
> >>> >> 581730
> >>> >>>>>> 1210373
> >>> >>>>>>
> >>> >>>>>>
> >>> >>>>>
> >>> >>>>
> >>> >>
> >>>
> /var/lib/db/couchdb/.active_context_schedule_design/c64c047dcff74ea2122356efb1042759.view
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   67u      REG      
      253,10
> >>> >> 1818727
> >>> >>>>>> 1204247 /var/lib/db/couchdb/mapping_cache.couch
(deleted)
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   78u      REG      
      253,10
> >>> >> 609905146
> >>> >>>>>> 1204234 /var/lib/db/couchdb/active_context.couch
(deleted)
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   80u      REG      
      253,10
> >>> >> 545919085
> >>> >>>>>> 1994753 /var/lib/db/couchdb/active_context.couch
(deleted)
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   81u      REG      
      253,10
> >>> >> 71032932
> >>> >>>>>> 1204249 /var/lib/db/couchdb/active_context_schedule.couch
> (deleted)
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   84u      REG      
      253,10
> >>> >> 548866111
> >>> >>>>>> 1204239 /var/lib/db/couchdb/active_context.couch
(deleted)
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   85u      REG      
      253,10
> >>> >> 564612402
> >>> >>>>>> 1204233 /var/lib/db/couchdb/active_context.couch
(deleted)
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb   87u      REG      
      253,10
> >>> >> 615996179
> >>> >>>>>> 1204228 /var/lib/db/couchdb/active_context.couch
(deleted)
> >>> >>>>>>
> >>> >>>>>> beam.smp  21352   couchdb  102u      REG      
      253,10
> >>> >> 561111149
> >>> >>>>>> 2023425 /var/lib/db/couchdb/active_context.couch
(deleted)
> >>> >>>>>>
> >>> >>>>>>
> >>> >>>>>> Has anyone seen this behaviour before or knows
of a way around
> this
> >>> >> ?
> >>> >>>>>> Currently we have to restart the couchdb service
in order to
> >>> restore
> >>> >>>> the
> >>> >>>>>> disk space.
> >>> >>>>>>
> >>> >>>>>> Any help/advice is much appreciated.
> >>> >>>>>>
> >>> >>>>>> Regards
> >>> >>>>>>
> >>> >>>>>> Stephen
> >>> >>>>>>
> >>> >>>>>
> >>> >>>>
> >>> >>>
> >>> >>
> >>>
> >>>
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message