jakarta-jcs-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Smuts <asm...@yahoo.com>
Subject RE: index cache corruption
Date Wed, 12 Aug 2009 14:11:45 GMT
I think I found it.  During partial key removeall the Disk Element Descriptor, which identifies
the slot on disk, was being added to the recylebin.  Then the removeSingleItem was called
for each match.  The remove single item method also adds the item to the recylebin.  The recyclebin
is not a set.  The spot would get entered twice.  This is very bad.  Here's why:

The spot gets used.  Another put uses the same spot.  Both keys in the key map point to the
same spot.  A get for the first key will pull the data for the second.  Not so good.  

I'm verifying this and I'll have a fix soon.

Aaron

--- On Wed, 8/12/09, Aaron Smuts <asmuts@yahoo.com> wrote:

> From: Aaron Smuts <asmuts@yahoo.com>
> Subject: RE: index cache corruption
> To: "JCS Users List" <jcs-users@jakarta.apache.org>
> Date: Wednesday, August 12, 2009, 6:52 AM
> 
> So were you getting back the wrong data or data that should
> have been deleted?  
> 
> Are you running a recent temp build?
> 
> I'll dig into the partial key removal.  This should
> probably deprecated for the remove matching method. . .
> .  Either way, it should work.
> 
> Aaron
> 
> --- On Tue, 8/11/09, Tim Cronin <Tim.Cronin@autonomy.com>
> wrote:
> 
> > From: Tim Cronin <Tim.Cronin@autonomy.com>
> > Subject: RE: index cache corruption
> > To: "JCS Users List" <jcs-users@jakarta.apache.org>
> > Date: Tuesday, August 11, 2009, 2:30 PM
> > Good to know, thanks.
> > 
> > Yes that's when we see it too, under high load. 
> > 
> > For us it seems to happen around removes, we use the
> key:
> > linkage and
> > It happens to data that's linked and would get cleared
> with
> > the key:
> > call.
> > 
> > 
> > -----Original Message-----
> > From: Al Forbes [mailto:forbes.al@googlemail.com]
> > 
> > Sent: Tuesday, August 11, 2009 4:13 PM
> > To: JCS Users List
> > Subject: Re: index cache corruption
> > 
> > Hi Tim,
> > 
> > I had the same problem. It seems to happen under high
> load,
> > with fairly
> > large objects, but I could not reproduce it offline.
> I
> > changed to using
> > a
> > JDBC cache and have not had a problem since then.
> > 
> > We still use disk caches for more static data - mostly
> read
> > only, and
> > the
> > problem never happens there, so I assume it's the
> writes
> > that cause the
> > corruption.
> > 
> > Regards
> > Al
> > 
> > 2009/8/11 Tim Cronin <Tim.Cronin@autonomy.com>
> > 
> > > We initially were using indexed disk cache but
> ran
> > into cache
> > corruption
> > > where the data returned for a key would not be
> the
> > data associated for
> > > that key. I haven't been able to come up with a
> good
> > repro scenario...
> > >
> > > We had to work around it with the following
> code:
> > >
> > >  /**
> > >   * is the cache element not the
> > correct object for the key
> > >   * @param key the current key
> > >   * @param element the current element
> > associated with the key
> > >   * @param hasReadLock whether caller
> > has read lock
> > >   * @return true if key mismatch els
> > false
> > >   * @throws InterruptedException if
> > locking fails
> > >   */
> > >  private boolean isCorrupt(String key,
> > ICacheElement element, boolean
> > > hasReadLock) throws InterruptedException
> > >  {
> > >    boolean corrupt =
> > !key.equals(element.getKey());
> > >    if (corrupt)
> > >    {
> > >      mLogger.error("cache corruption!!!
> > [" + (mCorruptionCounter++) +
> > > "]");
> > >
> > >      if (mLogger.isDebugEnabled())
> > >      {
> > >       mLogger.error("culprit
> > stack...", new Exception("cache
> > > corruption"));
> > >      }
> > >
> > >      try
> > >      {
> > >        if (hasReadLock)
> > >        {
> > >         
> > mLock.readLock().release();
> > >        }
> > >       
> > mLock.writeLock().acquire();
> > >        try
> > >        {
> > >         
> > mLogger.error("purging " + key);
> > >          mCache.remove(key);
> > >          mCache.remove(key +
> > ":");
> > >        }
> > >        catch (Exception e)
> > >        {
> > >         
> > mLogger.error("failed to purge key " + key, e);
> > >        }
> > >        try
> > >        {
> > >          String k =
> > (String)element.getKey();
> > >         
> > mLogger.error("purging " + k);
> > >          mCache.remove(k);
> > >          mCache.remove(k +
> > ":");
> > >        }
> > >        catch (Exception e)
> > >        {
> > >         
> > mLogger.error("failed to purge key " +
> element.getKey(),
> > e);
> > >        }
> > >      }
> > >      finally
> > >      {
> > >        if (hasReadLock)
> > >        {
> > >         
> > mLock.readLock().acquire();
> > >        }
> > >       
> > mLock.writeLock().release();
> > >      }
> > >    }
> > >    return corrupt;
> > >  }
> > >
> > >
> > >
> >
> ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: jcs-users-help@jakarta.apache.org
> > >
> > >
> > 
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: jcs-users-help@jakarta.apache.org
> > 
> >
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jcs-users-help@jakarta.apache.org
> 
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org


Mime
View raw message