jakarta-jcs-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Horton Simon" <Simon.Hor...@uk.mizuho-sc.com>
Subject RE: index cache corruption
Date Wed, 12 Aug 2009 14:22:26 GMT
I'd like to second that, well done Aaron for your hard work and
commitment in producing a brilliant cache! The same goes to the other
JCS developers to.

Thanks,
Simon


-----Original Message-----
From: Josh Szmajda [mailto:joshsz@gmail.com] 
Sent: 12 August 2009 15:17
To: JCS Users List
Subject: Re: index cache corruption

Aaron I just wanted to say how thankful I am to have such a committed
maintainer for JCS! Thanks for all your help, this project is rock solid
because of you. :)

On Wed, Aug 12, 2009 at 10:11 AM, Aaron Smuts <asmuts@yahoo.com> wrote:

> I think I found it.  During partial key removeall the Disk Element
> Descriptor, which identifies the slot on disk, was being added to the
> recylebin.  Then the removeSingleItem was called for each match.  The
remove
> single item method also adds the item to the recylebin.  The
recyclebin is
> not a set.  The spot would get entered twice.  This is very bad.
Here's
> why:
>
> The spot gets used.  Another put uses the same spot.  Both keys in the
key
> map point to the same spot.  A get for the first key will pull the
data for
> the second.  Not so good.
>
> I'm verifying this and I'll have a fix soon.
>
> Aaron
>
> --- On Wed, 8/12/09, Aaron Smuts <asmuts@yahoo.com> wrote:
>
> > From: Aaron Smuts <asmuts@yahoo.com>
> > Subject: RE: index cache corruption
> > To: "JCS Users List" <jcs-users@jakarta.apache.org>
> > Date: Wednesday, August 12, 2009, 6:52 AM
> >
> > So were you getting back the wrong data or data that should
> > have been deleted?
> >
> > Are you running a recent temp build?
> >
> > I'll dig into the partial key removal.  This should
> > probably deprecated for the remove matching method. . .
> > .  Either way, it should work.
> >
> > Aaron
> >
> > --- On Tue, 8/11/09, Tim Cronin <Tim.Cronin@autonomy.com>
> > wrote:
> >
> > > From: Tim Cronin <Tim.Cronin@autonomy.com>
> > > Subject: RE: index cache corruption
> > > To: "JCS Users List" <jcs-users@jakarta.apache.org>
> > > Date: Tuesday, August 11, 2009, 2:30 PM
> > > Good to know, thanks.
> > >
> > > Yes that's when we see it too, under high load.
> > >
> > > For us it seems to happen around removes, we use the
> > key:
> > > linkage and
> > > It happens to data that's linked and would get cleared
> > with
> > > the key:
> > > call.
> > >
> > >
> > > -----Original Message-----
> > > From: Al Forbes [mailto:forbes.al@googlemail.com]
> > >
> > > Sent: Tuesday, August 11, 2009 4:13 PM
> > > To: JCS Users List
> > > Subject: Re: index cache corruption
> > >
> > > Hi Tim,
> > >
> > > I had the same problem. It seems to happen under high
> > load,
> > > with fairly
> > > large objects, but I could not reproduce it offline.
> > I
> > > changed to using
> > > a
> > > JDBC cache and have not had a problem since then.
> > >
> > > We still use disk caches for more static data - mostly
> > read
> > > only, and
> > > the
> > > problem never happens there, so I assume it's the
> > writes
> > > that cause the
> > > corruption.
> > >
> > > Regards
> > > Al
> > >
> > > 2009/8/11 Tim Cronin <Tim.Cronin@autonomy.com>
> > >
> > > > We initially were using indexed disk cache but
> > ran
> > > into cache
> > > corruption
> > > > where the data returned for a key would not be
> > the
> > > data associated for
> > > > that key. I haven't been able to come up with a
> > good
> > > repro scenario...
> > > >
> > > > We had to work around it with the following
> > code:
> > > >
> > > >  /**
> > > >   * is the cache element not the
> > > correct object for the key
> > > >   * @param key the current key
> > > >   * @param element the current element
> > > associated with the key
> > > >   * @param hasReadLock whether caller
> > > has read lock
> > > >   * @return true if key mismatch els
> > > false
> > > >   * @throws InterruptedException if
> > > locking fails
> > > >   */
> > > >  private boolean isCorrupt(String key,
> > > ICacheElement element, boolean
> > > > hasReadLock) throws InterruptedException
> > > >  {
> > > >    boolean corrupt =
> > > !key.equals(element.getKey());
> > > >    if (corrupt)
> > > >    {
> > > >      mLogger.error("cache corruption!!!
> > > [" + (mCorruptionCounter++) +
> > > > "]");
> > > >
> > > >      if (mLogger.isDebugEnabled())
> > > >      {
> > > >       mLogger.error("culprit
> > > stack...", new Exception("cache
> > > > corruption"));
> > > >      }
> > > >
> > > >      try
> > > >      {
> > > >        if (hasReadLock)
> > > >        {
> > > >
> > > mLock.readLock().release();
> > > >        }
> > > >
> > > mLock.writeLock().acquire();
> > > >        try
> > > >        {
> > > >
> > > mLogger.error("purging " + key);
> > > >          mCache.remove(key);
> > > >          mCache.remove(key +
> > > ":");
> > > >        }
> > > >        catch (Exception e)
> > > >        {
> > > >
> > > mLogger.error("failed to purge key " + key, e);
> > > >        }
> > > >        try
> > > >        {
> > > >          String k =
> > > (String)element.getKey();
> > > >
> > > mLogger.error("purging " + k);
> > > >          mCache.remove(k);
> > > >          mCache.remove(k +
> > > ":");
> > > >        }
> > > >        catch (Exception e)
> > > >        {
> > > >
> > > mLogger.error("failed to purge key " +
> > element.getKey(),
> > > e);
> > > >        }
> > > >      }
> > > >      finally
> > > >      {
> > > >        if (hasReadLock)
> > > >        {
> > > >
> > > mLock.readLock().acquire();
> > > >        }
> > > >
> > > mLock.writeLock().release();
> > > >      }
> > > >    }
> > > >    return corrupt;
> > > >  }
> > > >
> > > >
> > > >
> > >
> >
---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
> > > > For additional commands, e-mail:
jcs-users-help@jakarta.apache.org
> > > >
> > > >
> > >
> > >
> >
---------------------------------------------------------------------
> > > To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: jcs-users-help@jakarta.apache.org
> > >
> > >
> >
> >
---------------------------------------------------------------------
> > To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: jcs-users-help@jakarta.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jcs-users-help@jakarta.apache.org
>
>

This message and any files transmitted with it are confidential and intended solely for the
use of the individual or entity to whom they are addressed. If you have received this message
in error please delete it and any files transmitted with it, after notifying postmaster@uk.mizuho-sc.com

Any opinions expressed in this message may be those of the author and not necessarily those
of the company. The company accepts no responsibility for the accuracy or completeness of
any information contained herein. This message is not intended to create legal relations between
the company and the recipient. 
Recipients should please note that messages sent via the Internet may be intercepted and that
caution should therefore be exercised before dispatching to the company any confidential or
sensitive information. 
Mizuho International plc Bracken House, One Friday Street, London EC4M 9JA. TEL. 020 72361090.
Wholly owned subsidiary of Mizuho Securities Co., Ltd. Member of Mizuho Financial Group. Authorised
and regulated by the Financial Services Authority. Member of the London Stock Exchange. 

Registered in England No. 1203696. Registered office as above.



---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org


Mime
View raw message