jakarta-jcs-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Smuts <asm...@yahoo.com>
Subject RE: index cache corruption
Date Wed, 12 Aug 2009 14:41:16 GMT
The fix is in tempbuild 1.3.3.5-RC.

http://svn.apache.org/viewvc/jakarta/jcs/tempbuilds/

Please let me know if you see the problem again, or any other problems. 

We need to issue a new formal release. . . .

Cheers,

Aaron



--- On Wed, 8/12/09, Aaron Smuts <asmuts@yahoo.com> wrote:

> From: Aaron Smuts <asmuts@yahoo.com>
> Subject: RE: index cache corruption
> To: "JCS Users List" <jcs-users@jakarta.apache.org>
> Date: Wednesday, August 12, 2009, 7:33 AM
> Thanks for helping narrow down the
> problem . . .
> 
> I'm also considering putting in a check in the read
> method.  If the keys don't match, I'll return null and
> log an error. 
> 
> Aaron
> 
> --- On Wed, 8/12/09, Tim Cronin <Tim.Cronin@autonomy.com>
> wrote:
> 
> > From: Tim Cronin <Tim.Cronin@autonomy.com>
> > Subject: RE: index cache corruption
> > To: "JCS Users List" <jcs-users@jakarta.apache.org>
> > Date: Wednesday, August 12, 2009, 7:28 AM
> > Amen!
> > 
> > Thanks for the quick response to my issues.
> > 
> > 
> > -----Original Message-----
> > From: Josh Szmajda [mailto:joshsz@gmail.com]
> > 
> > Sent: Wednesday, August 12, 2009 9:17 AM
> > To: JCS Users List
> > Subject: Re: index cache corruption
> > 
> > Aaron I just wanted to say how thankful I am to have
> such a
> > committed
> > maintainer for JCS! Thanks for all your help, this
> project
> > is rock solid
> > because of you. :)
> > 
> > On Wed, Aug 12, 2009 at 10:11 AM, Aaron Smuts <asmuts@yahoo.com>
> > wrote:
> > 
> > > I think I found it.  During partial key
> removeall
> > the Disk Element
> > > Descriptor, which identifies the slot on disk,
> was
> > being added to the
> > > recylebin.  Then the removeSingleItem was
> called
> > for each match.  The
> > remove
> > > single item method also adds the item to the
> > recylebin.  The
> > recyclebin is
> > > not a set.  The spot would get entered
> > twice.  This is very bad.
> > Here's
> > > why:
> > >
> > > The spot gets used.  Another put uses the same
> > spot.  Both keys in the
> > key
> > > map point to the same spot.  A get for the
> first
> > key will pull the
> > data for
> > > the second.  Not so good.
> > >
> > > I'm verifying this and I'll have a fix soon.
> > >
> > > Aaron
> > >
> > > --- On Wed, 8/12/09, Aaron Smuts <asmuts@yahoo.com>
> > wrote:
> > >
> > > > From: Aaron Smuts <asmuts@yahoo.com>
> > > > Subject: RE: index cache corruption
> > > > To: "JCS Users List" <jcs-users@jakarta.apache.org>
> > > > Date: Wednesday, August 12, 2009, 6:52 AM
> > > >
> > > > So were you getting back the wrong data or
> data
> > that should
> > > > have been deleted?
> > > >
> > > > Are you running a recent temp build?
> > > >
> > > > I'll dig into the partial key removal. 
> This
> > should
> > > > probably deprecated for the remove matching
> > method. . .
> > > > .  Either way, it should work.
> > > >
> > > > Aaron
> > > >
> > > > --- On Tue, 8/11/09, Tim Cronin <Tim.Cronin@autonomy.com>
> > > > wrote:
> > > >
> > > > > From: Tim Cronin <Tim.Cronin@autonomy.com>
> > > > > Subject: RE: index cache corruption
> > > > > To: "JCS Users List" <jcs-users@jakarta.apache.org>
> > > > > Date: Tuesday, August 11, 2009, 2:30
> PM
> > > > > Good to know, thanks.
> > > > >
> > > > > Yes that's when we see it too, under
> high
> > load.
> > > > >
> > > > > For us it seems to happen around
> removes, we
> > use the
> > > > key:
> > > > > linkage and
> > > > > It happens to data that's linked and
> would
> > get cleared
> > > > with
> > > > > the key:
> > > > > call.
> > > > >
> > > > >
> > > > > -----Original Message-----
> > > > > From: Al Forbes [mailto:forbes.al@googlemail.com]
> > > > >
> > > > > Sent: Tuesday, August 11, 2009 4:13 PM
> > > > > To: JCS Users List
> > > > > Subject: Re: index cache corruption
> > > > >
> > > > > Hi Tim,
> > > > >
> > > > > I had the same problem. It seems to
> happen
> > under high
> > > > load,
> > > > > with fairly
> > > > > large objects, but I could not
> reproduce it
> > offline.
> > > > I
> > > > > changed to using
> > > > > a
> > > > > JDBC cache and have not had a problem
> since
> > then.
> > > > >
> > > > > We still use disk caches for more
> static
> > data - mostly
> > > > read
> > > > > only, and
> > > > > the
> > > > > problem never happens there, so I
> assume
> > it's the
> > > > writes
> > > > > that cause the
> > > > > corruption.
> > > > >
> > > > > Regards
> > > > > Al
> > > > >
> > > > > 2009/8/11 Tim Cronin <Tim.Cronin@autonomy.com>
> > > > >
> > > > > > We initially were using indexed
> disk
> > cache but
> > > > ran
> > > > > into cache
> > > > > corruption
> > > > > > where the data returned for a key
> would
> > not be
> > > > the
> > > > > data associated for
> > > > > > that key. I haven't been able to
> come
> > up with a
> > > > good
> > > > > repro scenario...
> > > > > >
> > > > > > We had to work around it with the
> > following
> > > > code:
> > > > > >
> > > > > >  /**
> > > > > >   * is the cache element
> > not the
> > > > > correct object for the key
> > > > > >   * @param key the
> > current key
> > > > > >   * @param element the
> > current element
> > > > > associated with the key
> > > > > >   * @param hasReadLock
> > whether caller
> > > > > has read lock
> > > > > >   * @return true if key
> > mismatch els
> > > > > false
> > > > > >   * @throws
> > InterruptedException if
> > > > > locking fails
> > > > > >   */
> > > > > >  private boolean
> isCorrupt(String
> > key,
> > > > > ICacheElement element, boolean
> > > > > > hasReadLock) throws
> > InterruptedException
> > > > > >  {
> > > > > >    boolean corrupt =
> > > > > !key.equals(element.getKey());
> > > > > >    if (corrupt)
> > > > > >    {
> > > > > >     
> > mLogger.error("cache corruption!!!
> > > > > [" + (mCorruptionCounter++) +
> > > > > > "]");
> > > > > >
> > > > > >      if
> > (mLogger.isDebugEnabled())
> > > > > >      {
> > > > > >   
> >    mLogger.error("culprit
> > > > > stack...", new Exception("cache
> > > > > > corruption"));
> > > > > >      }
> > > > > >
> > > > > >      try
> > > > > >      {
> > > > > >        if
> > (hasReadLock)
> > > > > >        {
> > > > > >
> > > > > mLock.readLock().release();
> > > > > >        }
> > > > > >
> > > > > mLock.writeLock().acquire();
> > > > > >        try
> > > > > >        {
> > > > > >
> > > > > mLogger.error("purging " + key);
> > > > > >         
> > mCache.remove(key);
> > > > > >         
> > mCache.remove(key +
> > > > > ":");
> > > > > >        }
> > > > > >        catch
> > (Exception e)
> > > > > >        {
> > > > > >
> > > > > mLogger.error("failed to purge key " +
> key,
> > e);
> > > > > >        }
> > > > > >        try
> > > > > >        {
> > > > > >         
> > String k =
> > > > > (String)element.getKey();
> > > > > >
> > > > > mLogger.error("purging " + k);
> > > > > >         
> > mCache.remove(k);
> > > > > >         
> > mCache.remove(k +
> > > > > ":");
> > > > > >        }
> > > > > >        catch
> > (Exception e)
> > > > > >        {
> > > > > >
> > > > > mLogger.error("failed to purge key " +
> > > > element.getKey(),
> > > > > e);
> > > > > >        }
> > > > > >      }
> > > > > >      finally
> > > > > >      {
> > > > > >        if
> > (hasReadLock)
> > > > > >        {
> > > > > >
> > > > > mLock.readLock().acquire();
> > > > > >        }
> > > > > >
> > > > > mLock.writeLock().release();
> > > > > >      }
> > > > > >    }
> > > > > >    return corrupt;
> > > > > >  }
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> >
> ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
> > > > > > For additional commands, e-mail:
> > jcs-users-help@jakarta.apache.org
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
> > > > > For additional commands, e-mail: jcs-users-help@jakarta.apache.org
> > > > >
> > > > >
> > > >
> > > >
> >
> ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
> > > > For additional commands, e-mail: jcs-users-help@jakarta.apache.org
> > > >
> > > >
> > >
> > >
> >
> ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: jcs-users-help@jakarta.apache.org
> > >
> > >
> > 
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: jcs-users-help@jakarta.apache.org
> > 
> >
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jcs-users-help@jakarta.apache.org
> 
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org


Mime
View raw message