commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Renè Glanzer <rene.glan...@gmail.com>
Subject Re: [collections] LRUMap Problem ConcurrentModificationException
Date Wed, 17 Jun 2009 07:47:46 GMT
OK so my search will continue :-)
Meanwhile I'll consider to change my implementation, which i'd like to
prevent.

Maybe somebody of you knows a time and size based cache system where i can
map a key to an object?

René


2009/6/16 Otis Gospodnetic <otis_gospodnetic@yahoo.com>

>
> That's the one, René.  Yeah, no real solution in that JIRA issue I'm
> afraid. :(  But it shows you what's already been looked at.
>
>  Otis
> --
> Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
>
>
>
> ----- Original Message ----
> > From: Renè Glanzer <rene.glanzer@gmail.com>
> > To: Commons Users List <user@commons.apache.org>
> > Sent: Tuesday, June 16, 2009 6:05:26 AM
> > Subject: Re: [collections] LRUMap Problem ConcurrentModificationException
> >
> > Hey Otis,
> >
> > i found this one:
> > https://issues.apache.org/jira/browse/COLLECTIONS-3
> >
> > but there is no solutions only an assumption at the end of the thread??
> >
> > René
> >
> > 2009/6/15 Otis Gospodnetic :
> > >
> > > Btw. I think you'll find a report about this from a few years ago in
> the
> > Collections JIRA.  Just search for my name.
> > >
> > >  Otis
> > > --
> > > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
> > >
> > >
> > >
> > > ----- Original Message ----
> > >> From: Renè Glanzer
> > >> To: Commons Users List
> > >> Sent: Monday, June 15, 2009 12:00:54 PM
> > >> Subject: Re: [collections] LRUMap Problem
> ConcurrentModificationException
> > >>
> > >> Yes of course,
> > >>
> > >> the code with the time based cache systems was set up with an ordniary
> > >> HashMap. But when rapidly seach requests are
> > >> generated the time based mechanism fails to delete old entries - cause
> > >> they are not old enough - and so the cache will
> > >> raise up until the entire VM memory is used. Then i found the LRUMap
> > >> switched to it and bang Exception in my delete method.
> > >>
> > >> When i switch back to HashMap the code runs well - as currently on the
> > >> productive system - except the open memory leak.
> > >>
> > >> René
> > >>
> > >> 2009/6/15 Leon Rosenberg :
> > >> > just out of curiosity have you tried the same code with a standard
> hashmap?
> > >> >
> > >> > Leon
> > >> >
> > >> > On Mon, Jun 15, 2009 at 4:37 PM, Renè Glanzerwrote:
> > >> >> Hello,
> > >> >>
> > >> >> side note accepted :-)
> > >> >>
> > >> >> In my class I checked the get, put and remove methods. All are
> > synchronized.
> > >> >> As you can see also the code which wants to delete the elements
is
> > >> synchronized.
> > >> >> So I've no clue where the "ConcurrentModificationException" is
> comming
> > from
> > >> :-(
> > >> >>
> > >> >> Thanks René
> > >> >>
> > >> >> 2009/6/15 Leon Rosenberg :
> > >> >>> Hello,
> > >> >>>
> > >> >>> on a side note, generics make reading of code easier :-)
> > >> >>>
> > >> >>> you haven't posted the whole code, but have you (double)checked
> that
> > >> >>> all other acesses to store are synchronized?
> > >> >>>
> > >> >>> regards
> > >> >>> Leon
> > >> >>>
> > >> >>> On Mon, Jun 15, 2009 at 2:31 PM, Renè Glanzerwrote:
> > >> >>>> I'm calling the remove() method on the iterator. I know
when i
> call
> > >> >>>> the remove on the map itself it will cause the problem
with my
> > >> >>>> currently running iterator, but i'm aware of that.
> > >> >>>>
> > >> >>>> Here is the code block which should delete old entries:
> > >> >>>>
> > >> >>>> store: is the LRUMap
> > >> >>>> birth: is a long which keeps the creation time when this
is set
> to 0
> > >> >>>> the item should be deleted
> > >> >>>>
> > >> >>>> public void removeAllExpiredItems()
> > >> >>>>  {
> > >> >>>>   synchronized(this.store)
> > >> >>>>   {
> > >> >>>>     Iterator it=this.store.keySet().iterator();
> > >> >>>>     while(it.hasNext())
> > >> >>>>     {
> > >> >>>>       Object key=it.next();
> > >> >>>>       Object o=this.get(key);
> > >> >>>>       if(o != null)
> > >> >>>>       {
> > >> >>>>         Item iEntry=(Item)this.store.get(key);
> > >> >>>>         if(iEntry.birth==0)
> > >> >>>>           it.remove(); //Here the exception occurs
> > >> >>>>       }
> > >> >>>>     }
> > >> >>>>     this.setLastCleanDate(new Date()); //only to know
when the
> > >> >>>> deleter run the last time
> > >> >>>>   }
> > >> >>>>  }
> > >> >>>>
> > >> >>>> Thanks for any help :-)
> > >> >>>> René
> > >> >>>>
> > >> >>>> 2009/6/15 James Carman :
> > >> >>>>> Are you calling remove() on the iterator or on the
map itself?
> > >> >>>>>
> > >> >>>>> On Mon, Jun 15, 2009 at 6:37 AM, Renè Glanzer
> > >> wrote:
> > >> >>>>>> Hello,
> > >> >>>>>>
> > >> >>>>>> is there still no help for me?
> > >> >>>>>> Is somebody able to explain me, why i get this
> > >> >>>>>> "java.util.ConcurrentModificationException"
> > >> >>>>>> on iterating and calling remove() on the LRUMap?
> > >> >>>>>>
> > >> >>>>>> Please
> > >> >>>>>> René
> > >> >>>>>>
> > >> >>>>>> 2009/6/10 Renè Glanzer :
> > >> >>>>>>> Hello Ted,
> > >> >>>>>>>
> > >> >>>>>>> thanks for the fast response. I understand
that simultaneously
> puts
> > >> >>>>>>> and gets do not cause the exception cause
they are
> synchronized in my
> > >> >>>>>>> class.
> > >> >>>>>>> And my stated code-fragment is the part which
wants to delete
> the
> > >> >>>>>>> entries from the underlying LRUMap. And my
code is wrapped in
> the
> > >> >>>>>>> synchronized block. But i think the LRUMap
herselfe also
> iterates the
> > >> >>>>>>> entries and this maybe happens at the same
time when my
> iterator is
> > >> >>>>>>> checking and trying to delete elements. My
class is not
> implementing
> > >> >>>>>>> the LRUMap but has one LRUMap as a variable.
> > >> >>>>>>> So your solution to override the removeLRU(Entry)
methode
> would not
> > >> >>>>>>> help me....unfortunatelly :-(
> > >> >>>>>>>
> > >> >>>>>>> For now i really do not know how to solve
the problem in an
> easy way
> > >> >>>>>>> without refractoring the entire code?
> > >> >>>>>>> Any other solutions?
> > >> >>>>>>> Thanks René
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> 2009/6/9 Ted Dunning :
> > >> >>>>>>>> I apologize for not reading your thread
with great care, but
> I think
> > >> that
> > >> >>>>>>>> your problem is pretty clear.
> > >> >>>>>>>>
> > >> >>>>>>>> The issue is not to do with gets and puts
overlapping.  The
> issue is
> > >> that a
> > >> >>>>>>>> put or remove happened during the life
of your iterator.
> > >> >>>>>>>>
> > >> >>>>>>>> One way to avoid this is to synchronize
the entire method
> that does
> > the
> > >> >>>>>>>> deletion of old elements.  To speed that
up, you might just
> > synchronize
> > >> the
> > >> >>>>>>>> scan for elements to delete and collect
them into a list.
>  Then you
> > can
> > >> >>>>>>>> delete them outside the loop.  Since your
scan is probably
> pretty
> > fast,
> > >> this
> > >> >>>>>>>> may be sufficient.  With very high levels
of updates and
> threading,
> > it
> > >> would
> > >> >>>>>>>> cause problems.
> > >> >>>>>>>>
> > >> >>>>>>>> Another option is to clone the table.
 I believe that some
> > >> implementations
> > >> >>>>>>>> of LRUMap in Lucene do copy-on-write semantics,
but I don't
> think
> > that
> > >> >>>>>>>> collections has these.  Without that,
cloning will be as slow
> or
> > slower
> > >> than
> > >> >>>>>>>> your scan so the first option would be
better.
> > >> >>>>>>>>
> > >> >>>>>>>> I am curious, however, why you didn't
make use of the
> built-in
> > >> capabilities
> > >> >>>>>>>> of the LRUMap to help you with this. 
Notably, you should
> probably
> > just
> > >> >>>>>>>> over-ride the removeLRU(Entry) method
and set the
> scanUntilRemovable
> > >> flag.
> > >> >>>>>>>> I think that this would take the place
of your loop and would
> be
> > much
> > >> more
> > >> >>>>>>>> efficient.
> > >> >>>>>>>>
> > >> >>>>>>>> On Tue, Jun 9, 2009 at 9:33 AM, Renè
Glanzer
> > >> wrote:
> > >> >>>>>>>>
> > >> >>>>>>>>> ... Additionally to the maximum number
of the LRUMap i also
> > >> implemented a
> > >> >>>>>>>>> thread which periodicly checks the
age of the entries and
> deletes
> > them
> > >> >>>>>>>>> when they are too old.
> > >> >>>>>>>>>
> > >> >>>>>>>>> For this i generated an Iterator which
delivers me each
> entry and
> > if
> > >> >>>>>>>>> the creation date is too old i call
iterator.remove().
> > >> >>>>>>>>> But exactly on this line i get an
> > >> >>>>>>>>> "java.util.ConcurrentModificationException"
although i
> wrapped all
> > >> >>>>>>>>> access to the map with synchronized
blocks. So every put and
> get
> > >> >>>>>>>>> methods are synchronized.
> > >> >>>>>>>>>
> > >> >>>>>>>>>
> > >> >>>>>>>>
> > >> >>>>>>>
> > >> >>>>>>
> > >> >>>>>>
> ---------------------------------------------------------------------
> > >> >>>>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > >> >>>>>> For additional commands, e-mail: user-help@commons.apache.org
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>>>
> > >> >>>>>
> ---------------------------------------------------------------------
> > >> >>>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > >> >>>>> For additional commands, e-mail: user-help@commons.apache.org
> > >> >>>>>
> > >> >>>>>
> > >> >>>>
> > >> >>>>
> ---------------------------------------------------------------------
> > >> >>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > >> >>>> For additional commands, e-mail: user-help@commons.apache.org
> > >> >>>>
> > >> >>>>
> > >> >>>
> > >> >>>
> ---------------------------------------------------------------------
> > >> >>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > >> >>> For additional commands, e-mail: user-help@commons.apache.org
> > >> >>>
> > >> >>>
> > >> >>
> > >> >>
> ---------------------------------------------------------------------
> > >> >> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > >> >> For additional commands, e-mail: user-help@commons.apache.org
> > >> >>
> > >> >>
> > >> >
> > >> >
> ---------------------------------------------------------------------
> > >> > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > >> > For additional commands, e-mail: user-help@commons.apache.org
> > >> >
> > >> >
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > >> For additional commands, e-mail: user-help@commons.apache.org
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > > For additional commands, e-mail: user-help@commons.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > For additional commands, e-mail: user-help@commons.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message