commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis_gospodne...@yahoo.com>
Subject Re: [collections] LRUMap Problem ConcurrentModificationException
Date Tue, 16 Jun 2009 17:42:24 GMT

That's the one, René.  Yeah, no real solution in that JIRA issue I'm afraid. :(  But it shows
you what's already been looked at.

 Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



----- Original Message ----
> From: Renè Glanzer <rene.glanzer@gmail.com>
> To: Commons Users List <user@commons.apache.org>
> Sent: Tuesday, June 16, 2009 6:05:26 AM
> Subject: Re: [collections] LRUMap Problem ConcurrentModificationException
> 
> Hey Otis,
> 
> i found this one:
> https://issues.apache.org/jira/browse/COLLECTIONS-3
> 
> but there is no solutions only an assumption at the end of the thread??
> 
> René
> 
> 2009/6/15 Otis Gospodnetic :
> >
> > Btw. I think you'll find a report about this from a few years ago in the 
> Collections JIRA.  Just search for my name.
> >
> >  Otis
> > --
> > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
> >
> >
> >
> > ----- Original Message ----
> >> From: Renè Glanzer 
> >> To: Commons Users List 
> >> Sent: Monday, June 15, 2009 12:00:54 PM
> >> Subject: Re: [collections] LRUMap Problem ConcurrentModificationException
> >>
> >> Yes of course,
> >>
> >> the code with the time based cache systems was set up with an ordniary
> >> HashMap. But when rapidly seach requests are
> >> generated the time based mechanism fails to delete old entries - cause
> >> they are not old enough - and so the cache will
> >> raise up until the entire VM memory is used. Then i found the LRUMap
> >> switched to it and bang Exception in my delete method.
> >>
> >> When i switch back to HashMap the code runs well - as currently on the
> >> productive system - except the open memory leak.
> >>
> >> René
> >>
> >> 2009/6/15 Leon Rosenberg :
> >> > just out of curiosity have you tried the same code with a standard hashmap?
> >> >
> >> > Leon
> >> >
> >> > On Mon, Jun 15, 2009 at 4:37 PM, Renè Glanzerwrote:
> >> >> Hello,
> >> >>
> >> >> side note accepted :-)
> >> >>
> >> >> In my class I checked the get, put and remove methods. All are 
> synchronized.
> >> >> As you can see also the code which wants to delete the elements is
> >> synchronized.
> >> >> So I've no clue where the "ConcurrentModificationException" is comming

> from
> >> :-(
> >> >>
> >> >> Thanks René
> >> >>
> >> >> 2009/6/15 Leon Rosenberg :
> >> >>> Hello,
> >> >>>
> >> >>> on a side note, generics make reading of code easier :-)
> >> >>>
> >> >>> you haven't posted the whole code, but have you (double)checked
that
> >> >>> all other acesses to store are synchronized?
> >> >>>
> >> >>> regards
> >> >>> Leon
> >> >>>
> >> >>> On Mon, Jun 15, 2009 at 2:31 PM, Renè Glanzerwrote:
> >> >>>> I'm calling the remove() method on the iterator. I know when
i call
> >> >>>> the remove on the map itself it will cause the problem with
my
> >> >>>> currently running iterator, but i'm aware of that.
> >> >>>>
> >> >>>> Here is the code block which should delete old entries:
> >> >>>>
> >> >>>> store: is the LRUMap
> >> >>>> birth: is a long which keeps the creation time when this is
set to 0
> >> >>>> the item should be deleted
> >> >>>>
> >> >>>> public void removeAllExpiredItems()
> >> >>>>  {
> >> >>>>   synchronized(this.store)
> >> >>>>   {
> >> >>>>     Iterator it=this.store.keySet().iterator();
> >> >>>>     while(it.hasNext())
> >> >>>>     {
> >> >>>>       Object key=it.next();
> >> >>>>       Object o=this.get(key);
> >> >>>>       if(o != null)
> >> >>>>       {
> >> >>>>         Item iEntry=(Item)this.store.get(key);
> >> >>>>         if(iEntry.birth==0)
> >> >>>>           it.remove(); //Here the exception occurs
> >> >>>>       }
> >> >>>>     }
> >> >>>>     this.setLastCleanDate(new Date()); //only to know when
the
> >> >>>> deleter run the last time
> >> >>>>   }
> >> >>>>  }
> >> >>>>
> >> >>>> Thanks for any help :-)
> >> >>>> René
> >> >>>>
> >> >>>> 2009/6/15 James Carman :
> >> >>>>> Are you calling remove() on the iterator or on the map
itself?
> >> >>>>>
> >> >>>>> On Mon, Jun 15, 2009 at 6:37 AM, Renè Glanzer
> >> wrote:
> >> >>>>>> Hello,
> >> >>>>>>
> >> >>>>>> is there still no help for me?
> >> >>>>>> Is somebody able to explain me, why i get this
> >> >>>>>> "java.util.ConcurrentModificationException"
> >> >>>>>> on iterating and calling remove() on the LRUMap?
> >> >>>>>>
> >> >>>>>> Please
> >> >>>>>> René
> >> >>>>>>
> >> >>>>>> 2009/6/10 Renè Glanzer :
> >> >>>>>>> Hello Ted,
> >> >>>>>>>
> >> >>>>>>> thanks for the fast response. I understand that
simultaneously puts
> >> >>>>>>> and gets do not cause the exception cause they
are synchronized in my
> >> >>>>>>> class.
> >> >>>>>>> And my stated code-fragment is the part which wants
to delete the
> >> >>>>>>> entries from the underlying LRUMap. And my code
is wrapped in the
> >> >>>>>>> synchronized block. But i think the LRUMap herselfe
also iterates the
> >> >>>>>>> entries and this maybe happens at the same time
when my iterator is
> >> >>>>>>> checking and trying to delete elements. My class
is not implementing
> >> >>>>>>> the LRUMap but has one LRUMap as a variable.
> >> >>>>>>> So your solution to override the removeLRU(Entry)
methode would not
> >> >>>>>>> help me....unfortunatelly :-(
> >> >>>>>>>
> >> >>>>>>> For now i really do not know how to solve the problem
in an easy way
> >> >>>>>>> without refractoring the entire code?
> >> >>>>>>> Any other solutions?
> >> >>>>>>> Thanks René
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> 2009/6/9 Ted Dunning :
> >> >>>>>>>> I apologize for not reading your thread with
great care, but I think
> >> that
> >> >>>>>>>> your problem is pretty clear.
> >> >>>>>>>>
> >> >>>>>>>> The issue is not to do with gets and puts overlapping.
 The issue is
> >> that a
> >> >>>>>>>> put or remove happened during the life of your
iterator.
> >> >>>>>>>>
> >> >>>>>>>> One way to avoid this is to synchronize the
entire method that does 
> the
> >> >>>>>>>> deletion of old elements.  To speed that up,
you might just 
> synchronize
> >> the
> >> >>>>>>>> scan for elements to delete and collect them
into a list.  Then you 
> can
> >> >>>>>>>> delete them outside the loop.  Since your scan
is probably pretty 
> fast,
> >> this
> >> >>>>>>>> may be sufficient.  With very high levels of
updates and threading, 
> it
> >> would
> >> >>>>>>>> cause problems.
> >> >>>>>>>>
> >> >>>>>>>> Another option is to clone the table.  I believe
that some
> >> implementations
> >> >>>>>>>> of LRUMap in Lucene do copy-on-write semantics,
but I don't think 
> that
> >> >>>>>>>> collections has these.  Without that, cloning
will be as slow or 
> slower
> >> than
> >> >>>>>>>> your scan so the first option would be better.
> >> >>>>>>>>
> >> >>>>>>>> I am curious, however, why you didn't make
use of the built-in
> >> capabilities
> >> >>>>>>>> of the LRUMap to help you with this.  Notably,
you should probably 
> just
> >> >>>>>>>> over-ride the removeLRU(Entry) method and set
the scanUntilRemovable
> >> flag.
> >> >>>>>>>> I think that this would take the place of your
loop and would be 
> much
> >> more
> >> >>>>>>>> efficient.
> >> >>>>>>>>
> >> >>>>>>>> On Tue, Jun 9, 2009 at 9:33 AM, Renè Glanzer
> >> wrote:
> >> >>>>>>>>
> >> >>>>>>>>> ... Additionally to the maximum number
of the LRUMap i also
> >> implemented a
> >> >>>>>>>>> thread which periodicly checks the age
of the entries and deletes 
> them
> >> >>>>>>>>> when they are too old.
> >> >>>>>>>>>
> >> >>>>>>>>> For this i generated an Iterator which
delivers me each entry and 
> if
> >> >>>>>>>>> the creation date is too old i call iterator.remove().
> >> >>>>>>>>> But exactly on this line i get an
> >> >>>>>>>>> "java.util.ConcurrentModificationException"
although i wrapped all
> >> >>>>>>>>> access to the map with synchronized blocks.
So every put and get
> >> >>>>>>>>> methods are synchronized.
> >> >>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>
> >> >>>>>>
> >> >>>>>> ---------------------------------------------------------------------
> >> >>>>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> >> >>>>>> For additional commands, e-mail: user-help@commons.apache.org
> >> >>>>>>
> >> >>>>>>
> >> >>>>>
> >> >>>>> ---------------------------------------------------------------------
> >> >>>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> >> >>>>> For additional commands, e-mail: user-help@commons.apache.org
> >> >>>>>
> >> >>>>>
> >> >>>>
> >> >>>> ---------------------------------------------------------------------
> >> >>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> >> >>>> For additional commands, e-mail: user-help@commons.apache.org
> >> >>>>
> >> >>>>
> >> >>>
> >> >>> ---------------------------------------------------------------------
> >> >>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> >> >>> For additional commands, e-mail: user-help@commons.apache.org
> >> >>>
> >> >>>
> >> >>
> >> >> ---------------------------------------------------------------------
> >> >> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> >> >> For additional commands, e-mail: user-help@commons.apache.org
> >> >>
> >> >>
> >> >
> >> > ---------------------------------------------------------------------
> >> > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> >> > For additional commands, e-mail: user-help@commons.apache.org
> >> >
> >> >
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> >> For additional commands, e-mail: user-help@commons.apache.org
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > For additional commands, e-mail: user-help@commons.apache.org
> >
> >
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org


Mime
View raw message