commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis_gospodne...@yahoo.com>
Subject Re: [collections] LRUMap Problem ConcurrentModificationException
Date Mon, 15 Jun 2009 18:38:17 GMT

Btw. I think you'll find a report about this from a few years ago in the Collections JIRA.
 Just search for my name.

 Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



----- Original Message ----
> From: Renè Glanzer <rene.glanzer@gmail.com>
> To: Commons Users List <user@commons.apache.org>
> Sent: Monday, June 15, 2009 12:00:54 PM
> Subject: Re: [collections] LRUMap Problem ConcurrentModificationException
> 
> Yes of course,
> 
> the code with the time based cache systems was set up with an ordniary
> HashMap. But when rapidly seach requests are
> generated the time based mechanism fails to delete old entries - cause
> they are not old enough - and so the cache will
> raise up until the entire VM memory is used. Then i found the LRUMap
> switched to it and bang Exception in my delete method.
> 
> When i switch back to HashMap the code runs well - as currently on the
> productive system - except the open memory leak.
> 
> René
> 
> 2009/6/15 Leon Rosenberg :
> > just out of curiosity have you tried the same code with a standard hashmap?
> >
> > Leon
> >
> > On Mon, Jun 15, 2009 at 4:37 PM, Renè Glanzerwrote:
> >> Hello,
> >>
> >> side note accepted :-)
> >>
> >> In my class I checked the get, put and remove methods. All are synchronized.
> >> As you can see also the code which wants to delete the elements is 
> synchronized.
> >> So I've no clue where the "ConcurrentModificationException" is comming from

> :-(
> >>
> >> Thanks René
> >>
> >> 2009/6/15 Leon Rosenberg :
> >>> Hello,
> >>>
> >>> on a side note, generics make reading of code easier :-)
> >>>
> >>> you haven't posted the whole code, but have you (double)checked that
> >>> all other acesses to store are synchronized?
> >>>
> >>> regards
> >>> Leon
> >>>
> >>> On Mon, Jun 15, 2009 at 2:31 PM, Renè Glanzerwrote:
> >>>> I'm calling the remove() method on the iterator. I know when i call
> >>>> the remove on the map itself it will cause the problem with my
> >>>> currently running iterator, but i'm aware of that.
> >>>>
> >>>> Here is the code block which should delete old entries:
> >>>>
> >>>> store: is the LRUMap
> >>>> birth: is a long which keeps the creation time when this is set to 0
> >>>> the item should be deleted
> >>>>
> >>>> public void removeAllExpiredItems()
> >>>>  {
> >>>>   synchronized(this.store)
> >>>>   {
> >>>>     Iterator it=this.store.keySet().iterator();
> >>>>     while(it.hasNext())
> >>>>     {
> >>>>       Object key=it.next();
> >>>>       Object o=this.get(key);
> >>>>       if(o != null)
> >>>>       {
> >>>>         Item iEntry=(Item)this.store.get(key);
> >>>>         if(iEntry.birth==0)
> >>>>           it.remove(); //Here the exception occurs
> >>>>       }
> >>>>     }
> >>>>     this.setLastCleanDate(new Date()); //only to know when the
> >>>> deleter run the last time
> >>>>   }
> >>>>  }
> >>>>
> >>>> Thanks for any help :-)
> >>>> René
> >>>>
> >>>> 2009/6/15 James Carman :
> >>>>> Are you calling remove() on the iterator or on the map itself?
> >>>>>
> >>>>> On Mon, Jun 15, 2009 at 6:37 AM, Renè Glanzer
> wrote:
> >>>>>> Hello,
> >>>>>>
> >>>>>> is there still no help for me?
> >>>>>> Is somebody able to explain me, why i get this
> >>>>>> "java.util.ConcurrentModificationException"
> >>>>>> on iterating and calling remove() on the LRUMap?
> >>>>>>
> >>>>>> Please
> >>>>>> René
> >>>>>>
> >>>>>> 2009/6/10 Renè Glanzer :
> >>>>>>> Hello Ted,
> >>>>>>>
> >>>>>>> thanks for the fast response. I understand that simultaneously
puts
> >>>>>>> and gets do not cause the exception cause they are synchronized
in my
> >>>>>>> class.
> >>>>>>> And my stated code-fragment is the part which wants to delete
the
> >>>>>>> entries from the underlying LRUMap. And my code is wrapped
in the
> >>>>>>> synchronized block. But i think the LRUMap herselfe also
iterates the
> >>>>>>> entries and this maybe happens at the same time when my
iterator is
> >>>>>>> checking and trying to delete elements. My class is not
implementing
> >>>>>>> the LRUMap but has one LRUMap as a variable.
> >>>>>>> So your solution to override the removeLRU(Entry) methode
would not
> >>>>>>> help me....unfortunatelly :-(
> >>>>>>>
> >>>>>>> For now i really do not know how to solve the problem in
an easy way
> >>>>>>> without refractoring the entire code?
> >>>>>>> Any other solutions?
> >>>>>>> Thanks René
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> 2009/6/9 Ted Dunning :
> >>>>>>>> I apologize for not reading your thread with great care,
but I think 
> that
> >>>>>>>> your problem is pretty clear.
> >>>>>>>>
> >>>>>>>> The issue is not to do with gets and puts overlapping.
 The issue is 
> that a
> >>>>>>>> put or remove happened during the life of your iterator.
> >>>>>>>>
> >>>>>>>> One way to avoid this is to synchronize the entire method
that does the
> >>>>>>>> deletion of old elements.  To speed that up, you might
just synchronize 
> the
> >>>>>>>> scan for elements to delete and collect them into a
list.  Then you can
> >>>>>>>> delete them outside the loop.  Since your scan is probably
pretty fast, 
> this
> >>>>>>>> may be sufficient.  With very high levels of updates
and threading, it 
> would
> >>>>>>>> cause problems.
> >>>>>>>>
> >>>>>>>> Another option is to clone the table.  I believe that
some 
> implementations
> >>>>>>>> of LRUMap in Lucene do copy-on-write semantics, but
I don't think that
> >>>>>>>> collections has these.  Without that, cloning will be
as slow or slower 
> than
> >>>>>>>> your scan so the first option would be better.
> >>>>>>>>
> >>>>>>>> I am curious, however, why you didn't make use of the
built-in 
> capabilities
> >>>>>>>> of the LRUMap to help you with this.  Notably, you should
probably just
> >>>>>>>> over-ride the removeLRU(Entry) method and set the scanUntilRemovable

> flag.
> >>>>>>>> I think that this would take the place of your loop
and would be much 
> more
> >>>>>>>> efficient.
> >>>>>>>>
> >>>>>>>> On Tue, Jun 9, 2009 at 9:33 AM, Renè Glanzer 
> wrote:
> >>>>>>>>
> >>>>>>>>> ... Additionally to the maximum number of the LRUMap
i also 
> implemented a
> >>>>>>>>> thread which periodicly checks the age of the entries
and deletes them
> >>>>>>>>> when they are too old.
> >>>>>>>>>
> >>>>>>>>> For this i generated an Iterator which delivers
me each entry and if
> >>>>>>>>> the creation date is too old i call iterator.remove().
> >>>>>>>>> But exactly on this line i get an
> >>>>>>>>> "java.util.ConcurrentModificationException" although
i wrapped all
> >>>>>>>>> access to the map with synchronized blocks. So every
put and get
> >>>>>>>>> methods are synchronized.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>> ---------------------------------------------------------------------
> >>>>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> >>>>>> For additional commands, e-mail: user-help@commons.apache.org
> >>>>>>
> >>>>>>
> >>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> >>>>> For additional commands, e-mail: user-help@commons.apache.org
> >>>>>
> >>>>>
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> >>>> For additional commands, e-mail: user-help@commons.apache.org
> >>>>
> >>>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> >>> For additional commands, e-mail: user-help@commons.apache.org
> >>>
> >>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> >> For additional commands, e-mail: user-help@commons.apache.org
> >>
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > For additional commands, e-mail: user-help@commons.apache.org
> >
> >
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org


Mime
View raw message