commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Renè Glanzer <rene.glan...@gmail.com>
Subject Re: [collections] LRUMap Problem ConcurrentModificationException
Date Tue, 16 Jun 2009 10:05:26 GMT
Hey Otis,

i found this one:
https://issues.apache.org/jira/browse/COLLECTIONS-3

but there is no solutions only an assumption at the end of the thread??

René

2009/6/15 Otis Gospodnetic <otis_gospodnetic@yahoo.com>:
>
> Btw. I think you'll find a report about this from a few years ago in the Collections
JIRA.  Just search for my name.
>
>  Otis
> --
> Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
>
>
>
> ----- Original Message ----
>> From: Renè Glanzer <rene.glanzer@gmail.com>
>> To: Commons Users List <user@commons.apache.org>
>> Sent: Monday, June 15, 2009 12:00:54 PM
>> Subject: Re: [collections] LRUMap Problem ConcurrentModificationException
>>
>> Yes of course,
>>
>> the code with the time based cache systems was set up with an ordniary
>> HashMap. But when rapidly seach requests are
>> generated the time based mechanism fails to delete old entries - cause
>> they are not old enough - and so the cache will
>> raise up until the entire VM memory is used. Then i found the LRUMap
>> switched to it and bang Exception in my delete method.
>>
>> When i switch back to HashMap the code runs well - as currently on the
>> productive system - except the open memory leak.
>>
>> René
>>
>> 2009/6/15 Leon Rosenberg :
>> > just out of curiosity have you tried the same code with a standard hashmap?
>> >
>> > Leon
>> >
>> > On Mon, Jun 15, 2009 at 4:37 PM, Renè Glanzerwrote:
>> >> Hello,
>> >>
>> >> side note accepted :-)
>> >>
>> >> In my class I checked the get, put and remove methods. All are synchronized.
>> >> As you can see also the code which wants to delete the elements is
>> synchronized.
>> >> So I've no clue where the "ConcurrentModificationException" is comming from
>> :-(
>> >>
>> >> Thanks René
>> >>
>> >> 2009/6/15 Leon Rosenberg :
>> >>> Hello,
>> >>>
>> >>> on a side note, generics make reading of code easier :-)
>> >>>
>> >>> you haven't posted the whole code, but have you (double)checked that
>> >>> all other acesses to store are synchronized?
>> >>>
>> >>> regards
>> >>> Leon
>> >>>
>> >>> On Mon, Jun 15, 2009 at 2:31 PM, Renè Glanzerwrote:
>> >>>> I'm calling the remove() method on the iterator. I know when i call
>> >>>> the remove on the map itself it will cause the problem with my
>> >>>> currently running iterator, but i'm aware of that.
>> >>>>
>> >>>> Here is the code block which should delete old entries:
>> >>>>
>> >>>> store: is the LRUMap
>> >>>> birth: is a long which keeps the creation time when this is set
to 0
>> >>>> the item should be deleted
>> >>>>
>> >>>> public void removeAllExpiredItems()
>> >>>>  {
>> >>>>   synchronized(this.store)
>> >>>>   {
>> >>>>     Iterator it=this.store.keySet().iterator();
>> >>>>     while(it.hasNext())
>> >>>>     {
>> >>>>       Object key=it.next();
>> >>>>       Object o=this.get(key);
>> >>>>       if(o != null)
>> >>>>       {
>> >>>>         Item iEntry=(Item)this.store.get(key);
>> >>>>         if(iEntry.birth==0)
>> >>>>           it.remove(); //Here the exception occurs
>> >>>>       }
>> >>>>     }
>> >>>>     this.setLastCleanDate(new Date()); //only to know when the
>> >>>> deleter run the last time
>> >>>>   }
>> >>>>  }
>> >>>>
>> >>>> Thanks for any help :-)
>> >>>> René
>> >>>>
>> >>>> 2009/6/15 James Carman :
>> >>>>> Are you calling remove() on the iterator or on the map itself?
>> >>>>>
>> >>>>> On Mon, Jun 15, 2009 at 6:37 AM, Renè Glanzer
>> wrote:
>> >>>>>> Hello,
>> >>>>>>
>> >>>>>> is there still no help for me?
>> >>>>>> Is somebody able to explain me, why i get this
>> >>>>>> "java.util.ConcurrentModificationException"
>> >>>>>> on iterating and calling remove() on the LRUMap?
>> >>>>>>
>> >>>>>> Please
>> >>>>>> René
>> >>>>>>
>> >>>>>> 2009/6/10 Renè Glanzer :
>> >>>>>>> Hello Ted,
>> >>>>>>>
>> >>>>>>> thanks for the fast response. I understand that simultaneously
puts
>> >>>>>>> and gets do not cause the exception cause they are synchronized
in my
>> >>>>>>> class.
>> >>>>>>> And my stated code-fragment is the part which wants
to delete the
>> >>>>>>> entries from the underlying LRUMap. And my code is wrapped
in the
>> >>>>>>> synchronized block. But i think the LRUMap herselfe
also iterates the
>> >>>>>>> entries and this maybe happens at the same time when
my iterator is
>> >>>>>>> checking and trying to delete elements. My class is
not implementing
>> >>>>>>> the LRUMap but has one LRUMap as a variable.
>> >>>>>>> So your solution to override the removeLRU(Entry) methode
would not
>> >>>>>>> help me....unfortunatelly :-(
>> >>>>>>>
>> >>>>>>> For now i really do not know how to solve the problem
in an easy way
>> >>>>>>> without refractoring the entire code?
>> >>>>>>> Any other solutions?
>> >>>>>>> Thanks René
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> 2009/6/9 Ted Dunning :
>> >>>>>>>> I apologize for not reading your thread with great
care, but I think
>> that
>> >>>>>>>> your problem is pretty clear.
>> >>>>>>>>
>> >>>>>>>> The issue is not to do with gets and puts overlapping.
 The issue is
>> that a
>> >>>>>>>> put or remove happened during the life of your iterator.
>> >>>>>>>>
>> >>>>>>>> One way to avoid this is to synchronize the entire
method that does the
>> >>>>>>>> deletion of old elements.  To speed that up, you
might just synchronize
>> the
>> >>>>>>>> scan for elements to delete and collect them into
a list.  Then you can
>> >>>>>>>> delete them outside the loop.  Since your scan
is probably pretty fast,
>> this
>> >>>>>>>> may be sufficient.  With very high levels of updates
and threading, it
>> would
>> >>>>>>>> cause problems.
>> >>>>>>>>
>> >>>>>>>> Another option is to clone the table.  I believe
that some
>> implementations
>> >>>>>>>> of LRUMap in Lucene do copy-on-write semantics,
but I don't think that
>> >>>>>>>> collections has these.  Without that, cloning will
be as slow or slower
>> than
>> >>>>>>>> your scan so the first option would be better.
>> >>>>>>>>
>> >>>>>>>> I am curious, however, why you didn't make use of
the built-in
>> capabilities
>> >>>>>>>> of the LRUMap to help you with this.  Notably,
you should probably just
>> >>>>>>>> over-ride the removeLRU(Entry) method and set the
scanUntilRemovable
>> flag.
>> >>>>>>>> I think that this would take the place of your loop
and would be much
>> more
>> >>>>>>>> efficient.
>> >>>>>>>>
>> >>>>>>>> On Tue, Jun 9, 2009 at 9:33 AM, Renè Glanzer
>> wrote:
>> >>>>>>>>
>> >>>>>>>>> ... Additionally to the maximum number of the
LRUMap i also
>> implemented a
>> >>>>>>>>> thread which periodicly checks the age of the
entries and deletes them
>> >>>>>>>>> when they are too old.
>> >>>>>>>>>
>> >>>>>>>>> For this i generated an Iterator which delivers
me each entry and if
>> >>>>>>>>> the creation date is too old i call iterator.remove().
>> >>>>>>>>> But exactly on this line i get an
>> >>>>>>>>> "java.util.ConcurrentModificationException"
although i wrapped all
>> >>>>>>>>> access to the map with synchronized blocks.
So every put and get
>> >>>>>>>>> methods are synchronized.
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>
>> >>>>>> ---------------------------------------------------------------------
>> >>>>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>> >>>>>> For additional commands, e-mail: user-help@commons.apache.org
>> >>>>>>
>> >>>>>>
>> >>>>>
>> >>>>> ---------------------------------------------------------------------
>> >>>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>> >>>>> For additional commands, e-mail: user-help@commons.apache.org
>> >>>>>
>> >>>>>
>> >>>>
>> >>>> ---------------------------------------------------------------------
>> >>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>> >>>> For additional commands, e-mail: user-help@commons.apache.org
>> >>>>
>> >>>>
>> >>>
>> >>> ---------------------------------------------------------------------
>> >>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>> >>> For additional commands, e-mail: user-help@commons.apache.org
>> >>>
>> >>>
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>> >> For additional commands, e-mail: user-help@commons.apache.org
>> >>
>> >>
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>> > For additional commands, e-mail: user-help@commons.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>> For additional commands, e-mail: user-help@commons.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org


Mime
View raw message