commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Leon Rosenberg <rosenberg.l...@googlemail.com>
Subject Re: [collections] LRUMap Problem ConcurrentModificationException
Date Mon, 15 Jun 2009 16:23:33 GMT
weird enough there are two LRUMap classes in commons collections,
org.apache.commons.collections and org.apache.commons.collections.map.
Which one are you using?

regards
Leon

On Mon, Jun 15, 2009 at 6:00 PM, Renè Glanzer<rene.glanzer@gmail.com> wrote:
> Yes of course,
>
> the code with the time based cache systems was set up with an ordniary
> HashMap. But when rapidly seach requests are
> generated the time based mechanism fails to delete old entries - cause
> they are not old enough - and so the cache will
> raise up until the entire VM memory is used. Then i found the LRUMap
> switched to it and bang Exception in my delete method.
>
> When i switch back to HashMap the code runs well - as currently on the
> productive system - except the open memory leak.
>
> René
>
> 2009/6/15 Leon Rosenberg <rosenberg.leon@googlemail.com>:
>> just out of curiosity have you tried the same code with a standard hashmap?
>>
>> Leon
>>
>> On Mon, Jun 15, 2009 at 4:37 PM, Renè Glanzer<rene.glanzer@gmail.com> wrote:
>>> Hello,
>>>
>>> side note accepted :-)
>>>
>>> In my class I checked the get, put and remove methods. All are synchronized.
>>> As you can see also the code which wants to delete the elements is synchronized.
>>> So I've no clue where the "ConcurrentModificationException" is comming from :-(
>>>
>>> Thanks René
>>>
>>> 2009/6/15 Leon Rosenberg <rosenberg.leon@googlemail.com>:
>>>> Hello,
>>>>
>>>> on a side note, generics make reading of code easier :-)
>>>>
>>>> you haven't posted the whole code, but have you (double)checked that
>>>> all other acesses to store are synchronized?
>>>>
>>>> regards
>>>> Leon
>>>>
>>>> On Mon, Jun 15, 2009 at 2:31 PM, Renè Glanzer<rene.glanzer@gmail.com>
wrote:
>>>>> I'm calling the remove() method on the iterator. I know when i call
>>>>> the remove on the map itself it will cause the problem with my
>>>>> currently running iterator, but i'm aware of that.
>>>>>
>>>>> Here is the code block which should delete old entries:
>>>>>
>>>>> store: is the LRUMap
>>>>> birth: is a long which keeps the creation time when this is set to 0
>>>>> the item should be deleted
>>>>>
>>>>> public void removeAllExpiredItems()
>>>>>  {
>>>>>   synchronized(this.store)
>>>>>   {
>>>>>     Iterator it=this.store.keySet().iterator();
>>>>>     while(it.hasNext())
>>>>>     {
>>>>>       Object key=it.next();
>>>>>       Object o=this.get(key);
>>>>>       if(o != null)
>>>>>       {
>>>>>         Item iEntry=(Item)this.store.get(key);
>>>>>         if(iEntry.birth==0)
>>>>>           it.remove(); //Here the exception occurs
>>>>>       }
>>>>>     }
>>>>>     this.setLastCleanDate(new Date()); //only to know when the
>>>>> deleter run the last time
>>>>>   }
>>>>>  }
>>>>>
>>>>> Thanks for any help :-)
>>>>> René
>>>>>
>>>>> 2009/6/15 James Carman <james@carmanconsulting.com>:
>>>>>> Are you calling remove() on the iterator or on the map itself?
>>>>>>
>>>>>> On Mon, Jun 15, 2009 at 6:37 AM, Renè Glanzer<rene.glanzer@gmail.com>
wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> is there still no help for me?
>>>>>>> Is somebody able to explain me, why i get this
>>>>>>> "java.util.ConcurrentModificationException"
>>>>>>> on iterating and calling remove() on the LRUMap?
>>>>>>>
>>>>>>> Please
>>>>>>> René
>>>>>>>
>>>>>>> 2009/6/10 Renè Glanzer <rene.glanzer@gmail.com>:
>>>>>>>> Hello Ted,
>>>>>>>>
>>>>>>>> thanks for the fast response. I understand that simultaneously
puts
>>>>>>>> and gets do not cause the exception cause they are synchronized
in my
>>>>>>>> class.
>>>>>>>> And my stated code-fragment is the part which wants to delete
the
>>>>>>>> entries from the underlying LRUMap. And my code is wrapped
in the
>>>>>>>> synchronized block. But i think the LRUMap herselfe also
iterates the
>>>>>>>> entries and this maybe happens at the same time when my iterator
is
>>>>>>>> checking and trying to delete elements. My class is not implementing
>>>>>>>> the LRUMap but has one LRUMap as a variable.
>>>>>>>> So your solution to override the removeLRU(Entry) methode
would not
>>>>>>>> help me....unfortunatelly :-(
>>>>>>>>
>>>>>>>> For now i really do not know how to solve the problem in
an easy way
>>>>>>>> without refractoring the entire code?
>>>>>>>> Any other solutions?
>>>>>>>> Thanks René
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> 2009/6/9 Ted Dunning <ted.dunning@gmail.com>:
>>>>>>>>> I apologize for not reading your thread with great care,
but I think that
>>>>>>>>> your problem is pretty clear.
>>>>>>>>>
>>>>>>>>> The issue is not to do with gets and puts overlapping.
 The issue is that a
>>>>>>>>> put or remove happened during the life of your iterator.
>>>>>>>>>
>>>>>>>>> One way to avoid this is to synchronize the entire method
that does the
>>>>>>>>> deletion of old elements.  To speed that up, you might
just synchronize the
>>>>>>>>> scan for elements to delete and collect them into a list.
 Then you can
>>>>>>>>> delete them outside the loop.  Since your scan is probably
pretty fast, this
>>>>>>>>> may be sufficient.  With very high levels of updates
and threading, it would
>>>>>>>>> cause problems.
>>>>>>>>>
>>>>>>>>> Another option is to clone the table.  I believe that
some implementations
>>>>>>>>> of LRUMap in Lucene do copy-on-write semantics, but I
don't think that
>>>>>>>>> collections has these.  Without that, cloning will be
as slow or slower than
>>>>>>>>> your scan so the first option would be better.
>>>>>>>>>
>>>>>>>>> I am curious, however, why you didn't make use of the
built-in capabilities
>>>>>>>>> of the LRUMap to help you with this.  Notably, you should
probably just
>>>>>>>>> over-ride the removeLRU(Entry) method and set the scanUntilRemovable
flag.
>>>>>>>>> I think that this would take the place of your loop and
would be much more
>>>>>>>>> efficient.
>>>>>>>>>
>>>>>>>>> On Tue, Jun 9, 2009 at 9:33 AM, Renè Glanzer <rene.glanzer@gmail.com>
wrote:
>>>>>>>>>
>>>>>>>>>> ... Additionally to the maximum number of the LRUMap
i also implemented a
>>>>>>>>>> thread which periodicly checks the age of the entries
and deletes them
>>>>>>>>>> when they are too old.
>>>>>>>>>>
>>>>>>>>>> For this i generated an Iterator which delivers me
each entry and if
>>>>>>>>>> the creation date is too old i call iterator.remove().
>>>>>>>>>> But exactly on this line i get an
>>>>>>>>>> "java.util.ConcurrentModificationException" although
i wrapped all
>>>>>>>>>> access to the map with synchronized blocks. So every
put and get
>>>>>>>>>> methods are synchronized.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>>>>>>> For additional commands, e-mail: user-help@commons.apache.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>>>>>> For additional commands, e-mail: user-help@commons.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>>>>> For additional commands, e-mail: user-help@commons.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail: user-help@commons.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: user-help@commons.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>> For additional commands, e-mail: user-help@commons.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org


Mime
View raw message