commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Renè Glanzer <rene.glan...@gmail.com>
Subject Re: [collections] LRUMap Problem ConcurrentModificationException
Date Mon, 15 Jun 2009 16:00:54 GMT
Yes of course,

the code with the time based cache systems was set up with an ordniary
HashMap. But when rapidly seach requests are
generated the time based mechanism fails to delete old entries - cause
they are not old enough - and so the cache will
raise up until the entire VM memory is used. Then i found the LRUMap
switched to it and bang Exception in my delete method.

When i switch back to HashMap the code runs well - as currently on the
productive system - except the open memory leak.

René

2009/6/15 Leon Rosenberg <rosenberg.leon@googlemail.com>:
> just out of curiosity have you tried the same code with a standard hashmap?
>
> Leon
>
> On Mon, Jun 15, 2009 at 4:37 PM, Renè Glanzer<rene.glanzer@gmail.com> wrote:
>> Hello,
>>
>> side note accepted :-)
>>
>> In my class I checked the get, put and remove methods. All are synchronized.
>> As you can see also the code which wants to delete the elements is synchronized.
>> So I've no clue where the "ConcurrentModificationException" is comming from :-(
>>
>> Thanks René
>>
>> 2009/6/15 Leon Rosenberg <rosenberg.leon@googlemail.com>:
>>> Hello,
>>>
>>> on a side note, generics make reading of code easier :-)
>>>
>>> you haven't posted the whole code, but have you (double)checked that
>>> all other acesses to store are synchronized?
>>>
>>> regards
>>> Leon
>>>
>>> On Mon, Jun 15, 2009 at 2:31 PM, Renè Glanzer<rene.glanzer@gmail.com>
wrote:
>>>> I'm calling the remove() method on the iterator. I know when i call
>>>> the remove on the map itself it will cause the problem with my
>>>> currently running iterator, but i'm aware of that.
>>>>
>>>> Here is the code block which should delete old entries:
>>>>
>>>> store: is the LRUMap
>>>> birth: is a long which keeps the creation time when this is set to 0
>>>> the item should be deleted
>>>>
>>>> public void removeAllExpiredItems()
>>>>  {
>>>>   synchronized(this.store)
>>>>   {
>>>>     Iterator it=this.store.keySet().iterator();
>>>>     while(it.hasNext())
>>>>     {
>>>>       Object key=it.next();
>>>>       Object o=this.get(key);
>>>>       if(o != null)
>>>>       {
>>>>         Item iEntry=(Item)this.store.get(key);
>>>>         if(iEntry.birth==0)
>>>>           it.remove(); //Here the exception occurs
>>>>       }
>>>>     }
>>>>     this.setLastCleanDate(new Date()); //only to know when the
>>>> deleter run the last time
>>>>   }
>>>>  }
>>>>
>>>> Thanks for any help :-)
>>>> René
>>>>
>>>> 2009/6/15 James Carman <james@carmanconsulting.com>:
>>>>> Are you calling remove() on the iterator or on the map itself?
>>>>>
>>>>> On Mon, Jun 15, 2009 at 6:37 AM, Renè Glanzer<rene.glanzer@gmail.com>
wrote:
>>>>>> Hello,
>>>>>>
>>>>>> is there still no help for me?
>>>>>> Is somebody able to explain me, why i get this
>>>>>> "java.util.ConcurrentModificationException"
>>>>>> on iterating and calling remove() on the LRUMap?
>>>>>>
>>>>>> Please
>>>>>> René
>>>>>>
>>>>>> 2009/6/10 Renè Glanzer <rene.glanzer@gmail.com>:
>>>>>>> Hello Ted,
>>>>>>>
>>>>>>> thanks for the fast response. I understand that simultaneously
puts
>>>>>>> and gets do not cause the exception cause they are synchronized
in my
>>>>>>> class.
>>>>>>> And my stated code-fragment is the part which wants to delete
the
>>>>>>> entries from the underlying LRUMap. And my code is wrapped in
the
>>>>>>> synchronized block. But i think the LRUMap herselfe also iterates
the
>>>>>>> entries and this maybe happens at the same time when my iterator
is
>>>>>>> checking and trying to delete elements. My class is not implementing
>>>>>>> the LRUMap but has one LRUMap as a variable.
>>>>>>> So your solution to override the removeLRU(Entry) methode would
not
>>>>>>> help me....unfortunatelly :-(
>>>>>>>
>>>>>>> For now i really do not know how to solve the problem in an easy
way
>>>>>>> without refractoring the entire code?
>>>>>>> Any other solutions?
>>>>>>> Thanks René
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2009/6/9 Ted Dunning <ted.dunning@gmail.com>:
>>>>>>>> I apologize for not reading your thread with great care,
but I think that
>>>>>>>> your problem is pretty clear.
>>>>>>>>
>>>>>>>> The issue is not to do with gets and puts overlapping.  The
issue is that a
>>>>>>>> put or remove happened during the life of your iterator.
>>>>>>>>
>>>>>>>> One way to avoid this is to synchronize the entire method
that does the
>>>>>>>> deletion of old elements.  To speed that up, you might just
synchronize the
>>>>>>>> scan for elements to delete and collect them into a list.
 Then you can
>>>>>>>> delete them outside the loop.  Since your scan is probably
pretty fast, this
>>>>>>>> may be sufficient.  With very high levels of updates and
threading, it would
>>>>>>>> cause problems.
>>>>>>>>
>>>>>>>> Another option is to clone the table.  I believe that some
implementations
>>>>>>>> of LRUMap in Lucene do copy-on-write semantics, but I don't
think that
>>>>>>>> collections has these.  Without that, cloning will be as
slow or slower than
>>>>>>>> your scan so the first option would be better.
>>>>>>>>
>>>>>>>> I am curious, however, why you didn't make use of the built-in
capabilities
>>>>>>>> of the LRUMap to help you with this.  Notably, you should
probably just
>>>>>>>> over-ride the removeLRU(Entry) method and set the scanUntilRemovable
flag.
>>>>>>>> I think that this would take the place of your loop and would
be much more
>>>>>>>> efficient.
>>>>>>>>
>>>>>>>> On Tue, Jun 9, 2009 at 9:33 AM, Renè Glanzer <rene.glanzer@gmail.com>
wrote:
>>>>>>>>
>>>>>>>>> ... Additionally to the maximum number of the LRUMap
i also implemented a
>>>>>>>>> thread which periodicly checks the age of the entries
and deletes them
>>>>>>>>> when they are too old.
>>>>>>>>>
>>>>>>>>> For this i generated an Iterator which delivers me each
entry and if
>>>>>>>>> the creation date is too old i call iterator.remove().
>>>>>>>>> But exactly on this line i get an
>>>>>>>>> "java.util.ConcurrentModificationException" although
i wrapped all
>>>>>>>>> access to the map with synchronized blocks. So every
put and get
>>>>>>>>> methods are synchronized.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>>>>>> For additional commands, e-mail: user-help@commons.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>>>>> For additional commands, e-mail: user-help@commons.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail: user-help@commons.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: user-help@commons.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>> For additional commands, e-mail: user-help@commons.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org


Mime
View raw message