directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohit Srivastava <srivastavamohi...@gmail.com>
Subject Re: [Mavibot] Need for a cache to replace the WeakReferences...
Date Wed, 04 Sep 2013 10:15:48 GMT
I have been looking into http://code.google.com/p/concurrentlinkedhashmap/ .
I like to know how it will help in maintaining versionned cache? May be we
have to create one more HashMap just for versions.


On 3 September 2013 12:23, Mohit Srivastava <srivastavamohit91@gmail.com>wrote:

> I am trying few examples of it now. Lets see what's it having.
>
> Cheers,
> Mohit
>
>
> On 3 September 2013 03:19, Emmanuel Lécharny <elecharny@gmail.com> wrote:
>
>>
>> Otherwise, I just found this :
>>
>> http://code.google.com/p/concurrentlinkedhashmap/
>>
>> AL 2.0, so we may use it.
>>
>> Worth a try, IMO.
>>
>> Le 9/2/13 7:11 PM, Mohit Srivastava a écrit :
>> > I think we can use EhCache for now. Also modify it for our purposes.
>> >
>> >
>> > On 2 September 2013 14:32, Emmanuel Lécharny <elecharny@gmail.com>
>> wrote:
>> >
>> >> Hi guys,
>> >>
>> >> I have done some tests a week ago on Mavibot partition. It appears that
>> >> we have a serious performance issue as soon as the JVM memory is
>> >> completly eaten, when adding some entries. What happens is that we
>> start
>> >> to discard some pages as they are hold by WeakReferences, and the GC
>> has
>> >> to do quite a expensive processing to get back some free memory. This
>> >> slows down the add operation so much (by at least a factor 10) that
>> it's
>> >> not possible to keep going with WeakReferences (or SoftReferences).
>> >>
>> >> At this point, the best solution would be to use a cache to replace
>> this
>> >> mechanism.
>> >>
>> >> There are different things we can cache :
>> >> o PageIo : it's useless, as we still have to deserialize them afterward
>> >> o Nodes and Leaves : They contain references to other Nodes or Leaves,
>> >> and for Leaves, reference to data. Holding the upper Nodes would save
>> us
>> >> a lot of time
>> >> o Data : the ultimate objects to cache : having the object in cache
>> >> spares us the time it takes to searcj it in a Btree
>> >>
>> >> A few other things to consider :
>> >> o we need a versionned cache : all the elements may be versionned.
>> >> o each BTree may be cached, with different configuation (some BTree may
>> >> be used rarely, other may be heavily used, requiring a more aggressive
>> >> cache configuration)
>> >> o the Cache must not use locks, but CAS (Compare-And-Swap) for better
>> >> performance
>> >>
>> >> At this point, I suggest using EhCache as a first approach, in order to
>> >> leverage the cache we already use in the server, but we may want to
>> >> design our own LRU cache (something that should not be too complex to
>> >> implement).
>> >>
>> >> Thoughts ?
>> >>
>> >> --
>> >> Regards,
>> >> Cordialement,
>> >> Emmanuel Lécharny
>> >> www.iktek.com
>> >>
>> >>
>>
>>
>> --
>> Regards,
>> Cordialement,
>> Emmanuel Lécharny
>> www.iktek.com
>>
>>
>

Mime
View raw message