directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohit Srivastava <srivastavamohi...@gmail.com>
Subject Re: [Mavibot] Need for a cache to replace the WeakReferences...
Date Tue, 03 Sep 2013 06:53:17 GMT
I am trying few examples of it now. Lets see what's it having.

Cheers,
Mohit


On 3 September 2013 03:19, Emmanuel Lécharny <elecharny@gmail.com> wrote:

>
> Otherwise, I just found this :
>
> http://code.google.com/p/concurrentlinkedhashmap/
>
> AL 2.0, so we may use it.
>
> Worth a try, IMO.
>
> Le 9/2/13 7:11 PM, Mohit Srivastava a écrit :
> > I think we can use EhCache for now. Also modify it for our purposes.
> >
> >
> > On 2 September 2013 14:32, Emmanuel Lécharny <elecharny@gmail.com>
> wrote:
> >
> >> Hi guys,
> >>
> >> I have done some tests a week ago on Mavibot partition. It appears that
> >> we have a serious performance issue as soon as the JVM memory is
> >> completly eaten, when adding some entries. What happens is that we start
> >> to discard some pages as they are hold by WeakReferences, and the GC has
> >> to do quite a expensive processing to get back some free memory. This
> >> slows down the add operation so much (by at least a factor 10) that it's
> >> not possible to keep going with WeakReferences (or SoftReferences).
> >>
> >> At this point, the best solution would be to use a cache to replace this
> >> mechanism.
> >>
> >> There are different things we can cache :
> >> o PageIo : it's useless, as we still have to deserialize them afterward
> >> o Nodes and Leaves : They contain references to other Nodes or Leaves,
> >> and for Leaves, reference to data. Holding the upper Nodes would save us
> >> a lot of time
> >> o Data : the ultimate objects to cache : having the object in cache
> >> spares us the time it takes to searcj it in a Btree
> >>
> >> A few other things to consider :
> >> o we need a versionned cache : all the elements may be versionned.
> >> o each BTree may be cached, with different configuation (some BTree may
> >> be used rarely, other may be heavily used, requiring a more aggressive
> >> cache configuration)
> >> o the Cache must not use locks, but CAS (Compare-And-Swap) for better
> >> performance
> >>
> >> At this point, I suggest using EhCache as a first approach, in order to
> >> leverage the cache we already use in the server, but we may want to
> >> design our own LRU cache (something that should not be too complex to
> >> implement).
> >>
> >> Thoughts ?
> >>
> >> --
> >> Regards,
> >> Cordialement,
> >> Emmanuel Lécharny
> >> www.iktek.com
> >>
> >>
>
>
> --
> Regards,
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
>
>

Mime
View raw message