incubator-s4-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Danilo Pinto da Silva <danilob...@gmail.com>
Subject Re: Garbage Collector
Date Thu, 16 Jan 2014 17:45:47 GMT
Thank you for all your assistance.

Finally,
After of a PE eviction, the value of the keyed attribute will be lost?


Best regards,
Danilo P.


2014/1/15 Matthieu Morel <mmorel@apache.org>

>
> On Jan 15, 2014, at 00:34 , Danilo Pinto da Silva <danilobits@gmail.com>
> wrote:
>
> > Thank you for your prompt reply.
> >
> > In general I am talking about Garbage Collection of PEs.
> > Removing PE objects based on the available system memory and the impact
> the
> > object may have on the overall performance of the system.
> > But I have some doubts, in terms of:
> >
> > How are PEs located?
> > ( All PEs stay in a cache managed by LoadingCache object or in another
> > place and only the most used are in the cache? )
>
> PEs are held in a cache with a TTL. You may hook into that to persist them
> by checkpointing upon eviction.
> And you can replace the cache implementation and use something like a
> multilevel cache, keeping hot PEs locally and others in memcached or
> something like that.
>
> >
> > How are PEs controled?
> > ( Are they just defined in setPECache()? )
>
> I don't get the question.
>
> >
> > What relation about the simple garbage collection process (or others) and
> > the java garbage collection?
>
> You can implement caches with weak/soft references. Eviction policy would
> probably be different than with TTL-based caches.
>
> >
> > And again,
> > Does anybody tried to implement it?
> > Is it a relevant feature for S4?
> > Is it possible?
>
> It can be useful and certainly possible. I would say it depends on the use
> case, and in particular, on the size of PEs, and the number of instances.
> You should try to find the bottlenecks there (most optimizations in S4 have
> been on serialization, parallelism and communication, not really on the
> cache implementation).
>
> Have a look at the guava cache implementation for a start, you can find
> some related presentations as well.
>
> Hope this helps,
>
> Matthieu
>
> >
> > Best regards,
> > Danilo P.
> >
> >
> >
> >
> > 2014/1/14 kishore g <g.kishore@gmail.com>
> >
> >> Can you provide more info on the topic. Are you talking about Garbage
> >> collection of PEs.  There is simple garbage collection process that
> expires
> >> a PE after certain time of inactivity.
> >>
> >> thanks,
> >> Kishore G
> >>
> >>
> >> On Tue, Jan 14, 2014 at 9:26 AM, Danilo Pinto da Silva <
> >> danilobits@gmail.com
> >>> wrote:
> >>
> >>> Hello,
> >>>
> >>> I'm starting my masters research, and I some colleagues suggested me
> >>> propose and implement a garbage collector for S4.
> >>> Does anybody tried to implement it? Is it a relevant feature for S4?
> >>> Any comments are welcome (even in private if you prefer).
> >>>
> >>> Best regards,
> >>> Danilo P.
> >>>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message