apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Weise <thomas.we...@gmail.com>
Subject Re: [malhar-users] Caching in Datatorrent
Date Tue, 01 Sep 2015 21:58:21 GMT
An operator can use any technology to implement the cache. Here is an
example how to structure a cache from the Malhar library:

https://github.com/apache/incubator-apex-malhar/blob/v3.0.0/library/src/main/java/com/datatorrent/lib/db/cache/AbstractDBLookupCacheBackedOperator.java


Please note that the resulting caching is local to each operator instance
(every partition has its own cache). The idea is that access to shared
state is minimized for more efficient processing.


On Mon, Aug 3, 2015 at 8:21 AM, Amol Kekre <amol@datatorrent.com> wrote:

>
> Chiranjeevi,
> We do have guava (
> https://code.google.com/p/guava-libraries/wiki/CachesExplained) available
> as a Malhar operator
>
>
> https://github.com/DataTorrent/Malhar/blob/master/contrib/src/main/java/org/apache/hadoop/io/file/tfile/CacheManager.java
>
> Do take a look.
>
> Thks,
> Amol
>
>
> On Mon, Aug 3, 2015 at 1:51 AM, ravi <chiru.vcj@gmail.com> wrote:
>
>> Hi Team,
>>
>> Do we have caching available in Datatorrent? If yes, Please let me know
>> how to implement.
>>
>> -many thanks
>> Chiranjeevi
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Malhar" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to malhar-users+unsubscribe@googlegroups.com.
>> To post to this group, send email to malhar-users@googlegroups.com.
>> Visit this group at http://groups.google.com/group/malhar-users.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Malhar" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to malhar-users+unsubscribe@googlegroups.com.
> To post to this group, send email to malhar-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/malhar-users.
> For more options, visit https://groups.google.com/d/optout.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message