apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gyula Fóra <gyula.f...@gmail.com>
Subject Re: Using HDHT
Date Thu, 29 Oct 2015 16:12:52 GMT
Thanks Thomas,

I will try it out and let you know if I run into any issues or further
questions :)

Cheers,
Gyula

Thomas Weise <thomas@datatorrent.com> ezt írta (időpont: 2015. okt. 29.,
Cs, 16:10):

> Hi Gyula,
>
> It is a key-value store, embedded into the Apache Apex operator model. You
> can read more about it here:
>
> https://www.datatorrent.com/data-store-for-scalable-stream-processing/
>
> The underlying abstraction are block indexed files with ordered keys in
> HDFS.
>
> Please note that post 2.x version this operator is supported as DT RTS
> extension only, but you can look at the code as of 2.x here:
>
>
> https://github.com/apache/incubator-apex-malhar/tree/release-2.1/contrib/src/main/java/com/datatorrent/contrib/hdht
>
> Thanks,
> Thomas
>
>
>
> On Thu, Oct 29, 2015 at 7:00 AM, Gyula Fóra <gyfora@apache.org> wrote:
>
> > Hey guys,
> >
> > I am very interested in learning a little bit more about the HDHT
> > implementation that I found here
> > <
> >
> https://github.com/apache/incubator-apex-malhar/tree/hdhtOrcExport/contrib/src/main/java/com/datatorrent/contrib/hdht
> > >
> > .
> >
> > Could you please point me to some examples using this feature? I am
> > actually interested in using this logic outside of the streaming context,
> > just to store multiple elements per key for up to billions of keys on
> HDFS.
> >
> > Is there some more documentation on this somewhere?
> >
> > Cheers,
> > Gyula
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message