asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Maxon <ima...@uci.edu>
Subject Re: Working with Hadoop
Date Thu, 21 Jul 2016 02:24:25 GMT
That's all the ways we use Hadoop at the moment that I can think of as
well. Maybe the two other minor ones are zookeeper and HDFS backup in
Managix.

For 1) and 2) it's using Hadoop 2.2.0 right now. In my experimental branch
for 3) I'm using 2.6.0, it doesn't cause any more issues for me than 2.2.0.
I believe 1) used to support Hadoop 0.20.0 and other 1.x versions but I'm
not sure if that works anymore.

On Wed, Jul 20, 2016 at 7:14 PM, Till Westmann <tillw@apache.org> wrote:

> Hi everybody,
>
> recently the topic of Hadoop support came up and I realized that my
> understanding is quite spotty so I’m trying to understand where we are.
>
> AFAIK we support
> 1) HDFS for (potentially indexed) external datasets,
> 2) YARN as a resource manager, and
> 3) HDFS as a basis for internal storage.
> Is this list complete or do we have other Hadoop touchpoints?
>
> I believe that 1) and 2) should be reasonable stable and that 3) is still
> in
> the works. Is that correct?
>
> Further I'm wondering
> a) which versions of Hadoop we support and
> b) which ones we should support for all the cases.
> Please chime in on this as well.
>
> Any other things that anybody working with AsterixDB and Hadoop should be
> aware
> of?
>
> Thanks!
> Till
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message