hdt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Evert Lammerts <ev...@apache.org>
Subject Re: [VOTE] Retire HDT
Date Fri, 28 Nov 2014 19:13:21 GMT
Hi Rahul, others,

Bob, Mirko and myself agree that a development environment for Hadoop can
be of added value to the community. After all effort that's gone into
making packaging (BigTop), deploying and tuning (Ambari, Cloudera Manager,
etc) Hadoop, and considering the ease with which a Hadoop / Spark cluster
can be deployed on public clouds, it'd be a logical step: as soon as you
have a Hadoop environment up and running, how are people going to write
jobs, and interact with the cluster in general?

But regrettably we can't currently commit enough time among the three of us
to make any serious progress. So as far as we're concerned you can go ahead
and retire HDT Rahul!


On Sun Nov 23 2014 at 8:49:01 PM Evert Lammerts <evert@apache.org> wrote:

> A little update to prevent the vote from closing: we're still discussing
> possible steps to make, see details in the thread below. We should have
> some sort of plan in a couple of days. In the meantime: contributions are
> very welcome!
> Evert
> ---------- Forwarded message ---------
> From: Evert Lammerts <evert.lammerts@gmail.com>
> Date: Sun Nov 23 2014 at 2:05:27 PM
> Subject: Re: [VOTE] Retire HDT
> To: Bob Kerns <rwk@acm.org>, Mirko Kämpf <mirko.kaempf@gmail.com>
> I think that an environment that allows fast experimentation adds most
> value: building MapReduce and Spark jobs through Java, Scala, Python and
> Pig. It can provide a number of commonly used libraries (Mahout, Giraph,
> MLlib, GraphX), UDF's, Input- OutputFormats, Pig Readers and Writers and
> code templates. Would you guys commit to making an attempt at building
> this? If so then I propose we write a short proposal and send it to the
> list. In the meantime I'll forward this thread.
> Bob, I don't know enough about virtualenv internals. Do you think it'd be
> possible to plug that into Orion or Cloud9's server?
> Mirko, I'm open to exploring possibilities within Oozie, but personally
> I've never used it: I just write drivers that chain jobs. So I can't judge
> its applicability right now. Could we create a standalone IDE and when (if)
> we get that to work, plug it into Oozie?
> How about effort? I'm willing to commit to putting some time in the coming
> few months, but it'll be hard to do more than 10 hours weekly or so.
> Cheers,
> Evert

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message