spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <>
Subject Re: SPARK-13843 and future of streaming backends
Date Sat, 19 Mar 2016 12:35:39 GMT

> On 18 Mar 2016, at 22:24, Marcelo Vanzin <> wrote:
> On Fri, Mar 18, 2016 at 2:12 PM, chrismattmann <> wrote:
>> So, my comment here is that any code *cannot* be removed from an Apache
>> project if there is a VETO issued which so far I haven't seen, though maybe
>> Marcelo can clarify that.
> No, my intention was not to veto the change. I'm actually for the
> removal of components if the community thinks they don't add much to
> the project. (I'm also not sure I can even veto things, not being a
> PMC member.)
> I mainly wanted to know what was the path forward for those components
> because, with Cloudera's hat on, we care about one of them (streaming
> integration with flume), and we'd prefer if that code remained under
> the ASF umbrella in some way.

I'd be supportive of a spark-extras project; it'd actually be  place to keep stuff I've worked
 -the yarn ATS 1/1.5 integration
 -that mutant hive JAR which has the consistent kryo dependency and different shadings

... etc

There's also the fact that the twitter streaming is a common example to play with, flume is
popular in places too.

If you want to set up a new incubator with a goal of graduating fast, I'd help. As a key metric
of getting out of incubator is active development, you just need to "recruit" contributors
and keep them engaged.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message