spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <>
Subject Re: SPARK-13843 Next steps
Date Tue, 29 Mar 2016 11:04:27 GMT
while sonatype are utterly strict about the org.apache namespace (it guarantees that all such
artifacts have come through the ASF release process, ideally including code-signing), nobody
checks the org.apache internals, or worries too much about them. Note that spark itself has
some bits of code in org.apache.hive so as to subclass the thriftserver.

What are the costs of having a project's package used externally?

1. interesting debugging sessions if JARs with conflicting classes are loaded.
2. you can't sign the JARs in the metadata. Nobody does that with the maven artifacts anyway.
3. whoever's package name it is often gets to see the stack traces in bug reports filed against

On 29 Mar 2016, at 01:47, Sean Owen <<>>

I tend to agree. If it's going to present a significant technical hurdle and the software
is clearly non ASF like via a different artifact, there's a decent argument the namespace
should stay. The artifact has to change though and that is what David was referring to in
his other message.

On Mon, Mar 28, 2016, 08:33 Cody Koeninger <<>>
I really think the only thing that should have to change is the maven
group and identifier, not the java namespace.

There are compatibility problems with the java namespace changing
(e.g. access to private[spark]), and I don't think that someone who
takes the time to change their build file to download a maven artifact
without "apache" in the identifier is at significant risk of consumer

I've tried to get a straight answer from ASF trademarks on this point,
but the answers I've been getting are mixed, and personally disturbing
to me in terms of over-reaching.

On Sat, Mar 26, 2016 at 9:03 AM, Sean Owen <<>>
> Looks like this is done; docs have been moved, flume is back in, etc.
> For the moment Kafka streaming is still in the project and I know
> there's still discussion about how to manage multiple versions within
> the project.
> One other thing we need to finish up is stuff like the namespace of
> the code that was moved out. I believe it'll have to move out of the
> org.apache namespace as well as change its artifact group. At least,
> David indicated Sonatype wouldn't let someone non-ASF push an artifact
> from that group anyway.
> Also might be worth adding a description at
> explaining that these are just some
> unofficial Spark-related packages.
> On Tue, Mar 22, 2016 at 7:27 AM, Kostas Sakellis <<>>
>> Hello all,
>> I'd like to close out the discussion on SPARK-13843 by getting a poll from
>> the community on which components we should seriously reconsider re-adding
>> back to Apache Spark. For reference, here are the modules that were removed
>> as part of SPARK-13843 and pushed to:
>> streaming-flume
>> streaming-akka
>> streaming-mqtt
>> streaming-zeromq
>> streaming-twitter
>> For us, we'd like to see the streaming-flume added back to Apache Spark.
>> Thanks,
>> Kostas
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:<>
> For additional commands, e-mail:<>

View raw message