spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Wendell <>
Subject Re: [DISCUSS] Necessity of Maven *and* SBT Build in Spark
Date Fri, 21 Feb 2014 05:18:54 GMT
Hey Henry,

Yep, I wanted to reboot this since some time has passed and people may
have new or changed ways of using the build.

Maven makes the Apache publishing fairly seamless, but after the last
two releases I believe we could make it work with sbt as well. sbt
also supports publishing and other Apache projects such as Kafka
publish with sbt.

On Thu, Feb 20, 2014 at 8:50 PM, Henry Saputra <> wrote:
> Thanks for bringing back the build systems discussions, Patrick.
> There was a long discussion way back before Spark joining ASF and as I
> remember there has not been clear "winner" between using sbt or maven.
> Maven makes it easier to publish the artifacts to Nexus repository,
> not sure if sbt can do  the same, and as I remember one of the
> limitations or drawbacks about maven is the use of profiles.
> Matei had suggested using some kind of Hadoop client detection as in
> Parquet project to manage the Hadoop versions to avoid profiles.
> - Henry
> On Thu, Feb 20, 2014 at 8:03 PM, Patrick Wendell <> wrote:
>> Hey All,
>> It's very high overhead having two build systems in Spark. Before
>> getting into a long discussion about the merits of sbt vs maven, I
>> wanted to pose a simple question to the dev list:
>> Is there anyone who feels that dropping either sbt or maven would have
>> a major consequence for them?
>> And I say "major consequence" meaning something becomes completely
>> impossible now and can't be worked around. This is different from an
>> "inconvenience", i.e., something which can be worked around but will
>> require some investment.
>> I'm posing the question in this way because, if there are features in
>> either build system that are absolutely-un-available in the other,
>> then we'll have to maintain both for the time being. I'm merely trying
>> to see whether this is the case...
>> - Patrick

View raw message