spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicholas Chammas <nicholas.cham...@gmail.com>
Subject Re: Building Spark with Pants
Date Tue, 03 Feb 2015 00:40:46 GMT
I'm asking from an experimental standpoint; this is not happening anytime
soon.

Of course, if the experiment turns out very well, Pants would replace both
sbt and Maven (like it has at Twitter, for example). Pants also works with
IDEs <http://pantsbuild.github.io/index.html#using-pants-with>.

On Mon Feb 02 2015 at 4:33:11 PM Stephen Boesch <javadba@gmail.com> wrote:

> There is a significant investment in sbt and maven - and they are not at
> all likely to be going away. A third build tool?  Note that there is also
> the perspective of building within an IDE - which actually works presently
> for sbt and with a little bit of tweaking with maven as well.
>
> 2015-02-02 16:25 GMT-08:00 Nicholas Chammas <nicholas.chammas@gmail.com>:
>
>> Does anyone here have experience with Pants
>>
> <http://pantsbuild.github.io/index.html> or interest in trying to build
>
>
>> Spark with it?
>>
>> Pants has an interesting story. It was born at Twitter to help them build
>> their Scala, Java, and Python projects as several independent components
>> in
>> one monolithic repo. (It was inspired by a similar build tool at Google
>> called blaze.) The mix of languages and sub-projects at Twitter seems
>> similar to the breakdown we have in Spark.
>>
>> Pants has an interesting take on how a build system should work, and
>> Twitter and Foursquare (who use Pants as their primary build tool) claim
>> it
>> helps enforce better build hygiene and maintainability.
>>
>> Some relevant talks:
>>
>>    - Building Scala Hygienically with Pants
>>    <https://www.youtube.com/watch?v=ukqke8iTuH0>
>>    - The Pants Build Tool at Twitter
>>    <
>> https://engineering.twitter.com/university/videos/the-pants-build-tool-at-twitter
>> >
>>    - Getting Started with the Pants Build System: Why Pants?
>>    <
>> https://engineering.twitter.com/university/videos/getting-started-with-the-pants-build-system-why-pants
>> >
>
>
>>
>> At some point I may take a shot at converting Spark to use Pants as an
>> experiment and just see what it’s like.
>>
>> Nick
>> ​
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message