spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jey Kottalam <...@cs.berkeley.edu>
Subject Re: Build Changes for SBT Users
Date Sat, 04 Jan 2014 21:24:10 GMT
I'm in full agreement with Holden. We should provide a smooth out of the
box experience while also not getting in the way of those who provide their
own sbt installs.

On Saturday, January 4, 2014, Holden Karau wrote:

> That makes sense, I think we could structure a script in such a way that it
> would overcome these problems though and probably provide a fair a mount of
> benefit for people who just want to get started quickly.
>
> The easiest would be to have it use the system sbt if present and then fall
> back to downloading the sbt jar. As far as stability of the URL goes we
> could solve this by either having it point at a domain we control, or just
> with an clear error message indicating it failed to download sbt and the
> user needs to install sbt.
>
> If a restructured script in that manner would be useful I could whip up a
> pull request :)
>
>
> On Sat, Jan 4, 2014 at 10:56 AM, Patrick Wendell <pwendell@gmail.com<javascript:;>>
> wrote:
>
> > We thought about this but elected not to do this for a few reasons.
> >
> > 1. Some people build from machines that do not have internet access
> > for security reasons and retrieve dependency from internal nexus
> > repositories. So having a build dependency that relies on internet
> > downloads is not desirable.
> >
> > 2. It's a hard to ensure stability of a particular URL in perpetuity.
> > This is why maven central and other mirror networks exist. Keep in
> > mind that we can't change the release code ever once we release it,
> > and if something changed about the particular URL it could break the
> > build.
> >
> > - Patrick
> >
> > On Sat, Jan 4, 2014 at 9:34 AM, Andrew Ash <andrew@andrewash.com<javascript:;>>
> wrote:
> > > +1 on bundling a script similar to that one
> > >
> > >
> > > On Sat, Jan 4, 2014 at 4:48 AM, Holden Karau <holden@pigscanfly.ca<javascript:;>
> >
> > wrote:
> > >
> > >> Could we ship a shell script which downloads the sbt jar if not
> present
> > >> (like for example https://github.com/holdenk/slashem/blob/master/sbt)?
> > >>
> > >>
> > >> On Sat, Jan 4, 2014 at 12:02 AM, Patrick Wendell <pwendell@gmail.com<javascript:;>
> >
> > >> wrote:
> > >>
> > >> > Hey All,
> > >> >
> > >> > Due to an ASF requirement, we recently merged a patch which removes
> > >> > the sbt jar from the build. This is necessary because we aren't
> > >> > allowed to distributed binary artifacts with our source packages.
> > >> >
> > >> > This means that instead of building Spark with "sbt/sbt XXX", you'll
> > >> > need to have sbt yourself and just run "sbt XXX" from within the
> Spark
> > >> > directory. This is similar to the maven build, where we expect users
> > >> > already have maven installed.
> > >> >
> > >> > You can download sbt at http://www.scala-sbt.org/. It's okay to
> just
> > >> > download the most recent version of sbt, since sbt knows how to
> fetch
> > >> > other versions of itself and will always use the one we specify in
> our
> > >> > build file to compile spark.
> > >> >
> > >> > - Patrick
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> Cell : 425-233-8271
> > >>
> >
>
>
>
> --
> Cell : 425-233-8271
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message