spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Reynold Xin <r...@databricks.com>
Subject Re: Build Changes for SBT Users
Date Sun, 05 Jan 2014 00:28:37 GMT
Doesn't Apache do redirection from incubation. to the normal website also?
 By the time that happens, we can also update the URL in the script?


On Sat, Jan 4, 2014 at 4:13 PM, Patrick Wendell <pwendell@gmail.com> wrote:

> Hey Holden,
>
> That sounds reasonable to me. Where would we get a url we can control
> though? Right now the project has web space is at incubator.apache...
> but later this will change to a full apache domain. Is there somewhere
> in maven central these jars are hosted... that would be the nicest
> because things like repo1.maven.org basically never changes.
>
> - Patrick
>
> On Sat, Jan 4, 2014 at 1:20 PM, Holden Karau <holden@pigscanfly.ca> wrote:
> > That makes sense, I think we could structure a script in such a way that
> it
> > would overcome these problems though and probably provide a fair a mount
> of
> > benefit for people who just want to get started quickly.
> >
> > The easiest would be to have it use the system sbt if present and then
> fall
> > back to downloading the sbt jar. As far as stability of the URL goes we
> > could solve this by either having it point at a domain we control, or
> just
> > with an clear error message indicating it failed to download sbt and the
> > user needs to install sbt.
> >
> > If a restructured script in that manner would be useful I could whip up a
> > pull request :)
> >
> >
> > On Sat, Jan 4, 2014 at 10:56 AM, Patrick Wendell <pwendell@gmail.com>
> wrote:
> >
> >> We thought about this but elected not to do this for a few reasons.
> >>
> >> 1. Some people build from machines that do not have internet access
> >> for security reasons and retrieve dependency from internal nexus
> >> repositories. So having a build dependency that relies on internet
> >> downloads is not desirable.
> >>
> >> 2. It's a hard to ensure stability of a particular URL in perpetuity.
> >> This is why maven central and other mirror networks exist. Keep in
> >> mind that we can't change the release code ever once we release it,
> >> and if something changed about the particular URL it could break the
> >> build.
> >>
> >> - Patrick
> >>
> >> On Sat, Jan 4, 2014 at 9:34 AM, Andrew Ash <andrew@andrewash.com>
> wrote:
> >> > +1 on bundling a script similar to that one
> >> >
> >> >
> >> > On Sat, Jan 4, 2014 at 4:48 AM, Holden Karau <holden@pigscanfly.ca>
> >> wrote:
> >> >
> >> >> Could we ship a shell script which downloads the sbt jar if not
> present
> >> >> (like for example https://github.com/holdenk/slashem/blob/master/sbt)?
> >> >>
> >> >>
> >> >> On Sat, Jan 4, 2014 at 12:02 AM, Patrick Wendell <pwendell@gmail.com
> >
> >> >> wrote:
> >> >>
> >> >> > Hey All,
> >> >> >
> >> >> > Due to an ASF requirement, we recently merged a patch which removes
> >> >> > the sbt jar from the build. This is necessary because we aren't
> >> >> > allowed to distributed binary artifacts with our source packages.
> >> >> >
> >> >> > This means that instead of building Spark with "sbt/sbt XXX",
> you'll
> >> >> > need to have sbt yourself and just run "sbt XXX" from within the
> Spark
> >> >> > directory. This is similar to the maven build, where we expect
> users
> >> >> > already have maven installed.
> >> >> >
> >> >> > You can download sbt at http://www.scala-sbt.org/. It's okay to
> just
> >> >> > download the most recent version of sbt, since sbt knows how to
> fetch
> >> >> > other versions of itself and will always use the one we specify
in
> our
> >> >> > build file to compile spark.
> >> >> >
> >> >> > - Patrick
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Cell : 425-233-8271
> >> >>
> >>
> >
> >
> >
> > --
> > Cell : 425-233-8271
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message