spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Wendell <pwend...@gmail.com>
Subject Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc2)
Date Mon, 09 Dec 2013 04:02:41 GMT
Hey Mark,

What I'm asking is whether this patch is sufficient to have a working
debian build in 0.8.1, or are there other outstanding issues to make
it work? By working I mean, within the initial design that was
contributed (with repl-bin) it works according to that approach.

We can redesign this packaging in 0.9. That will require having a PR
against Apache Spark, discussing, etc. But it doesn't need to be on
the critical path for this release.

- Patrick

On Sun, Dec 8, 2013 at 7:54 PM, Mark Hamstra <mark@clearstorydata.com> wrote:
> Whatever Debian package gets built has to work, so that's the first
> requirement.  I don't know how to decide whether a change is acceptable in
> 0.8 or has to wait until 0.9, but the 0.9 packaging should definitely
> leverage the assembly sub-project, making repl-bin unnecessary.
>
>
> On Sun, Dec 8, 2013 at 7:46 PM, Patrick Wendell <pwendell@gmail.com> wrote:
>
>> Looked into this a bit more - I think removing repl-bin is something
>> we should wait until 0.9 to do, because we've published it to maven in
>> 0.8.0 and people might expect it to be there in 0.8.1.
>>
>> Merging the directly referenced pull request (195) seems like a good
>> idea though since it fixes a bug in the script.
>>
>> Is that what you are suggesting?
>>
>> - Patrick
>>
>> On Sun, Dec 8, 2013 at 7:30 PM, Patrick Wendell <pwendell@gmail.com>
>> wrote:
>> > Hey Mark - ya this would be good to get in.
>> >
>> > Does merging that particular PR put this in sufficient shape for the
>> > 0.8.1 release or are there other open patches we need to look at?
>> >
>> > - Patrick
>> >
>> > On Sun, Dec 8, 2013 at 6:05 PM, Mark Hamstra <mark@clearstorydata.com>
>> wrote:
>> >> SPARK-962 should be resolved before release.  See also:
>> >> https://github.com/apache/incubator-spark/pull/195
>> >>
>> >> With the references to the way I changed Debian packaging for
>> ClearStory,
>> >> we should be at least 90% of the way toward doing it right for Apache.
>> >>
>> >>
>> >> On Sun, Dec 8, 2013 at 5:29 PM, Patrick Wendell <pwendell@gmail.com>
>> wrote:
>> >>
>> >>> For my own part I'll give a +1 to this RC.
>> >>>
>> >>> On Sun, Dec 8, 2013 at 4:30 PM, Taka Shinagawa <taka.epsilon@gmail.com
>> >
>> >>> wrote:
>> >>> > OK. I will post the entire output via separate email. I just upgraded
>> >>> > Hadoop to 2.2.0 recently. So there might be something I need to
>> >>> > remove/clean up.
>> >>> >
>> >>> >
>> >>> > On Sun, Dec 8, 2013 at 4:24 PM, Patrick Wendell <pwendell@gmail.com>
>> >>> wrote:
>> >>> >
>> >>> >> Hey Take,
>> >>> >>
>> >>> >> Could you start a separate thread to debug your build issue?
In that
>> >>> >> thread, could you paste the exact build command and entire
output?
>> The
>> >>> log
>> >>> >> you posted here suggests the first build detected hadoop 1.0.4
not
>> 2.2.0
>> >>> >> based on the assembly file name it is logging.
>> >>> >>
>> >>> >> ---
>> >>> >> sent from my phone
>> >>> >> On Dec 8, 2013 4:13 PM, "Taka Shinagawa" <taka.epsilon@gmail.com>
>> >>> wrote:
>> >>> >>
>> >>> >> > With Hadoop 2.2.0 (& Java 1.7.0_45) installed, I'm
having trouble
>> >>> >> > completing the build process (sbt/sbt assembly) on Macbook.
The
>> sbt
>> >>> >> command
>> >>> >> > hangs at the last step.
>> >>> >> >
>> >>> >> > ...
>> >>> >> > ...
>> >>> >> > [info] SHA-1: ce8275f5841002164c4305c912a2892ec7c1d395
>> >>> >> > [info] Packaging
>> >>> >> >
>> >>> >> >
>> >>> >>
>> >>>
>> /Users/taka/Documents/Spark/Releases/spark-0.8.1-incubating-rc2/tools/target/scala-2.9.3/spark-tools-assembly-0.8.1-incubating.jar
>> >>> >> > ...
>> >>> >> > [info] SHA-1: 0657a347240266230247693f265a5797d40c326a
>> >>> >> > [info] Packaging
>> >>> >> >
>> >>> >> >
>> >>> >>
>> >>>
>> /Users/taka/Documents/Spark/Releases/spark-0.8.1-incubating-rc2/assembly/target/scala-2.9.3/spark-assembly-0.8.1-incubating-hadoop1.0.4.jar
>> >>> >> > ...
>> >>> >> > (hangs here)
>> >>> >> > --------------------------
>> >>> >> >
>> >>> >> >
>> >>> >> > On another Macbook with Hadoop 1.1.1 (& Java 1.7.0_45)
installed,
>> I
>> >>> was
>> >>> >> > able to build it successfully.
>> >>> >> > ..
>> >>> >> > ..
>> >>> >> > [info] SHA-1: 77109cd085bd4f0d2b601b3451b35b961d357534
>> >>> >> > [info] Packaging
>> >>> >> >
>> >>> >> >
>> >>> >>
>> >>>
>> /Users/tshinagawa/Documents/Spark/RCs/spark-0.8.1-incubating/examples/target/scala-2.9.3/spark-examples-assembly-0.8.1-incubating.jar
>> >>> >> > ...
>> >>> >> > [info] Done packaging.
>> >>> >> > [success] Total time: 266 s, completed Dec 8, 2013 3:03:10
PM
>> >>> >> > --------------------------
>> >>> >> >
>> >>> >> >
>> >>> >> >
>> >>> >> > On Sun, Dec 8, 2013 at 12:41 PM, Patrick Wendell <
>> pwendell@gmail.com>
>> >>> >> > wrote:
>> >>> >> >
>> >>> >> > > Please vote on releasing the following candidate
as Apache Spark
>> >>> >> > > (incubating) version 0.8.1.
>> >>> >> > >
>> >>> >> > > The tag to be voted on is v0.8.1-incubating (commit
bf23794a):
>> >>> >> > >
>> >>> >> > >
>> >>> >> >
>> >>> >>
>> >>>
>> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=tag;h=e6ba91b5a7527316202797fc3dce469ff86cf203
>> >>> >> > >
>> >>> >> > > The release files, including signatures, digests,
etc can be
>> found
>> >>> at:
>> >>> >> > > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc2/
>> >>> >> > >
>> >>> >> > > Release artifacts are signed with the following key:
>> >>> >> > > https://people.apache.org/keys/committer/pwendell.asc
>> >>> >> > >
>> >>> >> > > The staging repository for this release can be found
at:
>> >>> >> > >
>> >>> https://repository.apache.org/content/repositories/orgapachespark-024/
>> >>> >> > >
>> >>> >> > > The documentation corresponding to this release can
be found at:
>> >>> >> > >
>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc2-docs/
>> >>> >> > >
>> >>> >> > > For information about the contents of this release
see:
>> >>> >> > > <attached> draft of release notes
>> >>> >> > > <attached> draft of release credits
>> >>> >> > >
>> >>> https://github.com/apache/incubator-spark/blob/branch-0.8/CHANGES.txt
>> >>> >> > >
>> >>> >> > > Please vote on releasing this package as Apache Spark
>> >>> 0.8.1-incubating!
>> >>> >> > >
>> >>> >> > > The vote is open until Wednesday, December 11th at
21:00 UTC and
>> >>> >> > > passes if a majority of at least 3 +1 PPMC votes
are cast.
>> >>> >> > >
>> >>> >> > > [ ] +1 Release this package as Apache Spark 0.8.1-incubating
>> >>> >> > > [ ] -1 Do not release this package because ...
>> >>> >> > >
>> >>> >> > > To learn more about Apache Spark, please see
>> >>> >> > > http://spark.incubator.apache.org/
>> >>> >> > >
>> >>> >> >
>> >>> >>
>> >>>
>>

Mime
View raw message