spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Wendell <pwend...@gmail.com>
Subject Re: [VOTE] Release Apache Spark 1.0.1 (RC2)
Date Fri, 11 Jul 2014 16:37:45 GMT
Hey Gary,

Why do you think the akka frame size changed? It didn't change - we
added some fixes for cases where users were setting non-default
values.

On Fri, Jul 11, 2014 at 9:31 AM, Gary Malouf <malouf.gary@gmail.com> wrote:
> Hi Matei,
>
> We have not had time to re-deploy the rc today, but one thing that jumps
> out is the shrinking of the default akka frame size from 10MB to around
> 128KB by default.  That is my first suspicion for our issue - could imagine
> that biting others as well.
>
> I'll try to re-test that today - either way, understand moving forward at
> this point.
>
> Gary
>
>
> On Fri, Jul 11, 2014 at 12:08 PM, Matei Zaharia <matei.zaharia@gmail.com>
> wrote:
>
>> Unless you can diagnose the problem quickly, Gary, I think we need to go
>> ahead with this release as is. This release didn't touch the Mesos support
>> as far as I know, so the problem might be a nondeterministic issue with
>> your application. But on the other hand the release does fix some critical
>> bugs that affect all users. We can always do 1.0.2 later if we discover a
>> problem.
>>
>> Matei
>>
>> On Jul 10, 2014, at 9:40 PM, Patrick Wendell <pwendell@gmail.com> wrote:
>>
>> > Hey Gary,
>> >
>> > The vote technically doesn't close until I send the vote summary
>> > e-mail, but I was planning to close and package this tonight. It's too
>> > bad if there is a regression, it might be worth holding the release
>> > but it really requires narrowing down the issue to get more
>> > information about the scope and severity. Could you fork another
>> > thread for this?
>> >
>> > - Patrick
>> >
>> > On Thu, Jul 10, 2014 at 6:28 PM, Gary Malouf <malouf.gary@gmail.com>
>> wrote:
>> >> -1 I honestly do not know the voting rules for the Spark community, so
>> >> please excuse me if I am out of line or if Mesos compatibility is not a
>> >> concern at this point.
>> >>
>> >> We just tried to run this version built against 2.3.0-cdh5.0.2 on mesos
>> >> 0.18.2.  All of our jobs with data above a few gigabytes hung
>> indefinitely.
>> >> Downgrading back to the 1.0.0 stable release of Spark built the same way
>> >> worked for us.
>> >>
>> >>
>> >> On Mon, Jul 7, 2014 at 5:17 PM, Tom Graves <tgraves_cs@yahoo.com.invalid
>> >
>> >> wrote:
>> >>
>> >>> +1. Ran some Spark on yarn jobs on a hadoop 2.4 cluster with
>> >>> authentication on.
>> >>>
>> >>> Tom
>> >>>
>> >>>
>> >>> On Friday, July 4, 2014 2:39 PM, Patrick Wendell <pwendell@gmail.com>
>> >>> wrote:
>> >>>
>> >>>
>> >>>
>> >>> Please vote on releasing the following candidate as Apache Spark
>> version
>> >>> 1.0.1!
>> >>>
>> >>> The tag to be voted on is v1.0.1-rc1 (commit 7d1043c):
>> >>>
>> >>>
>> https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=7d1043c99303b87aef8ee19873629c2bfba4cc78
>> >>>
>> >>> The release files, including signatures, digests, etc. can be found
at:
>> >>> http://people.apache.org/~pwendell/spark-1.0.1-rc2/
>> >>>
>> >>> Release artifacts are signed with the following key:
>> >>> https://people.apache.org/keys/committer/pwendell.asc
>> >>>
>> >>> The staging repository for this release can be found at:
>> >>>
>> https://repository.apache.org/content/repositories/orgapachespark-1021/
>> >>>
>> >>> The documentation corresponding to this release can be found at:
>> >>> http://people.apache.org/~pwendell/spark-1.0.1-rc2-docs/
>> >>>
>> >>> Please vote on releasing this package as Apache Spark 1.0.1!
>> >>>
>> >>> The vote is open until Monday, July 07, at 20:45 UTC and passes if
>> >>> a majority of at least 3 +1 PMC votes are cast.
>> >>>
>> >>> [ ] +1 Release this package as Apache Spark 1.0.1
>> >>> [ ] -1 Do not release this package because ...
>> >>>
>> >>> To learn more about Apache Spark, please see
>> >>> http://spark.apache.org/
>> >>>
>> >>> === Differences from RC1 ===
>> >>> This release includes only one "blocking" patch from rc1:
>> >>> https://github.com/apache/spark/pull/1255
>> >>>
>> >>> There are also smaller fixes which came in over the last week.
>> >>>
>> >>> === About this release ===
>> >>> This release fixes a few high-priority bugs in 1.0 and has a variety
>> >>> of smaller fixes. The full list is here: http://s.apache.org/b45. Some
>> >>> of the more visible patches are:
>> >>>
>> >>> SPARK-2043: ExternalAppendOnlyMap doesn't always find matching keys
>> >>> SPARK-2156 and SPARK-1112: Issues with jobs hanging due to akka frame
>> size.
>> >>> SPARK-1790: Support r3 instance types on EC2.
>> >>>
>> >>> This is the first maintenance release on the 1.0 line. We plan to make
>> >>> additional maintenance releases as new fixes come in.
>> >>>
>>
>>

Mime
View raw message