spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matei Zaharia <matei.zaha...@gmail.com>
Subject Re: [VOTE] Release Apache Spark 1.1.1 (RC2)
Date Thu, 20 Nov 2014 19:59:15 GMT
Ah, I see. But the spark.shuffle.blockTransferService property doesn't exist in 1.1 (AFAIK)
-- what exactly are you doing to get this problem?

Matei

> On Nov 20, 2014, at 11:50 AM, Hector Yee <hector.yee@gmail.com> wrote:
> 
> This is whatever was in http://people.apache.org/~andrewor14/spark-1.1.1-rc2/ <http://people.apache.org/~andrewor14/spark-1.1.1-rc2/>
> 
> On Thu, Nov 20, 2014 at 11:48 AM, Matei Zaharia <matei.zaharia@gmail.com <mailto:matei.zaharia@gmail.com>>
wrote:
> Hector, is this a comment on 1.1.1 or on the 1.2 preview?
> 
> Matei
> 
> > On Nov 20, 2014, at 11:39 AM, Hector Yee <hector.yee@gmail.com <mailto:hector.yee@gmail.com>>
wrote:
> >
> > I think it is a race condition caused by netty deactivating a channel while
> > it is active.
> > Switched to nio and it works fine
> > --conf spark.shuffle.blockTransferService=nio
> >
> > On Thu, Nov 20, 2014 at 10:44 AM, Hector Yee <hector.yee@gmail.com <mailto:hector.yee@gmail.com>>
wrote:
> >
> >> I'm still seeing the fetch failed error and updated
> >> https://issues.apache.org/jira/browse/SPARK-3633 <https://issues.apache.org/jira/browse/SPARK-3633>
> >>
> >> On Thu, Nov 20, 2014 at 10:21 AM, Marcelo Vanzin <vanzin@cloudera.com <mailto:vanzin@cloudera.com>>
> >> wrote:
> >>
> >>> +1 (non-binding)
> >>>
> >>> . ran simple things on spark-shell
> >>> . ran jobs in yarn client & cluster modes, and standalone cluster mode
> >>>
> >>> On Wed, Nov 19, 2014 at 2:51 PM, Andrew Or <andrew@databricks.com <mailto:andrew@databricks.com>>
wrote:
> >>>> Please vote on releasing the following candidate as Apache Spark version
> >>>> 1.1.1.
> >>>>
> >>>> This release fixes a number of bugs in Spark 1.1.0. Some of the notable
> >>> ones
> >>>> are
> >>>> - [SPARK-3426] Sort-based shuffle compression settings are incompatible
> >>>> - [SPARK-3948] Stream corruption issues in sort-based shuffle
> >>>> - [SPARK-4107] Incorrect handling of Channel.read() led to data
> >>> truncation
> >>>> The full list is at http://s.apache.org/z9h <http://s.apache.org/z9h>
and in the CHANGES.txt
> >>> attached.
> >>>>
> >>>> Additionally, this candidate fixes two blockers from the previous RC:
> >>>> - [SPARK-4434] Cluster mode jar URLs are broken
> >>>> - [SPARK-4480][SPARK-4467] Too many open files exception from shuffle
> >>> spills
> >>>>
> >>>> The tag to be voted on is v1.1.1-rc2 (commit 3693ae5d):
> >>>> http://s.apache.org/p8 <http://s.apache.org/p8>
> >>>>
> >>>> The release files, including signatures, digests, etc can be found at:
> >>>> http://people.apache.org/~andrewor14/spark-1.1.1-rc2/ <http://people.apache.org/~andrewor14/spark-1.1.1-rc2/>
> >>>>
> >>>> Release artifacts are signed with the following key:
> >>>> https://people.apache.org/keys/committer/andrewor14.asc <https://people.apache.org/keys/committer/andrewor14.asc>
> >>>>
> >>>> The staging repository for this release can be found at:
> >>>> https://repository.apache.org/content/repositories/orgapachespark-1043/
<https://repository.apache.org/content/repositories/orgapachespark-1043/>
> >>>>
> >>>> The documentation corresponding to this release can be found at:
> >>>> http://people.apache.org/~andrewor14/spark-1.1.1-rc2-docs/ <http://people.apache.org/~andrewor14/spark-1.1.1-rc2-docs/>
> >>>>
> >>>> Please vote on releasing this package as Apache Spark 1.1.1!
> >>>>
> >>>> The vote is open until Saturday, November 22, at 23:00 UTC and passes
if
> >>>> a majority of at least 3 +1 PMC votes are cast.
> >>>> [ ] +1 Release this package as Apache Spark 1.1.1
> >>>> [ ] -1 Do not release this package because ...
> >>>>
> >>>> To learn more about Apache Spark, please see
> >>>> http://spark.apache.org/ <http://spark.apache.org/>
> >>>>
> >>>> Cheers,
> >>>> Andrew
> >>>>
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org <mailto:dev-unsubscribe@spark.apache.org>
> >>>> For additional commands, e-mail: dev-help@spark.apache.org <mailto:dev-help@spark.apache.org>
> >>>
> >>>
> >>>
> >>> --
> >>> Marcelo
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org <mailto:dev-unsubscribe@spark.apache.org>
> >>> For additional commands, e-mail: dev-help@spark.apache.org <mailto:dev-help@spark.apache.org>
> >>>
> >>>
> >>
> >>
> >> --
> >> Yee Yang Li Hector <http://google.com/+HectorYee <http://google.com/+HectorYee>>
> >> *google.com/+HectorYee <http://google.com/+HectorYee> <http://google.com/+HectorYee
<http://google.com/+HectorYee>>*
> >>
> >
> >
> >
> > --
> > Yee Yang Li Hector <http://google.com/+HectorYee <http://google.com/+HectorYee>>
> > *google.com/+HectorYee <http://google.com/+HectorYee> <http://google.com/+HectorYee
<http://google.com/+HectorYee>>*
> 
> 
> 
> 
> -- 
> Yee Yang Li Hector <http://google.com/+HectorYee>
> google.com/+HectorYee <http://google.com/+HectorYee>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message