spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom Graves <tgraves...@yahoo.com.INVALID>
Subject Re: [VOTE] Release Apache Spark 1.2.0 (RC2)
Date Sat, 13 Dec 2014 12:58:25 GMT
+1 built and tested on Yarn on Hadoop 2.x cluster.
Tom 

     On Saturday, December 13, 2014 12:48 AM, Denny Lee <denny.g.lee@gmail.com> wrote:
   

 +1 Tested on OSX

Tested Scala 2.10.3, SparkSQL with Hive 0.12 / Hadoop 2.5, Thrift Server,
MLLib SVD


On Fri Dec 12 2014 at 8:57:16 PM Mark Hamstra <mark@clearstorydata.com>
wrote:

> +1
>
> On Fri, Dec 12, 2014 at 8:00 PM, Josh Rosen <rosenville@gmail.com> wrote:
> >
> > +1.  Tested using spark-perf and the Spark EC2 scripts.  I didn’t notice
> > any performance regressions that could not be attributed to changes of
> > default configurations.  To be more specific, when running Spark 1.2.0
> with
> > the Spark 1.1.0 settings of spark.shuffle.manager=hash and
> > spark.shuffle.blockTransferService=nio, there was no performance
> regression
> > and, in fact, there were significant performance improvements for some
> > workloads.
> >
> > In Spark 1.2.0, the new default settings are spark.shuffle.manager=sort
> > and spark.shuffle.blockTransferService=netty.  With these new settings,
> I
> > noticed a performance regression in the scala-sort-by-key-int spark-perf
> > test.  However, Spark 1.1.0 and 1.1.1 exhibit a similar performance
> > regression for that same test when run with spark.shuffle.manager=sort,
> so
> > this regression seems explainable by the change of defaults.  Besides
> this,
> > most of the other tests ran at the same speeds or faster with the new
> 1.2.0
> > defaults.  Also, keep in mind that this is a somewhat artificial micro
> > benchmark; I have heard anecdotal reports from many users that their real
> > workloads have run faster with 1.2.0.
> >
> > Based on these results, I’m comfortable giving a +1 on 1.2.0 RC2.
> >
> > - Josh
> >
> > On December 11, 2014 at 9:52:39 AM, Sandy Ryza (sandy.ryza@cloudera.com)
> > wrote:
> >
> > +1 (non-binding). Tested on Ubuntu against YARN.
> >
> > On Thu, Dec 11, 2014 at 9:38 AM, Reynold Xin <rxin@databricks.com>
> wrote:
> >
> > > +1
> > >
> > > Tested on OS X.
> > >
> > > On Wednesday, December 10, 2014, Patrick Wendell <pwendell@gmail.com>
> > > wrote:
> > >
> > > > Please vote on releasing the following candidate as Apache Spark
> > version
> > > > 1.2.0!
> > > >
> > > > The tag to be voted on is v1.2.0-rc2 (commit a428c446e2):
> > > >
> > > >
> > >
> > https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=
> a428c446e23e628b746e0626cc02b7b3cadf588e
> > > >
> > > > The release files, including signatures, digests, etc. can be found
> at:
> > > > http://people.apache.org/~pwendell/spark-1.2.0-rc2/
> > > >
> > > > Release artifacts are signed with the following key:
> > > > https://people.apache.org/keys/committer/pwendell.asc
> > > >
> > > > The staging repository for this release can be found at:
> > > >
> > https://repository.apache.org/content/repositories/orgapachespark-1055/
> > > >
> > > > The documentation corresponding to this release can be found at:
> > > > http://people.apache.org/~pwendell/spark-1.2.0-rc2-docs/
> > > >
> > > > Please vote on releasing this package as Apache Spark 1.2.0!
> > > >
> > > > The vote is open until Saturday, December 13, at 21:00 UTC and passes
> > > > if a majority of at least 3 +1 PMC votes are cast.
> > > >
> > > > [ ] +1 Release this package as Apache Spark 1.2.0
> > > > [ ] -1 Do not release this package because ...
> > > >
> > > > To learn more about Apache Spark, please see
> > > > http://spark.apache.org/
> > > >
> > > > == What justifies a -1 vote for this release? ==
> > > > This vote is happening relatively late into the QA period, so
> > > > -1 votes should only occur for significant regressions from
> > > > 1.0.2. Bugs already present in 1.1.X, minor
> > > > regressions, or bugs related to new features will not block this
> > > > release.
> > > >
> > > > == What default changes should I be aware of? ==
> > > > 1. The default value of "spark.shuffle.blockTransferService" has
> been
> > > > changed to "netty"
> > > > --> Old behavior can be restored by switching to "nio"
> > > >
> > > > 2. The default value of "spark.shuffle.manager" has been changed to
> > > "sort".
> > > > --> Old behavior can be restored by setting "spark.shuffle.manager"
> to
> > > > "hash".
> > > >
> > > > == How does this differ from RC1 ==
> > > > This has fixes for a handful of issues identified - some of the
> > > > notable fixes are:
> > > >
> > > > [Core]
> > > > SPARK-4498: Standalone Master can fail to recognize completed/failed
> > > > applications
> > > >
> > > > [SQL]
> > > > SPARK-4552: Query for empty parquet table in spark sql hive get
> > > > IllegalArgumentException
> > > > SPARK-4753: Parquet2 does not prune based on OR filters on partition
> > > > columns
> > > > SPARK-4761: With JDBC server, set Kryo as default serializer and
> > > > disable reference tracking
> > > > SPARK-4785: When called with arguments referring column fields, PMOD
> > > > throws NPE
> > > >
> > > > - Patrick
> > > >
> > > > ------------------------------------------------------------
> ---------
> > > > To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> > <javascript:;>
> > > > For additional commands, e-mail: dev-help@spark.apache.org
> > > <javascript:;>
> > > >
> > > >
> > >
> >
>

   
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message