spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Hamstra <m...@clearstorydata.com>
Subject Re: Release date for new pyspark
Date Wed, 16 Jul 2014 23:21:19 GMT
You should expect master to compile and run: patches aren't merged unless
they build and pass tests on Jenkins.

You shouldn't expect new features to be added to stable code in maintenance
releases (e.g. 1.0.1).

AFAIK, we're still on track with Spark 1.1.0 development, which means that
it should be released sometime in the second half of next month (or shortly
thereafter).


On Wed, Jul 16, 2014 at 4:03 PM, Paul Wais <pwais@yelp.com> wrote:

> Dear List,
>
> The version of pyspark on master has a lot of nice new features, e.g.
> SequenceFile reading, pickle i/o, etc:
> https://github.com/apache/spark/blob/master/python/pyspark/context.py#L353
>
> I downloaded the recent 1.0.1 release and was surprised to see the
> distribution did not include these changes in master.  (I've tried pulling
> master [ 9c249743ea ] and compiling from source, but I get a build failure
> in TestSQLContext.scala FWIW).
>
> Is an updated pyspark scheduled for the next release?  (Also, am I wrong
> in expecting HEAD on master should probably compile and run?)
>
> Best Regards,
> -Paul Wais
>

Mime
View raw message