spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sameer Agarwal <sameer.a...@gmail.com>
Subject Re: [VOTE] Spark 2.3.0 (RC1)
Date Tue, 16 Jan 2018 23:00:00 GMT
Yes, I'll cut an RC2 as soon as the remaining blockers are resolved. In the
meantime, please continue to report any other issues here.

Here's a quick update on progress towards the next RC:

- SPARK-22908 <https://issues.apache.org/jira/browse/SPARK-22908>
(KafkaContiniousSourceSuite) has been reverted
- SPARK-23051 <https://issues.apache.org/jira/browse/SPARK-23051> (Spark
UI), SPARK-23063 <https://issues.apache.org/jira/browse/SPARK-23063> (k8s
packaging) and SPARK-23065
<https://issues.apache.org/jira/browse/SPARK-23065> (R API docs) have all
been resolved
- A fix for SPARK-23020 <https://issues.apache.org/jira/browse/SPARK-23020>
(SparkLauncherSuite) has been merged. We're monitoring the builds to make
sure that the flakiness has been resolved.



On 16 January 2018 at 13:21, Ted Yu <yuzhihong@gmail.com> wrote:

> Is there going to be another RC ?
>
> With KafkaContinuousSourceSuite hanging, it is hard to get the rest of
> the tests going.
>
> Cheers
>
> On Sat, Jan 13, 2018 at 7:29 AM, Sean Owen <sowen@cloudera.com> wrote:
>
>> The signatures and licenses look OK. Except for the missing k8s package,
>> the contents look OK. Tests look pretty good with "-Phive -Phadoop-2.7
>> -Pyarn" on Ubuntu 17.10, except that KafkaContinuousSourceSuite seems to
>> hang forever. That was just fixed and needs to get into an RC?
>>
>> Aside from the Blockers just filed for R docs, etc., we have:
>>
>> Blocker:
>> SPARK-23000 Flaky test suite DataSourceWithHiveMetastoreCatalogSuite in
>> Spark 2.3
>> SPARK-23020 Flaky Test: org.apache.spark.launcher.Spar
>> kLauncherSuite.testInProcessLauncher
>> SPARK-23051 job description in Spark UI is broken
>>
>> Critical:
>> SPARK-22739 Additional Expression Support for Objects
>>
>> I actually don't think any of those Blockers should be Blockers; not sure
>> if the last one is really critical either.
>>
>> I think this release will have to be re-rolled so I'd say -1 to RC1.
>>
>> On Fri, Jan 12, 2018 at 4:42 PM Sameer Agarwal <sameerag@apache.org>
>> wrote:
>>
>>> Please vote on releasing the following candidate as Apache Spark version
>>> 2.3.0. The vote is open until Thursday January 18, 2018 at 8:00:00 am UTC
>>> and passes if a majority of at least 3 PMC +1 votes are cast.
>>>
>>>
>>> [ ] +1 Release this package as Apache Spark 2.3.0
>>>
>>> [ ] -1 Do not release this package because ...
>>>
>>>
>>> To learn more about Apache Spark, please see https://spark.apache.org/
>>>
>>> The tag to be voted on is v2.3.0-rc1: https://github.com/apache/spar
>>> k/tree/v2.3.0-rc1 (964cc2e31b2862bca0bd968b3e9e2cbf8d3ba5ea)
>>>
>>> List of JIRA tickets resolved in this release can be found here:
>>> https://issues.apache.org/jira/projects/SPARK/versions/12339551
>>>
>>> The release files, including signatures, digests, etc. can be found at:
>>> https://dist.apache.org/repos/dist/dev/spark/v2.3.0-rc1-bin/
>>>
>>> Release artifacts are signed with the following key:
>>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>>
>>> The staging repository for this release can be found at:
>>> https://repository.apache.org/content/repositories/orgapachespark-1261/
>>>
>>> The documentation corresponding to this release can be found at:
>>> https://dist.apache.org/repos/dist/dev/spark/v2.3.0-rc1-docs
>>> /_site/index.html
>>>
>>>
>>> FAQ
>>>
>>> =========================
>>> How can I help test this release?
>>> =========================
>>>
>>> If you are a Spark user, you can help us test this release by taking an
>>> existing Spark workload and running on this release candidate, then
>>> reporting any regressions.
>>>
>>> If you're working in PySpark you can set up a virtual env and install
>>> the current RC and see if anything important breaks, in the Java/Scala you
>>> can add the staging repository to your projects resolvers and test with the
>>> RC (make sure to clean up the artifact cache before/after so you don't end
>>> up building with a out of date RC going forward).
>>>
>>> ===========================================
>>> What should happen to JIRA tickets still targeting 2.3.0?
>>> ===========================================
>>>
>>> Committers should look at those and triage. Extremely important bug
>>> fixes, documentation, and API tweaks that impact compatibility should be
>>> worked on immediately. Everything else please retarget to 2.3.1 or 2.3.0 as
>>> appropriate.
>>>
>>> ==================
>>> But my bug isn't fixed?
>>> ==================
>>>
>>> In order to make timely releases, we will typically not hold the release
>>> unless the bug in question is a regression from 2.2.0. That being said, if
>>> there is something which is a regression from 2.2.0 that has not been
>>> correctly targeted please ping me or a committer to help target the issue
>>> (you can see the open issues listed as impacting Spark 2.3.0 at
>>> https://s.apache.org/WmoI).
>>>
>>> =======================================
>>> What are the unresolved issues targeted for 2.3.0?
>>> =======================================
>>>
>>> Please see https://s.apache.org/oXKi. At the time of the writing, there
>>> are 19 JIRA issues targeting 2.3.0 tracking various QA/audit tasks, test
>>> failures and other feature/bugs. In particular, we've currently marked 3
>>> JIRAs as release blockers that are being actively worked on:
>>>
>>> 1. SPARK-23051 that tracks a regression in the Spark UI
>>> 2. SPARK-23020 and SPARK-23000 that track a couple of flaky tests that
>>> are responsible for build failures. Additionally,
>>> https://github.com/apache/spark/pull/20242 fixes a few Java linter
>>> errors in RC1.
>>>
>>> Given that these blockers are fairly isolated, in the sprit of starting
>>> a thorough QA early, this RC1 aims to serve as a good approximation of the
>>> functionality of final release.
>>>
>>> Regards,
>>> Sameer
>>>
>>
>


-- 
Sameer Agarwal
Computer Science | UC Berkeley
http://cs.berkeley.edu/~sameerag

Mime
View raw message