Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id BA95F200CF9 for ; Sun, 17 Sep 2017 10:31:29 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id B94311609D8; Sun, 17 Sep 2017 08:31:29 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 3BA7C1609BF for ; Sun, 17 Sep 2017 10:31:28 +0200 (CEST) Received: (qmail 38657 invoked by uid 500); 17 Sep 2017 08:31:26 -0000 Mailing-List: contact dev-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list dev@spark.apache.org Received: (qmail 38646 invoked by uid 99); 17 Sep 2017 08:31:26 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 17 Sep 2017 08:31:26 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id B61FF1A0C90 for ; Sun, 17 Sep 2017 08:31:25 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.879 X-Spam-Level: * X-Spam-Status: No, score=1.879 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id sU5HFxLwZ6zz for ; Sun, 17 Sep 2017 08:31:22 +0000 (UTC) Received: from mail-pg0-f67.google.com (mail-pg0-f67.google.com [74.125.83.67]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 3E0F65FDA1 for ; Sun, 17 Sep 2017 08:31:22 +0000 (UTC) Received: by mail-pg0-f67.google.com with SMTP id v5so3560806pgn.4 for ; Sun, 17 Sep 2017 01:31:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=hg0GH+PjODAEdPN7zTRJ1K2v7/Ppexu/e8hzNv6wF5I=; b=jRtM339TDiPL+igqvGEjesgH0Tbseiv69Ja1+ov6P6ZaMoFKK97Yiw/rzHjc7eqD6i i/f6u0LeJFhCBMWxbUXnqynJgX/8LVx4ejXrtlBgKMVPtGhwOzZkFyu5UOsqMp11wd0j hpc/fdeP44l3kKsZrsPQqj5NO35MQc+yy6qMb29hMZEMTUxayhOJIxoz5AU/WR9mQf7f gPFzHhpkJLrjVwKVD+iD0r6ODkzsc2jyxN5jIknmPp2MRhQUwcHNyOQ9FrN2eqhiIqtw ZcxpoYcNmbFnF02RIo4izuDNGC+IA15KYb0WPYA0rQ0v/i0RRUx9yRkGwYY+L/pyw6Xq N2vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=hg0GH+PjODAEdPN7zTRJ1K2v7/Ppexu/e8hzNv6wF5I=; b=mqYctMCm+BChMQAG+q4qJ3gX9SAj6b1kdr6oZZURYxMrBfwM99py903Nrw2DKLboTu HPXhhiHi1uybWlgP9IHDF3YFQTtI//9c/hIV+3uhSYo3c2hX4OPIbOAwO0d+mkPk+3q1 8TiKqpAf3a3pjZGI2yNVhPUUaIWEoV6pQ9hdZAyfOzTuiRii+GTNUEFOuWYCPiL9oEu8 //LLHIqHLANrkg0hDpbIj9LvHcMUzIro3YomYNO9mVXp3ypZExlZ9N0hFWK1ocvyFx4I yPtF/wbfyOp3DV2T78v8n037R4Vx0cVAMtQm3xLIUfQj6y5Ns72dz5IM0BzMbuQqQMma 2iJg== X-Gm-Message-State: AHPjjUjH49UcAHS1jzRV//CpHxQQVG1PzzWF3ozXdaERBa5Wd6YNeGID AjCMq2g0uVEctnoTxcUahA== X-Google-Smtp-Source: AOwi7QD0usiR+DzfGso5GrrS74OCgxzIuwW55kviXoab36PU8xhoFN9tPsllH6xopjCQoGHFtu9+AQ== X-Received: by 10.159.198.74 with SMTP id y10mr16694939plt.45.1505637081267; Sun, 17 Sep 2017 01:31:21 -0700 (PDT) Received: from [192.168.0.105] ([116.230.178.20]) by smtp.gmail.com with ESMTPSA id i63sm10201046pfk.34.2017.09.17.01.31.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 17 Sep 2017 01:31:20 -0700 (PDT) From: yuming wang Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_1AE908D5-EFF6-4B76-929B-1FD3AB1D2D78" Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: [VOTE] Spark 2.1.2 (RC1) Date: Sun, 17 Sep 2017 16:30:41 +0800 In-Reply-To: Cc: Xiao Li , Holden Karau , "dev@spark.apache.org" To: Sean Owen References: X-Mailer: Apple Mail (2.3273) archived-at: Sun, 17 Sep 2017 08:31:29 -0000 --Apple-Mail=_1AE908D5-EFF6-4B76-929B-1FD3AB1D2D78 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=gb2312 Yes, It doesn=A1=AFt work in 2.1.0 and 2.1.1, I create a PR for this: = https://github.com/apache/spark/pull/19259 = . > =D4=DA 2017=C4=EA9=D4=C217=C8=D5=A3=AC16:14=A3=ACSean Owen = =D0=B4=B5=C0=A3=BA >=20 > So, didn't work in 2.1.0 or 2.1.1? If it's not a regression and not = critical, it shouldn't block a release. It seems like this can only = affect Docker and/or Oracle JDBC? Well, if we need to roll another = release anyway, seems OK. >=20 > On Sun, Sep 17, 2017 at 6:06 AM Xiao Li > wrote: > This is a bug introduced in 2.1. It works fine in 2.0 >=20 > 2017-09-16 16:15 GMT-07:00 Holden Karau >: > Ok :) Was this working in 2.1.1? >=20 > On Sat, Sep 16, 2017 at 3:59 PM Xiao Li > wrote: > Still -1 >=20 > Unable to pass the tests in my local environment. Open a JIRA = https://issues.apache.org/jira/browse/SPARK-22041 = > - SPARK-16625: General data types to be mapped to Oracle *** FAILED = *** >=20 > types.apply(9).equals(org.apache.spark.sql.types.DateType) was false = (OracleIntegrationSuite.scala:158) >=20 > Xiao >=20 >=20 > 2017-09-15 17:35 GMT-07:00 Ryan Blue >: > -1 (with my Apache member hat on, non-binding) >=20 > I'll continue discussion in the other thread, but I don't think we = should share signing keys. >=20 > On Fri, Sep 15, 2017 at 5:14 PM, Holden Karau > wrote: > Indeed it's limited to a people with login permissions on the Jenkins = host (and perhaps further limited, I'm not certain). Shane probably = knows more about the ACLs, so I'll ask him in the other thread for = specifics. >=20 > This is maybe branching a bit from the question of the current RC = though, so I'd suggest we continue this discussion on the thread Sean = Owen made. >=20 > On Fri, Sep 15, 2017 at 4:04 PM Ryan Blue > wrote: > I'm not familiar with the release procedure, can you send a link to = this Jenkins job? Can anyone run this job, or is it limited to = committers? >=20 > rb >=20 > On Fri, Sep 15, 2017 at 12:28 PM, Holden Karau > wrote: > That's a good question, I built the release candidate however the = Jenkins scripts don't take a parameter for configuring who signs them = rather it always signs them with Patrick's key. You can see this from = previous releases which were managed by other folks but still signed by = Patrick. >=20 > On Fri, Sep 15, 2017 at 12:16 PM, Ryan Blue > wrote: > The signature is valid, but why was the release signed with Patrick = Wendell's private key? Did Patrick build the release candidate? >=20 > rb >=20 > On Fri, Sep 15, 2017 at 6:36 AM, Denny Lee > wrote: > +1 (non-binding) >=20 > On Thu, Sep 14, 2017 at 10:57 PM Felix Cheung = > wrote: > +1 tested SparkR package on Windows, r-hub, Ubuntu. >=20 > _____________________________ > From: Sean Owen > > Sent: Thursday, September 14, 2017 3:12 PM > Subject: Re: [VOTE] Spark 2.1.2 (RC1) > To: Holden Karau >, = > >=20 >=20 >=20 > +1 > Very nice. The sigs and hashes look fine, it builds fine for me on = Debian Stretch with Java 8, yarn/hive/hadoop-2.7 profiles, and passes = tests.=20 >=20 > Yes as you say, no outstanding issues except for this which doesn't = look critical, as it's not a regression. >=20 > SPARK-21985 PySpark PairDeserializer is broken for double-zipped RDDs >=20 >=20 > On Thu, Sep 14, 2017 at 7:47 PM Holden Karau > wrote: > Please vote on releasing the following candidate as Apache Spark = version 2.1.2. The vote is open until Friday September 22nd at 18:00 PST = and passes if a majority of at least 3 +1 PMC votes are cast. >=20 > [ ] +1 Release this package as Apache Spark 2.1.2 > [ ] -1 Do not release this package because ... >=20 >=20 > To learn more about Apache Spark, please see https://spark.apache.org/ = >=20 > The tag to be voted on is v2.1.2-rc1 = = (6f470323a0363656999dd36cb33f528afe627c12) >=20 > List of JIRA tickets resolved in this release can be found with this = filter. = >=20 > The release files, including signatures, digests, etc. can be found = at: > https://home.apache.org/~pwendell/spark-releases/spark-2.1.2-rc1-bin/ = >=20 > Release artifacts are signed with the following key: > https://people.apache.org/keys/committer/pwendell.asc = >=20 > The staging repository for this release can be found at: > = https://repository.apache.org/content/repositories/orgapachespark-1248/ = >=20 > The documentation corresponding to this release can be found at: > = https://people.apache.org/~pwendell/spark-releases/spark-2.1.2-rc1-docs/ = = >=20 >=20 > FAQ >=20 > How can I help test this release? >=20 > If you are a Spark user, you can help us test this release by taking = an existing Spark workload and running on this release candidate, then = reporting any regressions. >=20 > If you're working in PySpark you can set up a virtual env and install = the current RC and see if anything important breaks, in the Java/Scala = you can add the staging repository to your projects resolvers and test = with the RC (make sure to clean up the artifact cache before/after so = you don't end up building with a out of date RC going forward). >=20 > What should happen to JIRA tickets still targeting 2.1.2? >=20 > Committers should look at those and triage. Extremely important bug = fixes, documentation, and API tweaks that impact compatibility should be = worked on immediately. Everything else please retarget to 2.1.3. >=20 > But my bug isn't fixed!??! >=20 > In order to make timely releases, we will typically not hold the = release unless the bug in question is a regression from 2.1.1. That = being said if there is something which is a regression form 2.1.1 that = has not been correctly targeted please ping a committer to help target = the issue (you can see the open issues listed as impacting Spark 2.1.1 & = 2.1.2 = ) >=20 > What are the unresolved issues targeted for 2.1.2 = ? >=20 > At the time of the writing, there is one in progress major issue = SPARK-21985 , I = believe Andrew Ray & HyukjinKwon are looking into this one. >=20 > --=20 > Twitter: https://twitter.com/holdenkarau = >=20 >=20 >=20 >=20 >=20 > --=20 > Ryan Blue > Software Engineer > Netflix >=20 >=20 >=20 > --=20 > Twitter: https://twitter.com/holdenkarau = >=20 >=20 >=20 > --=20 > Ryan Blue > Software Engineer > Netflix > --=20 > Twitter: https://twitter.com/holdenkarau = >=20 >=20 >=20 > --=20 > Ryan Blue > Software Engineer > Netflix >=20 > --=20 > Twitter: https://twitter.com/holdenkarau = >=20 --Apple-Mail=_1AE908D5-EFF6-4B76-929B-1FD3AB1D2D78 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=gb2312
Yes, It doesn=A1=AFt work in 2.1.0 and 2.1.1, = I create a PR for this: https://github.com/apache/spark/pull/19259.


=D4=DA = 2017=C4=EA9=D4=C217=C8=D5=A3=AC16:14=A3=ACSean Owen <sowen@cloudera.com> = =D0=B4=B5=C0=A3=BA

So, didn't work in 2.1.0 or = 2.1.1? If it's not a regression and not critical, it shouldn't block a = release. It seems like this can only affect Docker and/or Oracle JDBC? = Well, if we need to roll another release anyway, seems OK.

On Sun, Sep 17, 2017 at 6:06 AM Xiao Li <gatorsmile@gmail.com> wrote:
This is a bug introduced in 2.1. It works fine in = 2.0

2017-09-16 16:15 GMT-07:00 Holden Karau <holden@pigscanfly.ca>:
Ok :) Was this working in 2.1.1?

On Sat, Sep 16, 2017 at 3:59 PM = Xiao Li <gatorsmile@gmail.com> wrote:
Still = -1

Unable to pass = the tests in my local environment. Open a JIRA https://issues.apache.org/jira/browse/SPARK-22041

- SPARK-16625: General data types to be mapped to Oracle *** = FAILED ***

  = types.apply(9).equals(org.apache.spark.sql.types.DateType) was = false = (OracleIntegrationSuite.scala:158)

Xiao


2017-09-15 17:35 GMT-07:00 Ryan = Blue <rblue@netflix.com.invalid>:
-1 = (with my Apache member hat on, non-binding)

I'll continue discussion in the other = thread, but I don't think we should share signing keys.

On = Fri, Sep 15, 2017 at 5:14 PM, Holden Karau <holden@pigscanfly.ca> wrote:
Indeed it's limited to a people = with login permissions on the Jenkins host (and perhaps further limited, = I'm not certain). Shane probably knows more about the ACLs, so I'll ask = him in the other thread for specifics.

This is = maybe branching a bit from the question of the current RC though, so I'd = suggest we continue this discussion on the thread Sean Owen = made.

On Fri, Sep 15, 2017 at 4:04 PM = Ryan Blue <rblue@netflix.com> wrote:
I'm = not familiar with the release procedure, can you send a link to this = Jenkins job? Can anyone run this job, or is it limited to = committers?

rb

On Fri, Sep 15, 2017 at 12:28 PM, Holden Karau = <holden@pigscanfly.ca> = wrote:
That's a good question, I built the release candidate however = the Jenkins scripts don't take a parameter for configuring who signs = them rather it always signs them with Patrick's key. You can see this = from previous releases which were managed by other folks but still = signed by Patrick.

On Fri, Sep 15, = 2017 at 12:16 PM, Ryan Blue <rblue@netflix.com> wrote:
The = signature is valid, but why was the release signed with Patrick = Wendell's private key? Did Patrick build the release candidate?

rb

On Fri, Sep 15, 2017 at 6:36 AM, Denny Lee <denny.g.lee@gmail.com> wrote:
+1 = (non-binding)

On Thu, Sep 14, = 2017 at 10:57 PM Felix Cheung <felixcheung_m@hotmail.com> wrote:
+1 tested SparkR package on = Windows, r-hub, Ubuntu.

_____________________________
From: Sean Owen <sowen@cloudera.com>
Sent: Thursday, September 14, 2017 3:12 PM
Subject: Re: [VOTE] Spark 2.1.2 (RC1)
To: Holden Karau <holden@pigscanfly.ca>, <dev@spark.apache.org>



+1
Very nice. The sigs and hashes look fine, it builds fine for me on = Debian Stretch with Java 8, yarn/hive/hadoop-2.7 profiles, and passes = tests. 

Yes as you say, no outstanding issues except for this = which doesn't look critical, as it's not a regression.

SPARK-21985 PySpark PairDeserializer is broken for = double-zipped RDDs


On Thu, Sep 14, 2017 at 7:47 PM Holden Karau <holden@pigscanfly.ca> wrote:
Please vote on releasing the following candidate as Apache Spark version 2.1.2. = The vote is open until Friday September 22nd at 18:00 PST and passes if a = majority of at least 3 +1 PMC votes are cast.

[ ] +1 Release this package = as Apache Spark 2.1.2
[ ] -1 Do not release this = package because ...


To learn more about Apache = Spark, please see https://spark.apache.org/

The tag to = be voted on is v2.1.2-rc1 (6f470323a0363656999dd36cb33f528afe627c12)

List of JIRA tickets resolved = in this release can be found with this filter.

The release files, including = signatures, digests, etc. can be found at:
https://home.apache.org/~pwendell/spark-releases/spark-2.1.2-rc= 1-bin/

Release artifacts are signed = with the following key:
https://people.apache.org/keys/committer/pwendell.asc
=

The staging repository for = this release can be found at:
https://repository.apache.org/content/repositories/orgapachespa= rk-1248/

The documentation = corresponding to this release can be found at:
https://people.apache.org/~pwendell/spark-releases/spark-2.1.2-= rc1-docs/


FAQ

How can I help = test this release?

If you are a Spark user, you = can help us test this release by taking an existing Spark workload and = running on this release candidate, then reporting any regressions.

If you're working in PySpark = you can set up a virtual env and install the current RC and see if = anything important breaks, in the Java/Scala you can add the staging = repository to your projects resolvers and test with the RC (make sure to clean up the artifact cache before/after so you don't end up = building with a out of date RC going forward).

What should = happen to JIRA tickets still targeting 2.1.2?

Committers should look at = those and triage. Extremely important bug fixes, documentation, and API = tweaks that impact compatibility should be worked on immediately. = Everything else please retarget to 2.1.3.

But my bug = isn't fixed!??!

In order to make timely = releases, we will typically not hold the release unless the bug in = question is a regression from 2.1.1. That being said if there is = something which is a regression form 2.1.1 that has not been correctly targeted please ping a committer to help target the issue (you can see = the open issues listed as impacting Spark 2.1.1 & 2.1.2)

What are the unresolved issues targeted for 2.1.2?

At the time of the writing, = there is one in progress major issue SPARK-21985, I believe Andrew Ray & = HyukjinKwon are looking into this one.

--
Twitter: https://twitter.com/holdenkarau





--
Ryan Blue
Software Engineer
Netflix



--
Twitter: https://twitter.com/holdenkarau



--
Ryan Blue
Software Engineer
Netflix
--
Twitter: https://twitter.com/holdenkarau



--
Ryan Blue
Software Engineer
Netflix

--
Twitter: https://twitter.com/holdenkarau


= --Apple-Mail=_1AE908D5-EFF6-4B76-929B-1FD3AB1D2D78--