From dev-return-28131-archive-asf-public=cust-asf.ponee.io@spark.apache.org Thu May 30 16:22:27 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 6B50F180638 for ; Thu, 30 May 2019 18:22:27 +0200 (CEST) Received: (qmail 27463 invoked by uid 500); 30 May 2019 16:22:05 -0000 Mailing-List: contact dev-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list dev@spark.apache.org Received: (qmail 27221 invoked by uid 99); 30 May 2019 16:22:05 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 30 May 2019 16:22:05 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 7B2A01A32BF; Thu, 30 May 2019 16:22:04 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.477 X-Spam-Level: * X-Spam-Status: No, score=1.477 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.324, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id VV-ywSUZHcQR; Thu, 30 May 2019 16:22:03 +0000 (UTC) Received: from mail-vk1-f193.google.com (mail-vk1-f193.google.com [209.85.221.193]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id E55635F366; Thu, 30 May 2019 16:22:02 +0000 (UTC) Received: by mail-vk1-f193.google.com with SMTP id t204so781380vke.4; Thu, 30 May 2019 09:22:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=v1Xyt75mgKccgPnPey2qiKt7jiLhQvCXVVRuskHkfzQ=; b=k1fPPfevlg8fxBNQ4Ik1AOu0rDCYgFa/4TRAke0YEvkfY0rV6VOjpPZqQuZZAriGfT ROsxxa4d0HZqnMRuGbFigXK1MjzqoCdwrbxDxjNAFXROrLcjCL70KLGpCj5I1JxkKAs7 GxUwlIJ9m9cDXfcHxIG5CTRXqK0MN3SOAlbubl1F6s0WJIlYY6DwmFQR6culrVoim/LJ u/SyHD9faGnoOISzu4yGAKT8hMIOGFPZrI2hwOs4oqqoNJBav+Ooy8yF12xed2KJSsNN lImnzI62Dzbpnsh9af5rSZlYmY3zdUeREc4iqSErVSKdpMMSHEyeMr1oxJZTftwG7MXB VK6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=v1Xyt75mgKccgPnPey2qiKt7jiLhQvCXVVRuskHkfzQ=; b=tw3QDKcL10hwMC6KI5vz/THJaEkk29F9Xa5Jkd1LsfGA4pU5ZTWe19QvlFfgYu22NB +qGTGWRhpb6GtHLpvYRCrA0gSsw8X9nQA1+fNojMrEkhrdfzhGJVVHdnFoA/hUOwox5F gh2sb15Ije/ThpU9V1kr7PQU2dgQe7LmAiHE2Ge2ptB1fpJXykAc+tha6CvQtXj2lekc HLko8Jz8zpUxXtYc7C31Fhc1nXCNj2U7UzaD1kRuuBHzW3g5/vDKEQEdDEeRv0cv8UWf UMwAs8ylQVank5PWfwHpmCIf6rqc8u08bjmHBf8DrrdBt1yIciexNKf2vv/huV3ocUX4 cDVg== X-Gm-Message-State: APjAAAVzsuvB1y28pSnCKd0h3PCRhjyc+tzofCUl/QhXqYsyg65NPHB9 t04D2IkzY0h8CpG79N2w/RXxGAJKBmao0LCg1qY= X-Google-Smtp-Source: APXvYqxmWTvIDtJTvCxIzQ7fp8Fk4hucpORtTV4Tm7mECAGF/b/ccFBuygrXgasKg08lAYDw9jcYzR86D+gDR1oklpg= X-Received: by 2002:a1f:6c6:: with SMTP id 189mr1957018vkg.46.1559233322310; Thu, 30 May 2019 09:22:02 -0700 (PDT) MIME-Version: 1.0 References: <24FB8BDE-8ABC-4EE4-9E4D-B61EF566F016@gmail.com> In-Reply-To: From: Xiangrui Meng Date: Thu, 30 May 2019 09:21:51 -0700 Message-ID: Subject: Re: Should python-2 be supported in Spark 3.0? To: Felix Cheung Cc: Reynold Xin , shane knapp , Erik Erlandson , Mark Hamstra , Matei Zaharia , Sean Owen , Wenchen Fen , dev , user Content-Type: multipart/alternative; boundary="0000000000004c5fdc058a1d4cec" --0000000000004c5fdc058a1d4cec Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I created https://issues.apache.org/jira/browse/SPARK-27884 to track the work. On Thu, May 30, 2019 at 2:18 AM Felix Cheung wrote: > We don=E2=80=99t usually reference a future release on website > > > Spark website and state that Python 2 is deprecated in Spark 3.0 > > I suspect people will then ask when is Spark 3.0 coming out then. Might > need to provide some clarity on that. > We can say the "next major release in 2019" instead of Spark 3.0. Spark 3.0 timeline certainly requires a new thread to discuss. > > > ------------------------------ > *From:* Reynold Xin > *Sent:* Thursday, May 30, 2019 12:59:14 AM > *To:* shane knapp > *Cc:* Erik Erlandson; Mark Hamstra; Matei Zaharia; Sean Owen; Wenchen > Fen; Xiangrui Meng; dev; user > *Subject:* Re: Should python-2 be supported in Spark 3.0? > > +1 on Xiangrui=E2=80=99s plan. > > On Thu, May 30, 2019 at 7:55 AM shane knapp wrote: > >> I don't have a good sense of the overhead of continuing to support >>> Python 2; is it large enough to consider dropping it in Spark 3.0? >>> >>> from the build/test side, it will actually be pretty easy to continue >> support for python2.7 for spark 2.x as the feature sets won't be expandi= ng. >> > >> that being said, i will be cracking a bottle of champagne when i can >> delete all of the ansible and anaconda configs for python2.x. :) >> > On the development side, in a future release that drops Python 2 support we can remove code that maintains python 2/3 compatibility and start using python 3 only features, which is also quite exciting. > >> shane >> -- >> Shane Knapp >> UC Berkeley EECS Research / RISELab Staff Technical Lead >> https://rise.cs.berkeley.edu >> > --0000000000004c5fdc058a1d4cec Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

On Thu, May 30, 2019 at 2:18 AM Felix Cheung <= felixcheung_m@hotmail.com&= gt; wrote:
We don=E2=80=99t usually reference a future re= lease on website

> Spark website and state that Python 2 is = deprecated in Spark 3.0

I suspect people will then ask when is Spark 3= .0 coming out then. Might need to provide some clarity on that.
=

We can say the "next majo= r release in 2019" instead of Spark 3.0. Spark 3.0 timeline certainly = requires a new thread to discuss.
=C2=A0


From= : Reynold Xin <rxin@databricks.com>
Sent: Thursday, May 30, 2019 12:59:14 AM
To: shane knapp
Cc: Erik Erlandson; Mark Hamstra; Matei Zaharia; Sean Owen; Wenchen = Fen; Xiangrui Meng; dev; user
Subject: Re: Should python-2 be supported in Spark 3.0?
=C2=A0
+1 on Xiangrui=E2=80=99s plan.

On Thu, May 30, 2019 at 7:55 AM shane= knapp <sknapp@= berkeley.edu> wrote:
I don't have a good sense of the overhead of continuing to support
Python 2; is it large enough to consider dropping it in Spark 3.0?

from the build/test side, it will actually be pretty easy to continue = support for python2.7 for spark 2.x as the feature sets won't be expand= ing.=C2=A0

that being said, i will be cracking a bottle of champagne when i can d= elete all of the ansible and anaconda configs for python2.x.=C2=A0 :)
=

On the development side, in a future release that drops Python 2 sup= port we can remove code that maintains python 2/3 compatibility and start u= sing python 3 only features, which is also quite exciting.
= =C2=A0

shane
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
--0000000000004c5fdc058a1d4cec--