Return-Path: 
 <user-return-538-apmail-crunch-user-archive=crunch.apache.org@crunch.apache.org>
X-Original-To: apmail-crunch-user-archive@www.apache.org
Delivered-To: apmail-crunch-user-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id EA9AD10A21
	for <apmail-crunch-user-archive@www.apache.org>;
 Tue, 29 Apr 2014 05:30:52 +0000 (UTC)
Received: (qmail 84369 invoked by uid 500); 29 Apr 2014 05:30:51 -0000
Delivered-To: apmail-crunch-user-archive@crunch.apache.org
Received: (qmail 84283 invoked by uid 500); 29 Apr 2014 05:30:50 -0000
Mailing-List: contact user-help@crunch.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@crunch.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@crunch.apache.org>
List-Post: <mailto:user@crunch.apache.org>
List-Id: <user.crunch.apache.org>
Reply-To: user@crunch.apache.org
Delivered-To: mailing list user@crunch.apache.org
Received: (qmail 84268 invoked by uid 99); 29 Apr 2014 05:30:50 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Apr 2014 05:30:50 +0000
X-ASF-Spam-Status: No, hits=2.2 required=5.0
	tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of somsatpathy@gmail.com
 designates 209.85.192.46 as permitted sender)
Received: from [209.85.192.46] (HELO mail-qg0-f46.google.com) (209.85.192.46)
    by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Apr 2014 05:30:46 +0000
Received: by mail-qg0-f46.google.com with SMTP id q107so7733616qgd.5
        for <multiple recipients>; Mon, 28 Apr 2014 22:30:23 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :cc:content-type;
        bh=qBt7CnVIR4WPJ7ffSkw/gS3gl9/XSXiJEelBjpvx6Co=;
        b=ysj+xyFEujhkbQtKjGrAfPdNuPx06rrgfD24d/tNzifu9gT+wUGxfdAjCLPubpd4F7
         XMTHm0+FVWOHbtWO1zZQ1el5K9HdkuQWDu5dOW29Hkqhqw45hloQr8zkTHoNlZJfrzTt
         vcc4i/M2VT40k0JbVwQ+YmEwkLlZgu1Ko1Tgh/lcrHWG7oHi3vfFa7sjuCVKgsjQttSO
         Zy368OMvZot+OKr9P0KgIMtfMkOMLx1wq+tyrmGiIvXDQ3bOq7l4NshzXMDl3MgJnFvN
         /QgbvsAB7wceGqajpYgxyvehwAFhEVI5/huxU7qx0AvsawCtYv+CAI59o20F8OujYN97
         cXUQ==
MIME-Version: 1.0
X-Received: by 10.224.147.77 with SMTP id k13mr39376248qav.64.1398749423211;
 Mon, 28 Apr 2014 22:30:23 -0700 (PDT)
Received: by 10.140.94.73 with HTTP; Mon, 28 Apr 2014 22:30:23 -0700 (PDT)
In-Reply-To: 
 <CAH29n6M6VWns+wA4+LJ_-6jbPZjwEZwyyvaH0o=VG6nesM932Q@mail.gmail.com>
References: 
 <CAH29n6M6VWns+wA4+LJ_-6jbPZjwEZwyyvaH0o=VG6nesM932Q@mail.gmail.com>
Date: Mon, 28 Apr 2014 22:30:23 -0700
Message-ID: 
 <CAHCsPn-pHqqcqXg2TDrefq32PW=qpCKAX5Fn6R20y2MOB6mgYA@mail.gmail.com>
Subject: Re: spark/scala version changes for 0.10.0/0.8.3
From: Som Satpathy <somsatpathy@gmail.com>
To: "user@crunch.apache.org" <user@crunch.apache.org>
Cc: dev <dev@crunch.apache.org>
Content-Type: multipart/alternative; boundary=089e0149cda0801a8404f827ba28
X-Virus-Checked: Checked by ClamAV on apache.org

--089e0149cda0801a8404f827ba28
Content-Type: text/plain; charset=UTF-8

+1 for option 2.

Thanks,
Som


On Fri, Apr 25, 2014 at 1:45 PM, Josh Wills <jwills@cloudera.com> wrote:

> Hey all,
>
> I've been working on doing a new release candidate, and I think that we're
> going to need to upgrade to Spark 0.9.1 in order for Crunch-on-Spark to
> work in Spark standalone/Spark-on-YARN on top of Hadoop, as opposed to the
> local mode that works now. There are a couple of implications to this, but
> the big one is that Spark 0.9.1 is only developed against Scala 2.10, which
> would mean that we would need to switch over to 2.10 for at least
> crunch-spark, and it would seem much easier to me for us to upgrade Scrunch
> to 2.10 as well.
>
> Couple of options here off the top of my head:
> 1) Do the upgrade to Scala 2.10/Spark 0.9.1 in 0.10.0, but not 0.8.3.
> 2) Do the upgrade in both 0.8.3 and 0.10.0.
> 3) Only upgrade Spark to 2.10 (in either 0.8.3 or 0.10.0), but still have
> Scrunch build against 2.9.3.
>
> I'm in favor of #2 myself, but am open to suggestion here, and would
> especially like to know if anyone feels strongly about staying on 2.9.3 for
> any reason.
>
> Thanks!
> Josh
>
> Director of Data Science
> Cloudera <http://www.cloudera.com>
> Twitter: @josh_wills <http://twitter.com/josh_wills>
>

--089e0149cda0801a8404f827ba28
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">+1 for option 2.<div><br></div><div>Thanks,</div><div>Som<=
/div></div><div class=3D"gmail_extra"><br><br><div class=3D"gmail_quote">On=
 Fri, Apr 25, 2014 at 1:45 PM, Josh Wills <span dir=3D"ltr">&lt;<a href=3D"=
mailto:jwills@cloudera.com" target=3D"_blank">jwills@cloudera.com</a>&gt;</=
span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div dir=3D"ltr">Hey all,<div><br></div><div=
>I&#39;ve been working on doing a new release candidate, and I think that w=
e&#39;re going to need to upgrade to Spark 0.9.1 in order for Crunch-on-Spa=
rk to work in Spark standalone/Spark-on-YARN on top of Hadoop, as opposed t=
o the local mode that works now. There are a couple of implications to this=
, but the big one is that Spark 0.9.1 is only developed against Scala 2.10,=
 which would mean that we would need to switch over to 2.10 for at least cr=
unch-spark, and it would seem much easier to me for us to upgrade Scrunch t=
o 2.10 as well.</div>


<div><br></div><div>Couple of options here off the top of my head:</div><di=
v>1) Do the upgrade to Scala 2.10/Spark 0.9.1 in 0.10.0, but not 0.8.3.</di=
v><div>2) Do the upgrade in both 0.8.3 and 0.10.0.</div><div>3) Only upgrad=
e Spark to 2.10 (in either 0.8.3 or 0.10.0), but still have Scrunch build a=
gainst 2.9.3.</div>


<div><br></div><div>I&#39;m in favor of #2 myself, but am open to suggestio=
n here, and would especially like to know if anyone feels strongly about st=
aying on 2.9.3 for any reason.</div><div><br></div><div>Thanks!<br>Josh</di=
v>


<div><br><div>Director of Data Science</div><div><a href=3D"http://www.clou=
dera.com" target=3D"_blank">Cloudera</a></div><div>Twitter: <a href=3D"http=
://twitter.com/josh_wills" target=3D"_blank">@josh_wills</a></div>
</div></div>
</blockquote></div><br></div>

--089e0149cda0801a8404f827ba28--