Return-Path: X-Original-To: apmail-spark-dev-archive@minotaur.apache.org Delivered-To: apmail-spark-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 97B9018072 for ; Thu, 9 Jul 2015 15:49:15 +0000 (UTC) Received: (qmail 39328 invoked by uid 500); 9 Jul 2015 15:49:13 -0000 Delivered-To: apmail-spark-dev-archive@spark.apache.org Received: (qmail 39242 invoked by uid 500); 9 Jul 2015 15:49:13 -0000 Mailing-List: contact dev-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list dev@spark.apache.org Received: (qmail 39221 invoked by uid 99); 9 Jul 2015 15:49:13 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 09 Jul 2015 15:49:13 +0000 X-ASF-Spam-Status: No, hits=1.5 required=10.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of yuzhihong@gmail.com designates 209.85.160.174 as permitted sender) Received: from [209.85.160.174] (HELO mail-yk0-f174.google.com) (209.85.160.174) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 09 Jul 2015 15:46:59 +0000 Received: by ykeo3 with SMTP id o3so119112388yke.0 for ; Thu, 09 Jul 2015 08:48:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=bTJUd/J4g+booIW5sqIVLl9K73YWdt3dEKS3JXh3FPc=; b=dRhiQfv7l9gNDJ+LkHqJgB6pZLiT1aoxaFF/2kMduVfFJKrimOzTnz5cXyrH6206pu Z3U9Xb/LOadlbsnFkKZJMMDljf3xzM0ergKK6/wIinxvLoIsE8tMyh+yfceJvRaV7UBj uKqL5AzGBcN2Tg2CSTzPIL3j/Dcm73oCnYRyWS0ROGwVEffhwHh+2D1ktnaxFi2lRulM gXIIHdNNMQmHW1lqqmTBXtT5ELemUJd3B1RgsUnTBzYk0hGLvNxWtlUEn2Y/DzUYvtZW PaWE3tpTHMnT7C+oDXmQVdQaU/GHZeIZ4IH1zErXFmu7DotU/IVWfI595HuSaBqq1gu0 ucNg== MIME-Version: 1.0 X-Received: by 10.170.118.80 with SMTP id k77mr18261976ykb.64.1436456926883; Thu, 09 Jul 2015 08:48:46 -0700 (PDT) Received: by 10.37.208.142 with HTTP; Thu, 9 Jul 2015 08:48:46 -0700 (PDT) In-Reply-To: References: Date: Thu, 9 Jul 2015 08:48:46 -0700 Message-ID: Subject: Re: The latest master branch didn't compile with -Phive? From: Ted Yu To: Sean Owen Cc: Yijie Shen , Spark-dev Content-Type: multipart/alternative; boundary=001a1138ea3cdcb701051a7330e9 X-Virus-Checked: Checked by ClamAV on apache.org --001a1138ea3cdcb701051a7330e9 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I guess the compilation issue didn't surface in QA run because sbt was used= : [info] Building Spark (w/Hive 0.13.1) using SBT with these arguments: -Pyarn -Phadoop-2.3 -Dhadoop.version=3D2.3.0 -Pkinesis-asl -Phive-thriftserver -Phive package assembly/assembly streaming-kafka-assembly/assembly streaming-flume-assembly/assembly Cheers On Thu, Jul 9, 2015 at 7:58 AM, Ted Yu wrote: > From > https://amplab.cs.berkeley.edu/jenkins/job/Spark-Master-Maven-with-YARN/H= ADOOP_PROFILE=3Dhadoop-2.4,label=3Dcentos/2875/consoleFull > : > > + build/mvn -DzincPort=3D3439 -DskipTests -Phadoop-2.4 -Pyarn -Phive -Phi= ve-thriftserver -Pkinesis-asl clean package > > > FYI > > > On Thu, Jul 9, 2015 at 7:51 AM, Sean Owen wrote: > >> This is an error from scalac and not Spark. I find it happens >> frequently for me but goes away on a clean build. *shrug* >> >> >> On Thu, Jul 9, 2015 at 3:45 PM, Ted Yu wrote: >> > Looking at >> > >> https://amplab.cs.berkeley.edu/jenkins/job/Spark-Master-Maven-with-YARN/= HADOOP_PROFILE=3Dhadoop-2.4,label=3Dcentos/2875/consoleFull >> > : >> > >> > [error] >> > [error] while compiling: >> > >> /home/jenkins/workspace/Spark-Master-Maven-with-YARN/HADOOP_PROFILE/hado= op-2.4/label/centos/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveM= etastoreCatalog.scala >> > [error] during phase: typer >> > [error] library version: version 2.10.4 >> > [error] compiler version: version 2.10.4 >> > >> > >> > I traced back to build #2869 and the error was there - didn't go back >> > further. >> > >> > >> > FYI >> > >> > >> > On Thu, Jul 9, 2015 at 7:24 AM, Yijie Shen >> > wrote: >> >> >> >> Hi, >> >> >> >> I use the clean version just clone from the master branch, build with= : >> >> >> >> build/mvn -Phive -Phadoop-2.4 -DskipTests package >> >> >> >> And BUILD FAILURE at last, due to: >> >> >> >> [error] while compiling: >> >> >> /Users/yijie/spark/sql/hive/src/main/scala/org/apache/spark/sql/hive/Hiv= eMetastoreCatalog.scala >> >> [error] during phase: typer >> >> [error] library version: version 2.10.4 >> >> [error] compiler version: version 2.10.4 >> >> ... >> >> [error] >> >> [error] last tree to typer: Ident(Warehouse) >> >> [error] symbol: (flags: ) >> >> [error] symbol definition: >> >> [error] symbol owners: >> >> [error] context owners: lazy value hiveWarehouse -> class >> >> HiveMetastoreCatalog -> package hive >> >> [error] >> >> [error] =3D=3D Enclosing template or block =3D=3D >> >> [error] >> >> [error] Template( // val : in >> class >> >> HiveMetastoreCatalog >> >> [error] "Catalog", "Logging" // parents >> >> [error] ValDef( >> >> [error] private >> >> [error] "_" >> >> [error] >> >> [error] >> >> [error] ) >> >> [error] // 24 statements >> >> [error] ValDef( // private[this] val client: >> >> org.apache.spark.sql.hive.client.ClientInterface in class >> >> HiveMetastoreCatalog >> >> [error] private >> >> [error] "client" >> >> [error] "ClientInterface" >> >> [error] >> >> =E2=80=A6 >> >> >> >> >> https://gist.github.com/yijieshen/e0925e2227a312ae4c64#file-build_failur= e >> >> >> >> Did I make a silly mistake? >> >> >> >> Thanks, Yijie >> > >> > >> > > --001a1138ea3cdcb701051a7330e9 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
I guess the compilation issue didn't surface in QA run= because sbt was used:

[info] Building Spark (w/Hive 0.13.1) using SBT with =
these arguments:  -Pyarn -Phadoop-2.3 -Dhadoop.version=3D2.3.0 -Pkinesis-as=
l -Phive-thriftserver -Phive package assembly/assembly streaming-kafka-asse=
mbly/assembly streaming-flume-assembly/assembly

Cheers

On Thu, Jul 9, 2015 at 7:5= 8 AM, Ted Yu <yuzhihong@gmail.com> wrote:
From=C2=A0https://amplab.cs.= berkeley.edu/jenkins/job/Spark-Master-Maven-with-YARN/HADOOP_PROFILE=3Dhado= op-2.4,label=3Dcentos/2875/consoleFull :

+ build/mvn -DzincPort=3D3439 -Dski=
pTests -Phadoop-2.4 -Pyarn -Phive -Phive-thriftserver -Pkinesis-asl clean p=
ackage

FYI

On Thu, Jul 9, 2015 at 7= :51 AM, Sean Owen <sowen@cloudera.com> wrote:
This is an error from scalac and not Spark. I find it = happens
frequently for me but goes away on a clean build. *shrug*


On Thu, Jul 9, 2015 at 3:45 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> Looking at
> https://amplab.cs.berkeley.edu/jenkins/= job/Spark-Master-Maven-with-YARN/HADOOP_PROFILE=3Dhadoop-2.4,label=3Dcentos= /2875/consoleFull
> :
>
> [error]
> [error]=C2=A0 =C2=A0 =C2=A0 while compiling:
> /home/jenkins/workspace/Spark-Master-Maven-with-YARN/HADOOP_PROFILE/ha= doop-2.4/label/centos/sql/hive/src/main/scala/org/apache/spark/sql/hive/Hiv= eMetastoreCatalog.scala
> [error]=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0during phase: typer
> [error]=C2=A0 =C2=A0 =C2=A0 library version: version 2.10.4
> [error]=C2=A0 =C2=A0 =C2=A0compiler version: version 2.10.4
>
>
> I traced back to build #2869 and the error was there - didn't go b= ack
> further.
>
>
> FYI
>
>
> On Thu, Jul 9, 2015 at 7:24 AM, Yijie Shen <henry.yijieshen@gmail.com> > wrote:
>>
>> Hi,
>>
>> I use the clean version just clone from the master branch, build w= ith:
>>
>> build/mvn -Phive -Phadoop-2.4 -DskipTests package
>>
>> And BUILD FAILURE at last, due to:
>>
>> [error]=C2=A0 =C2=A0 =C2=A0 while compiling:
>> /Users/yijie/spark/sql/hive/src/main/scala/org/apache/spark/sql/hi= ve/HiveMetastoreCatalog.scala
>> [error]=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0during phase: typer
>> [error]=C2=A0 =C2=A0 =C2=A0 library version: version 2.10.4
>> [error]=C2=A0 =C2=A0 =C2=A0compiler version: version 2.10.4
>> ...
>> [error]
>> [error]=C2=A0 =C2=A0last tree to typer: Ident(Warehouse)
>> [error]=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0symb= ol: <none> (flags: )
>> [error]=C2=A0 =C2=A0 symbol definition: <none>
>> [error]=C2=A0 =C2=A0 =C2=A0 =C2=A0 symbol owners:
>> [error]=C2=A0 =C2=A0 =C2=A0 =C2=A0context owners: lazy value hiveW= arehouse -> class
>> HiveMetastoreCatalog -> package hive
>> [error]
>> [error] =3D=3D Enclosing template or block =3D=3D
>> [error]
>> [error] Template( // val <local HiveMetastoreCatalog>: <n= otype> in class
>> HiveMetastoreCatalog
>> [error]=C2=A0 =C2=A0"Catalog", "Logging" // pa= rents
>> [error]=C2=A0 =C2=A0ValDef(
>> [error]=C2=A0 =C2=A0 =C2=A0private
>> [error]=C2=A0 =C2=A0 =C2=A0"_"
>> [error]=C2=A0 =C2=A0 =C2=A0<tpt>
>> [error]=C2=A0 =C2=A0 =C2=A0<empty>
>> [error]=C2=A0 =C2=A0)
>> [error]=C2=A0 =C2=A0// 24 statements
>> [error]=C2=A0 =C2=A0ValDef( // private[this] val client:
>> org.apache.spark.sql.hive.client.ClientInterface in class
>> HiveMetastoreCatalog
>> [error]=C2=A0 =C2=A0 =C2=A0private <local> <paramaccessor= >
>> [error]=C2=A0 =C2=A0 =C2=A0"client"
>> [error]=C2=A0 =C2=A0 =C2=A0"ClientInterface"
>> [error]=C2=A0 =C2=A0 =C2=A0<empty>
>> =E2=80=A6
>>
>> https://gist.githu= b.com/yijieshen/e0925e2227a312ae4c64#file-build_failure
>>
>> Did I make a silly mistake?
>>
>> Thanks, Yijie
>
>


--001a1138ea3cdcb701051a7330e9--