Return-Path: <user-return-56415-archive-asf-public=cust-asf.ponee.io@spark.apache.org>
X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io
Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io
Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183])
	by cust-asf2.ponee.io (Postfix) with ESMTP id 19C0E2007D1
	for <archive-asf-public-internal@cust-asf2.ponee.io>; Thu, 12 May 2016 23:20:47 +0200 (CEST)
Received: by cust-asf.ponee.io (Postfix)
	id 18624160939; Thu, 12 May 2016 21:20:47 +0000 (UTC)
Delivered-To: archive-asf-public@cust-asf.ponee.io
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by cust-asf.ponee.io (Postfix) with SMTP id 952381602BF
	for <archive-asf-public@cust-asf.ponee.io>; Thu, 12 May 2016 23:20:45 +0200 (CEST)
Received: (qmail 34823 invoked by uid 500); 12 May 2016 21:20:43 -0000
Mailing-List: contact user-help@spark.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@spark.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@spark.apache.org>
List-Post: <mailto:user@spark.apache.org>
List-Id: <user.spark.apache.org>
Delivered-To: mailing list user@spark.apache.org
Received: (qmail 34813 invoked by uid 99); 12 May 2016 21:20:43 -0000
Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 May 2016 21:20:43 +0000
Received: from localhost (localhost [127.0.0.1])
	by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 17E991A11A7
	for <user@spark.apache.org>; Thu, 12 May 2016 21:20:43 +0000 (UTC)
X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org
X-Spam-Flag: NO
X-Spam-Score: 1.198
X-Spam-Level: *
X-Spam-Status: No, score=1.198 tagged_above=-999 required=6.31
	tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,
	HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001,
	SPF_PASS=-0.001] autolearn=disabled
Authentication-Results: spamd2-us-west.apache.org (amavisd-new);
	dkim=pass (2048-bit key) header.d=gmail.com
Received: from mx2-lw-eu.apache.org ([10.40.0.8])
	by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024)
	with ESMTP id AlTdUHdqkH7n for <user@spark.apache.org>;
	Thu, 12 May 2016 21:20:38 +0000 (UTC)
Received: from mail-qg0-f49.google.com (mail-qg0-f49.google.com [209.85.192.49])
	by mx2-lw-eu.apache.org (ASF Mail Server at mx2-lw-eu.apache.org) with ESMTPS id 8DE825F2F2
	for <user@spark.apache.org>; Thu, 12 May 2016 21:20:37 +0000 (UTC)
Received: by mail-qg0-f49.google.com with SMTP id 90so48356991qgz.1
        for <user@spark.apache.org>; Thu, 12 May 2016 14:20:37 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :cc;
        bh=o6lE5XYCzSyO6IU1Pf7PuJWKSNksNes9XoEGU+dCEAw=;
        b=LkHFhPyzQPiVkpSXTul+iHG6tRvXlywGgtcktka3W7EjJepKxpcFUivAAbJcNrxMV2
         UhBYXrf5qIj1nYXOgPggXSLnUwfPDf2IM0rr3QW3psHx7jnetJ9kMpkG7m9GCikhMvjI
         rRdkVvuaw1kUUbcFJkbCI2ni4bElitM2PA0oY73GddbLqR1GisbaXHLLsrWI/GTbtZD1
         6mE/hSotx0vVhvc6OPOrDbSKJdcZCfv7oOuAeZ6FqpJJc44BSTWf0Ovq7uZF3RBwiZeA
         biC7pN00oubTY8YrUFCwQcOpiOh4gcEluao+ab6XxjfzSEb+nt0vqqSqwmskmA0hGtMl
         /07w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20130820;
        h=x-gm-message-state:mime-version:in-reply-to:references:date
         :message-id:subject:from:to:cc;
        bh=o6lE5XYCzSyO6IU1Pf7PuJWKSNksNes9XoEGU+dCEAw=;
        b=lwFXW+jF18i5JRrS4qk/0I2bP27rsZwwyAYAyY4NBx4+dRQAizttsbgaGqPVXm8Qu7
         STBy2CBLnVWAGnVk3us8qnXJCzp3lBZUPfeh19sJNm/YSssszTcHXRFgpWeGJ1AlZ7oM
         FZlegeWFGXowLhpo7syAmg4vpRccMsIjBSktUCowxE/wUrKNq2VBmeAjb/fRK6hXSXsK
         jF5mLa1fa5XyOGeJ3u/gktdN5WQDK3AuQSYTOHZQWrx5l2u1tNuUUG+3r21/M8R3jUga
         tjHlCmBn+Q1jxiCXMXKg6RlTeuJrSZaWWtKyvPmYUjgaRwZrW+/LYBHSX94CQHczowKk
         noUw==
X-Gm-Message-State: AOPr4FU8kX/8QjHzcwv2+CyQdiE8kDmDZa6T3ktAUcKeJbrxio0JR1iQyKtRmYkmNzKXgF3FYMrATQe8iCG4hw==
MIME-Version: 1.0
X-Received: by 10.140.246.68 with SMTP id r65mr12598629qhc.80.1463088030004;
 Thu, 12 May 2016 14:20:30 -0700 (PDT)
Received: by 10.55.70.69 with HTTP; Thu, 12 May 2016 14:20:29 -0700 (PDT)
In-Reply-To: <CAFSx364wYsmPS9gwLWCAZN6MEAsLQcHQNtqmt=Tpco=sD4myPg@mail.gmail.com>
References: <CAFSx364wYsmPS9gwLWCAZN6MEAsLQcHQNtqmt=Tpco=sD4myPg@mail.gmail.com>
Date: Thu, 12 May 2016 22:20:29 +0100
Message-ID: <CAJ3fcbB08w75W6zr+6hQXKDg0cvbSJ1zKVrYockJyk6SD8fUVw@mail.gmail.com>
Subject: Re: SQLContext and HiveContext parse a query string differently ?
From: Mich Talebzadeh <mich.talebzadeh@gmail.com>
To: Hao Ren <invkrh@gmail.com>
Cc: user <user@spark.apache.org>
Content-Type: multipart/alternative; boundary=001a1139bb244de0e50532abba13
archived-at: Thu, 12 May 2016 21:20:47 -0000

--001a1139bb244de0e50532abba13
Content-Type: text/plain; charset=UTF-8

yep the same error I got

root
 |-- a: array (nullable = true)
 |    |-- element: integer (containsNull = false)
 |-- b: integer (nullable = false)
NoViableAltException(35@[])
        at
org.apache.hadoop.hive.ql.parse.HiveParser.primitiveType(HiveParser.java:38886)
        at
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.castExpression(HiveParser_IdentifiersParser.java:4336)
        at
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.atomExpression(HiveParser_IdentifiersParser.java:6235)
        at
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceFieldExpression(HiveParser_IdentifiersParser.java:6383)
        at
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceUnaryPrefixExpression(HiveParser_IdentifiersParser.java:6768)
        at
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceUnarySuffixExpression(HiveParser_IdentifiersParser.java:6828)
        at
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceBitwiseXorExpression(HiveParser_IdentifiersParser.java:7012)
        at
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceStarExpression(HiveParser_IdentifiersParser.java:7172)
        at
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedencePlusExpression(HiveParser_IdentifiersParser.java:7332)
        at
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceAmpersandExpression(HiveParser_IdentifiersParser.java:7483)
        at
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceBitwiseOrExpression(HiveParser_IdentifiersParser.java:7634)
        at
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceEqualExpression(HiveParser_IdentifiersParser.java:8164)
        at
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceNotExpression(HiveParser_IdentifiersParser.java:9177)
        at
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceAndExpression(HiveParser_IdentifiersParser.java:9296)
        at
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceOrExpression(HiveParser_IdentifiersParser.java:9455)
        at
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.expression(HiveParser_IdentifiersParser.java:6105)
        at
org.apache.hadoop.hive.ql.parse.HiveParser.expression(HiveParser.java:45846)
        at
org.apache.hadoop.hive.ql.parse.HiveParser_SelectClauseParser.selectItem(HiveParser_SelectClauseParser.java:2907)
        at
org.apache.hadoop.hive.ql.parse.HiveParser_SelectClauseParser.selectList(HiveParser_SelectClauseParser.java:1373)
        at
org.apache.hadoop.hive.ql.parse.HiveParser_SelectClauseParser.selectClause(HiveParser_SelectClauseParser.java:1128)
        at
org.apache.hadoop.hive.ql.parse.HiveParser.selectClause(HiveParser.java:45817)
        at
org.apache.hadoop.hive.ql.parse.HiveParser.selectStatement(HiveParser.java:41495)
        at
org.apache.hadoop.hive.ql.parse.HiveParser.regularBody(HiveParser.java:41402)
        at
org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpressionBody(HiveParser.java:40413)
        at
org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:40283)
        at
org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:1590)
        at
org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1109)
        at
org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:202)
        at
org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:166)
        at org.apache.spark.sql.hive.HiveQl$.getAst(HiveQl.scala:276)
        at org.apache.spark.sql.hive.HiveQl$.createPlan(HiveQl.scala:303)
        at
org.apache.spark.sql.hive.ExtendedHiveQlParser$$anonfun$hiveQl$1.apply(ExtendedHiveQlParser.scala:41)
        at
org.apache.spark.sql.hive.ExtendedHiveQlParser$$anonfun$hiveQl$1.apply(ExtendedHiveQlParser.scala:40)
        at
scala.util.parsing.combinator.Parsers$Success.map(Parsers.scala:136)
        at
scala.util.parsing.combinator.Parsers$Success.map(Parsers.scala:135)
        at
scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:242)
        at
scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:242)
        at
scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:222)
        at
scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1$$anonfun$apply$2.apply(Parsers.scala:254)
        at
scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1$$anonfun$apply$2.apply(Parsers.scala:254)
        at
scala.util.parsing.combinator.Parsers$Failure.append(Parsers.scala:202)
        at
scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1.apply(Parsers.scala:254)
        at
scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1.apply(Parsers.scala:254)
        at
scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:222)
        at
scala.util.parsing.combinator.Parsers$$anon$2$$anonfun$apply$14.apply(Parsers.scala:891)
        at
scala.util.parsing.combinator.Parsers$$anon$2$$anonfun$apply$14.apply(Parsers.scala:891)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
        at
scala.util.parsing.combinator.Parsers$$anon$2.apply(Parsers.scala:890)
        at
scala.util.parsing.combinator.PackratParsers$$anon$1.apply(PackratParsers.scala:110)
        at
org.apache.spark.sql.catalyst.AbstractSparkSQLParser.parse(AbstractSparkSQLParser.scala:34)
        at org.apache.spark.sql.hive.HiveQl$.parseSql(HiveQl.scala:295)
        at
org.apache.spark.sql.hive.HiveQLDialect$$anonfun$parse$1.apply(HiveContext.scala:66)
        at
org.apache.spark.sql.hive.HiveQLDialect$$anonfun$parse$1.apply(HiveContext.scala:66)
        at
org.apache.spark.sql.hive.client.ClientWrapper$$anonfun$withHiveState$1.apply(ClientWrapper.scala:290)
        at
org.apache.spark.sql.hive.client.ClientWrapper.liftedTree1$1(ClientWrapper.scala:237)
        at
org.apache.spark.sql.hive.client.ClientWrapper.retryLocked(ClientWrapper.scala:236)
        at
org.apache.spark.sql.hive.client.ClientWrapper.withHiveState(ClientWrapper.scala:279)
        at
org.apache.spark.sql.hive.HiveQLDialect.parse(HiveContext.scala:65)
        at
org.apache.spark.sql.SQLContext$$anonfun$2.apply(SQLContext.scala:211)
        at
org.apache.spark.sql.SQLContext$$anonfun$2.apply(SQLContext.scala:211)
        at
org.apache.spark.sql.execution.SparkSQLParser$$anonfun$org$apache$spark$sql$execution$SparkSQLParser$$others$1.apply(SparkSQLParser.scala:114)
        at
org.apache.spark.sql.execution.SparkSQLParser$$anonfun$org$apache$spark$sql$execution$SparkSQLParser$$others$1.apply(SparkSQLParser.scala:113)
        at
scala.util.parsing.combinator.Parsers$Success.map(Parsers.scala:136)
        at
scala.util.parsing.combinator.Parsers$Success.map(Parsers.scala:135)
        at
scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:242)
        at
scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:242)
        at
scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:222)
        at
scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1$$anonfun$apply$2.apply(Parsers.scala:254)
        at
scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1$$anonfun$apply$2.apply(Parsers.scala:254)
        at
scala.util.parsing.combinator.Parsers$Failure.append(Parsers.scala:202)
        at
scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1.apply(Parsers.scala:254)
        at
scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1.apply(Parsers.scala:254)
        at
scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:222)
        at
scala.util.parsing.combinator.Parsers$$anon$2$$anonfun$apply$14.apply(Parsers.scala:891)
        at
scala.util.parsing.combinator.Parsers$$anon$2$$anonfun$apply$14.apply(Parsers.scala:891)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
        at
scala.util.parsing.combinator.Parsers$$anon$2.apply(Parsers.scala:890)
        at
scala.util.parsing.combinator.PackratParsers$$anon$1.apply(PackratParsers.scala:110)
        at
org.apache.spark.sql.catalyst.AbstractSparkSQLParser.parse(AbstractSparkSQLParser.scala:34)
        at
org.apache.spark.sql.SQLContext$$anonfun$1.apply(SQLContext.scala:208)
        at
org.apache.spark.sql.SQLContext$$anonfun$1.apply(SQLContext.scala:208)
        at
org.apache.spark.sql.execution.datasources.DDLParser.parse(DDLParser.scala:43)
        at org.apache.spark.sql.SQLContext.parseSql(SQLContext.scala:231)
        at
org.apache.spark.sql.hive.HiveContext.parseSql(HiveContext.scala:331)
        at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:817)
        at Test$delayedInit$body.apply(Test.scala:21)
        at scala.Function0$class.apply$mcV$sp(Function0.scala:40)
        at
scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
        at scala.App$$anonfun$main$1.apply(App.scala:71)
        at scala.App$$anonfun$main$1.apply(App.scala:71)
        at scala.collection.immutable.List.foreach(List.scala:318)
        at
scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:32)
        at scala.App$class.main(App.scala:71)
        at Test$.main(Test.scala:5)
        at Test.main(Test.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
        at
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
        at
org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Exception in thread "main" org.apache.spark.sql.AnalysisException: cannot
recognize input near 'array' '<' 'string' in primitive type specification;
line 1 pos 17


Let me investigate it further


Dr Mich Talebzadeh



LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*



http://talebzadehmich.wordpress.com



On 12 May 2016 at 12:09, Hao Ren <invkrh@gmail.com> wrote:

> HI,
>
> I just want to figure out why the two contexts behavior differently even
> on a simple query.
> In a netshell, I have a query in which there is a String containing single
> quote and casting to Array/Map.
> I have tried all the combination of diff type of sql context and query
> call api (sql, df.select, df.selectExpr).
> I can't find one rules all.
>
> Here is the code for reproducing the problem.
>
> -----------------------------------------------------------------------------
>
> import org.apache.spark.sql.SQLContext
> import org.apache.spark.sql.hive.HiveContext
> import org.apache.spark.{SparkConf, SparkContext}
>
> object Test extends App {
>
>   val sc          = new SparkContext("local[2]", "test", new SparkConf)
>   val hiveContext = new HiveContext(sc)
>   val sqlContext  = new SQLContext(sc)
>
>   val context = hiveContext
>   //  val context = sqlContext
>
>   import context.implicits._
>
>   val df = Seq((Seq(1, 2), 2)).toDF("a", "b")
>   df.registerTempTable("tbl")
>   df.printSchema()
>
>   // case 1
>   context.sql("select cast(a as array<string>) from tbl").show()
>   // HiveContext => org.apache.spark.sql.AnalysisException: cannot recognize input near 'array' '<' 'string' in primitive type specification; line 1 pos 17
>   // SQLContext => OK
>
>   // case 2
>   context.sql("select 'a\\'b'").show()
>   // HiveContext => OK
>   // SQLContext => failure: ``union'' expected but ErrorToken(unclosed string literal) found
>
>   // case 3
>   df.selectExpr("cast(a as array<string>)").show() // OK with HiveContext and SQLContext
>
>   // case 4
>   df.selectExpr("'a\\'b'").show() // HiveContext, SQLContext => failure: end of input expected
> }
>
> -----------------------------------------------------------------------------
>
> Any clarification / workaround is high appreciated.
>
> --
> Hao Ren
>
> Data Engineer @ leboncoin
>
> Paris, France
>

--001a1139bb244de0e50532abba13
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>yep the same error I got</div><div><br></div><div><fo=
nt color=3D"#0000ff" face=3D"monospace,monospace">root<br>=C2=A0|-- a: arra=
y (nullable =3D true)<br>=C2=A0|=C2=A0=C2=A0=C2=A0 |-- element: integer (co=
ntainsNull =3D false)<br>=C2=A0|-- b: integer (nullable =3D false)</font></=
div><div><font color=3D"#0000ff" face=3D"monospace,monospace">NoViableAltEx=
ception(35@[])<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.=
hadoop.hive.ql.parse.HiveParser.primitiveType(HiveParser.java:38886)<br>=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.hive.ql.parse.=
HiveParser_IdentifiersParser.castExpression(HiveParser_IdentifiersParser.ja=
va:4336)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop=
.hive.ql.parse.HiveParser_IdentifiersParser.atomExpression(HiveParser_Ident=
ifiersParser.java:6235)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at or=
g.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceFieldE=
xpression(HiveParser_IdentifiersParser.java:6383)<br>=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.hive.ql.parse.HiveParser_Identif=
iersParser.precedenceUnaryPrefixExpression(HiveParser_IdentifiersParser.jav=
a:6768)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.=
hive.ql.parse.HiveParser_IdentifiersParser.precedenceUnarySuffixExpression(=
HiveParser_IdentifiersParser.java:6828)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 at org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParse=
r.precedenceBitwiseXorExpression(HiveParser_IdentifiersParser.java:7012)<br=
>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.hive.ql.pa=
rse.HiveParser_IdentifiersParser.precedenceStarExpression(HiveParser_Identi=
fiersParser.java:7172)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org=
.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedencePlusExp=
ression(HiveParser_IdentifiersParser.java:7332)<br>=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.hive.ql.parse.HiveParser_Identifier=
sParser.precedenceAmpersandExpression(HiveParser_IdentifiersParser.java:748=
3)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.hive.=
ql.parse.HiveParser_IdentifiersParser.precedenceBitwiseOrExpression(HivePar=
ser_IdentifiersParser.java:7634)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 at org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precede=
nceEqualExpression(HiveParser_IdentifiersParser.java:8164)<br>=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.hive.ql.parse.HiveParse=
r_IdentifiersParser.precedenceNotExpression(HiveParser_IdentifiersParser.ja=
va:9177)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop=
.hive.ql.parse.HiveParser_IdentifiersParser.precedenceAndExpression(HivePar=
ser_IdentifiersParser.java:9296)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 at org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precede=
nceOrExpression(HiveParser_IdentifiersParser.java:9455)<br>=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.hive.ql.parse.HiveParser_I=
dentifiersParser.expression(HiveParser_IdentifiersParser.java:6105)<br>=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.hive.ql.parse.=
HiveParser.expression(HiveParser.java:45846)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 at org.apache.hadoop.hive.ql.parse.HiveParser_SelectClauseP=
arser.selectItem(HiveParser_SelectClauseParser.java:2907)<br>=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.hive.ql.parse.HiveParse=
r_SelectClauseParser.selectList(HiveParser_SelectClauseParser.java:1373)<br=
>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.hive.ql.pa=
rse.HiveParser_SelectClauseParser.selectClause(HiveParser_SelectClauseParse=
r.java:1128)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ha=
doop.hive.ql.parse.HiveParser.selectClause(HiveParser.java:45817)<br>=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.hive.ql.parse.Hiv=
eParser.selectStatement(HiveParser.java:41495)<br>=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.hive.ql.parse.HiveParser.regularBod=
y(HiveParser.java:41402)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at o=
rg.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpressionBody(Hive=
Parser.java:40413)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apa=
che.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.jav=
a:40283)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop=
.hive.ql.parse.HiveParser.execStatement(HiveParser.java:1590)<br>=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.hive.ql.parse.HivePa=
rser.statement(HiveParser.java:1109)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.=
java:202)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoo=
p.hive.ql.parse.ParseDriver.parse(ParseDriver.java:166)<br>=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.spark.sql.hive.HiveQl$.getAst(Hiv=
eQl.scala:276)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.=
spark.sql.hive.HiveQl$.createPlan(HiveQl.scala:303)<br>=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.spark.sql.hive.ExtendedHiveQlParser$=
$anonfun$hiveQl$1.apply(ExtendedHiveQlParser.scala:41)<br>=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.spark.sql.hive.ExtendedHiveQlPars=
er$$anonfun$hiveQl$1.apply(ExtendedHiveQlParser.scala:40)<br>=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.util.parsing.combinator.Parsers$Suc=
cess.map(Parsers.scala:136)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 a=
t scala.util.parsing.combinator.Parsers$Success.map(Parsers.scala:135)<br>=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.util.parsing.combinator=
.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:242)<br>=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.util.parsing.combinator.Parsers$Parser=
$$anonfun$map$1.apply(Parsers.scala:242)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers=
.scala:222)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.util.par=
sing.combinator.Parsers$Parser$$anonfun$append$1$$anonfun$apply$2.apply(Par=
sers.scala:254)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.util=
.parsing.combinator.Parsers$Parser$$anonfun$append$1$$anonfun$apply$2.apply=
(Parsers.scala:254)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.=
util.parsing.combinator.Parsers$Failure.append(Parsers.scala:202)<br>=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.util.parsing.combinator.Parse=
rs$Parser$$anonfun$append$1.apply(Parsers.scala:254)<br>=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0 at scala.util.parsing.combinator.Parsers$Parser$$a=
nonfun$append$1.apply(Parsers.scala:254)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers=
.scala:222)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.util.par=
sing.combinator.Parsers$$anon$2$$anonfun$apply$14.apply(Parsers.scala:891)<=
br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.util.parsing.combina=
tor.Parsers$$anon$2$$anonfun$apply$14.apply(Parsers.scala:891)<br>=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.util.DynamicVariable.withValue(D=
ynamicVariable.scala:57)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at s=
cala.util.parsing.combinator.Parsers$$anon$2.apply(Parsers.scala:890)<br>=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.util.parsing.combinator=
.PackratParsers$$anon$1.apply(PackratParsers.scala:110)<br>=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.spark.sql.catalyst.AbstractSparkS=
QLParser.parse(AbstractSparkSQLParser.scala:34)<br>=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 at org.apache.spark.sql.hive.HiveQl$.parseSql(HiveQl.sca=
la:295)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.spark.s=
ql.hive.HiveQLDialect$$anonfun$parse$1.apply(HiveContext.scala:66)<br>=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.spark.sql.hive.HiveQL=
Dialect$$anonfun$parse$1.apply(HiveContext.scala:66)<br>=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.spark.sql.hive.client.ClientWrapper$=
$anonfun$withHiveState$1.apply(ClientWrapper.scala:290)<br>=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.spark.sql.hive.client.ClientWrapp=
er.liftedTree1$1(ClientWrapper.scala:237)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 at org.apache.spark.sql.hive.client.ClientWrapper.retryLocked(=
ClientWrapper.scala:236)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at o=
rg.apache.spark.sql.hive.client.ClientWrapper.withHiveState(ClientWrapper.s=
cala:279)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.spark=
.sql.hive.HiveQLDialect.parse(HiveContext.scala:65)<br>=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.spark.sql.SQLContext$$anonfun$2.appl=
y(SQLContext.scala:211)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at or=
g.apache.spark.sql.SQLContext$$anonfun$2.apply(SQLContext.scala:211)<br>=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.spark.sql.execution.S=
parkSQLParser$$anonfun$org$apache$spark$sql$execution$SparkSQLParser$$other=
s$1.apply(SparkSQLParser.scala:114)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 at org.apache.spark.sql.execution.SparkSQLParser$$anonfun$org$apache=
$spark$sql$execution$SparkSQLParser$$others$1.apply(SparkSQLParser.scala:11=
3)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.util.parsing.comb=
inator.Parsers$Success.map(Parsers.scala:136)<br>=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 at scala.util.parsing.combinator.Parsers$Success.map(Par=
sers.scala:135)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.util=
.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:242)<=
br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.util.parsing.combina=
tor.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:242)<br>=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.util.parsing.combinator.Parsers$$an=
on$3.apply(Parsers.scala:222)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
 at scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1$$anonfun=
$apply$2.apply(Parsers.scala:254)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 at scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1$$a=
nonfun$apply$2.apply(Parsers.scala:254)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 at scala.util.parsing.combinator.Parsers$Failure.append(Parser=
s.scala:202)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.util.pa=
rsing.combinator.Parsers$Parser$$anonfun$append$1.apply(Parsers.scala:254)<=
br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.util.parsing.combina=
tor.Parsers$Parser$$anonfun$append$1.apply(Parsers.scala:254)<br>=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.util.parsing.combinator.Parsers$=
$anon$3.apply(Parsers.scala:222)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 at scala.util.parsing.combinator.Parsers$$anon$2$$anonfun$apply$14.appl=
y(Parsers.scala:891)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala=
.util.parsing.combinator.Parsers$$anon$2$$anonfun$apply$14.apply(Parsers.sc=
ala:891)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.util.Dynami=
cVariable.withValue(DynamicVariable.scala:57)<br>=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 at scala.util.parsing.combinator.Parsers$$anon$2.apply(P=
arsers.scala:890)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.ut=
il.parsing.combinator.PackratParsers$$anon$1.apply(PackratParsers.scala:110=
)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.spark.sql.cat=
alyst.AbstractSparkSQLParser.parse(AbstractSparkSQLParser.scala:34)<br>=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.spark.sql.SQLContext$=
$anonfun$1.apply(SQLContext.scala:208)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 at org.apache.spark.sql.SQLContext$$anonfun$1.apply(SQLContext.sc=
ala:208)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.spark.=
sql.execution.datasources.DDLParser.parse(DDLParser.scala:43)<br>=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.spark.sql.SQLContext.parseS=
ql(SQLContext.scala:231)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at o=
rg.apache.spark.sql.hive.HiveContext.parseSql(HiveContext.scala:331)<br>=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.spark.sql.SQLContext.=
sql(SQLContext.scala:817)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at =
Test$delayedInit$body.apply(Test.scala:21)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 at scala.Function0$class.apply$mcV$sp(Function0.scala:40)<b=
r>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.runtime.AbstractFunct=
ion0.apply$mcV$sp(AbstractFunction0.scala:12)<br>=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 at scala.App$$anonfun$main$1.apply(App.scala:71)<br>=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.App$$anonfun$main$1.apply(=
App.scala:71)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.collec=
tion.immutable.List.foreach(List.scala:318)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 at scala.collection.generic.TraversableForwarder$class.fore=
ach(TraversableForwarder.scala:32)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 at scala.App$class.main(App.scala:71)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 at Test$.main(Test.scala:5)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 at Test.main(Test.scala)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)=
<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at sun.reflect.NativeMethodA=
ccessorImpl.invoke(NativeMethodAccessorImpl.java:62)<br>=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0 at sun.reflect.DelegatingMethodAccessorImpl.invoke=
(DelegatingMethodAccessorImpl.java:43)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 at java.lang.reflect.Method.invoke(Method.java:498)<br>=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.spark.deploy.SparkSubmit$.o=
rg$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)<br>=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.spark.deploy.SparkSub=
mit$.doRunMain$1(SparkSubmit.scala:181)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.sca=
la:206)<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.spark.d=
eploy.SparkSubmit$.main(SparkSubmit.scala:121)<br>=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.=
scala)<br>Exception in thread &quot;main&quot; org.apache.spark.sql.Analysi=
sException: cannot recognize input near &#39;array&#39; &#39;&lt;&#39; &#39=
;string&#39; in primitive type specification; line 1 pos 17</font></div><p>=
<font color=3D"#0000ff" face=3D"Courier New"><br></font></p><div><font colo=
r=3D"#000000" face=3D"arial,helvetica,sans-serif">Let me investigate it fur=
ther</font></div><div><br></div></div><div class=3D"gmail_extra"><br clear=
=3D"all"><div><div class=3D"gmail_signature"><div dir=3D"ltr"><font color=
=3D"#000000" face=3D"Times New Roman" size=3D"3">

</font><p style=3D"margin:0cm 0cm 0pt"><font color=3D"#000000" face=3D"Cali=
bri" size=3D"3">Dr Mich Talebzadeh</font></p><font color=3D"#000000" face=
=3D"Times New Roman" size=3D"3">

</font><p style=3D"margin:0cm 0cm 0pt"><font color=3D"#000000" face=3D"Cali=
bri" size=3D"3">=C2=A0</font></p><font color=3D"#000000" face=3D"Times New =
Roman" size=3D"3">

</font><p style=3D"margin:0cm 0cm 0pt"><span style=3D"font-family:&quot;Ari=
al&quot;,sans-serif"><font color=3D"#000000" size=3D"3">LinkedIn </font></s=
pan><i><span style=3D"font-family:&quot;Arial&quot;,sans-serif;font-size:10=
pt"><font color=3D"#000000">=C2=A0</font><a href=3D"https://www.linkedin.co=
m/profile/view?id=3DAAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw" target=3D"_bla=
nk"><font color=3D"#0000ff">https://www.linkedin.com/profile/view?id=3DAAEA=
AAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw</font></a></span></i></p><font color=3D=
"#000000" face=3D"Times New Roman" size=3D"3">

</font><p style=3D"margin:0cm 0cm 0pt"><font color=3D"#000000" face=3D"Cali=
bri" size=3D"3">=C2=A0</font></p><font color=3D"#000000" face=3D"Times New =
Roman" size=3D"3">

</font><p style=3D"margin:0cm 0cm 0pt;text-align:justify"><span style=3D"fo=
nt-family:&quot;Arial&quot;,sans-serif;font-size:10pt"><a href=3D"http://ta=
lebzadehmich.wordpress.com/" target=3D"_blank"><font color=3D"#0000ff">http=
://talebzadehmich.wordpress.com</font></a></span></p><font color=3D"#000000=
" face=3D"Times New Roman" size=3D"3">

</font><p style=3D"margin:0cm 0cm 0pt"><span style=3D"font-family:&quot;Ari=
al&quot;,sans-serif;font-size:9pt"><font color=3D"#000000">=C2=A0</font></s=
pan></p><font color=3D"#000000" face=3D"Times New Roman" size=3D"3">

</font></div></div></div>
<br><div class=3D"gmail_quote">On 12 May 2016 at 12:09, Hao Ren <span dir=
=3D"ltr">&lt;<a href=3D"mailto:invkrh@gmail.com" target=3D"_blank">invkrh@g=
mail.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=
=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=
=3D"ltr"><div>HI,</div><div><br></div><div>I just want to figure out why th=
e two contexts behavior differently even on a simple query.</div><div><div>=
In a netshell, I have a query in which there is a String containing single =
quote and casting to Array/Map.</div><div>I have tried all the combination =
of diff type of sql context and query call api (sql, df.select, df.selectEx=
pr).</div></div><div>I can&#39;t find one rules all.</div><div><br></div><d=
iv>Here is the code for reproducing the problem.</div><div>----------------=
-------------------------------------------------------------<br></div><pre=
 style=3D"color:rgb(0,0,0);white-space:pre-wrap">import org.apache.spark.sq=
l.SQLContext
import org.apache.spark.sql.hive.HiveContext
import org.apache.spark.{SparkConf, SparkContext}

object Test extends App {

  val sc          =3D new SparkContext(&quot;local[2]&quot;, &quot;test&quo=
t;, new SparkConf)
  val hiveContext =3D new HiveContext(sc)
  val sqlContext  =3D new SQLContext(sc)

  val context =3D hiveContext
  //  val context =3D sqlContext

  import context.implicits._

  val df =3D Seq((Seq(1, 2), 2)).toDF(&quot;a&quot;, &quot;b&quot;)
  df.registerTempTable(&quot;tbl&quot;)
  df.printSchema()

  // case 1
  context.sql(&quot;select cast(a as array&lt;string&gt;) from tbl&quot;).s=
how()
  // HiveContext =3D&gt; org.apache.spark.sql.AnalysisException: cannot rec=
ognize input near &#39;array&#39; &#39;&lt;&#39; &#39;string&#39; in primit=
ive type specification; line 1 pos 17
  // SQLContext =3D&gt; OK

  // case 2
  context.sql(&quot;select &#39;a\\&#39;b&#39;&quot;).show()
  // HiveContext =3D&gt; OK
  // SQLContext =3D&gt; failure: ``union&#39;&#39; expected but ErrorToken(=
unclosed string literal) found

  // case 3
  df.selectExpr(&quot;cast(a as array&lt;string&gt;)&quot;).show() // OK wi=
th HiveContext and SQLContext

  // case 4
  df.selectExpr(&quot;&#39;a\\&#39;b&#39;&quot;).show() // HiveContext, SQL=
Context =3D&gt; failure: end of input expected
}</pre><pre style=3D"color:rgb(0,0,0);white-space:pre-wrap"><span style=3D"=
color:rgb(34,34,34);font-family:arial,sans-serif;white-space:normal">------=
-----------------------------------------------------------------------</sp=
an><br></pre><div>Any clarification / workaround is high appreciated.</div>=
<span class=3D"HOEnZb"><font color=3D"#888888"><div><br></div>-- <br><div><=
div dir=3D"ltr"><div><div dir=3D"ltr"><div>Hao Ren</div><div><br></div>Data=
 Engineer @ leboncoin<div><br></div><div>Paris, France</div></div></div></d=
iv></div>
</font></span></div>
</blockquote></div><br></div>

--001a1139bb244de0e50532abba13--