spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xiao Li <gatorsm...@gmail.com>
Subject Re: Quotes within a table name (phoenix table) getting failure: identifier expected at Spark level parsing
Date Mon, 10 Oct 2016 22:10:03 GMT
HI, Nico,

We use back ticks to quote it. For example,

CUSTOM_ENTITY.`z02`

Thanks,

Xiao Li

2016-10-10 12:49 GMT-07:00 Nico Pappagianis <nico.pappagianis@salesforce.com
>:

> Hello,
>
> *Some context:*
> I have a Phoenix tenant-specific view named CUSTOM_ENTITY."z02" (Phoenix
> tables can have quotes to specify case-sensitivity). I am attempting to
> write to this table using Spark via a scala script. I am performing the
> following read successfully:
>
> val table = """CUSTOM_ENTITY."z02""""
> val tenantId = "myTenantId"
> val urlWithTenant = "jdbc:phoenix:myZKHost1, myZKHost1, myZKHost2,
> myZKHost3:2181;TenantId=myTenantId"
> val driver = "org.apache.phoenix.jdbc.PhoenixDriver"
>
> val readOptions = Map(driver" -> driver, "url" -> urlWithTenant, "dbtable"
> -> table
> )
>
> val df = sqlContext.read.format("jdbc").options(readOptions).load
>
> This gives me the dataframe with data successfully read from my tenant
> view.
>
> Now when I try to write back with this dataframe:
>
> df.write.format("jdbc").insertInto(table)
>
>
> I am getting the following exception:
>
> java.lang.RuntimeException: [1.15] failure: identifier expected
>
> CUSTOM_ENTITY."z02"
>
>                               ^
>
> (caret is pointing under the '.' before "z02")
>
> at scala.sys.package$.error(package.scala:27)
>
> at org.apache.spark.sql.catalyst.SqlParser$.parseTableIdentifier(
> SqlParser.scala:56)
>
> at org.apache.spark.sql.DataFrameWriter.insertInto(
> DataFrameWriter.scala:164)
>
> Looking at the stack trace it appears that Spark doesn't know what to do
> with the quotes around z02. I've tried escaping them in every way I could
> think of but to no avail.
>
> Is there a way to have Spark not complain about the quotes and correctly
> pass them along?
>
> Thanks
>

Mime
View raw message