phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fengdong Yu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PHOENIX-2290) Spark Phoenix cannot recognize Phoenix view fields
Date Thu, 24 Sep 2015 09:03:04 GMT

     [ https://issues.apache.org/jira/browse/PHOENIX-2290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Fengdong Yu updated PHOENIX-2290:
---------------------------------
    Description: 
I created base table in base shell:

{code}
create 'test_table',  {NAME => 'cf1', VERSIONS => 1}
put 'test_table', 'row_key_1', 'cf1:col_1', '200'
{code}

This is a very simple table. then create phoenix view in Phoenix shell.
{code}
create view "test_table" (pk varchar primary key, "cf1"."col_1" varchar)
{code}

then do following in Spark shell:

val df = sqlContext.load("org.apache.phoenix.spark", Map("table" -> "\"test_table\"", 
"zkUrl" -> "localhost:2181"))
df.registerTempTable("temp")

sqlContext.sql("select * from temp")  ------> {color:red} This does work{color}

then:
sqlContext.sql("select * from temp where col_1='200' ")

java.lang.RuntimeException: org.apache.phoenix.schema.ColumnNotFoundException: ERROR 504 (42703):
Undefined column. columnName=col_1
	at org.apache.phoenix.mapreduce.PhoenixInputFormat.getQueryPlan(PhoenixInputFormat.java:125)
	at org.apache.phoenix.mapreduce.PhoenixInputFormat.getSplits(PhoenixInputFormat.java:80)
	at org.apache.spark.rdd.NewHadoopRDD.getPartitions(NewHadoopRDD.scala:95)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:219)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:217)
	at scala.Option.getOrElse(Option.scala:120)
	at org.apache.spark.rdd.RDD.partitions(RDD.scala:217)
	at org.apache.phoenix.spark.PhoenixRDD.getPartitions(PhoenixRDD.scala:47)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:219)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:217)
	at scala.Option.getOrElse(Option.scala:120)

{color:red}
I also tried:
{code}
sqlContext.sql("select * from temp where \"col_1\"='200' ")  --> EMPTY result, no exception
{code}
{code}
sqlContext.sql("select * from temp where \"cf1\".\"col_1\"='200' ")  --> exception, cannot
recognize SQL
{code}

{color}


  was:
I created base table in base shell:

{code}
create 'test_table',  {NAME => 'cf1', VERSIONS => 1}
put 'test_table', 'row_key_1', 'cf1:col_1', '200'
{code}

This is a very simple table. then create phoenix view in Phoenix shell.
{code}
create view "test_table" (pk varchar primary key, "cf1"."col_1" varchar)
{code}

then do following in Spark shell:

val df = sqlContext.load("org.apache.phoenix.spark", Map("table" -> "\"test_table\"", 
"zkUrl" -> "localhost:2181"))
df.registerTempTable("temp")

sqlContext.sql("select * from temp")  ------> {color:red} This does work{color}

then:
sqlContext.sql("select * from temp where col_1='200' ")

java.lang.RuntimeException: org.apache.phoenix.schema.ColumnNotFoundException: ERROR 504 (42703):
Undefined column. columnName=col_1
	at org.apache.phoenix.mapreduce.PhoenixInputFormat.getQueryPlan(PhoenixInputFormat.java:125)
	at org.apache.phoenix.mapreduce.PhoenixInputFormat.getSplits(PhoenixInputFormat.java:80)
	at org.apache.spark.rdd.NewHadoopRDD.getPartitions(NewHadoopRDD.scala:95)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:219)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:217)
	at scala.Option.getOrElse(Option.scala:120)
	at org.apache.spark.rdd.RDD.partitions(RDD.scala:217)
	at org.apache.phoenix.spark.PhoenixRDD.getPartitions(PhoenixRDD.scala:47)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:219)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:217)
	at scala.Option.getOrElse(Option.scala:120)

{color:red}
I also tried:
{code}
sqlContext.sql("select * from temp where \"col_1\"='200' ")  --> EMPTY result, no exception
{code}

{color}



> Spark Phoenix cannot recognize Phoenix view fields
> --------------------------------------------------
>
>                 Key: PHOENIX-2290
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-2290
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.5.1
>            Reporter: Fengdong Yu
>
> I created base table in base shell:
> {code}
> create 'test_table',  {NAME => 'cf1', VERSIONS => 1}
> put 'test_table', 'row_key_1', 'cf1:col_1', '200'
> {code}
> This is a very simple table. then create phoenix view in Phoenix shell.
> {code}
> create view "test_table" (pk varchar primary key, "cf1"."col_1" varchar)
> {code}
> then do following in Spark shell:
> val df = sqlContext.load("org.apache.phoenix.spark", Map("table" -> "\"test_table\"",
 "zkUrl" -> "localhost:2181"))
> df.registerTempTable("temp")
> sqlContext.sql("select * from temp")  ------> {color:red} This does work{color}
> then:
> sqlContext.sql("select * from temp where col_1='200' ")
> java.lang.RuntimeException: org.apache.phoenix.schema.ColumnNotFoundException: ERROR
504 (42703): Undefined column. columnName=col_1
> 	at org.apache.phoenix.mapreduce.PhoenixInputFormat.getQueryPlan(PhoenixInputFormat.java:125)
> 	at org.apache.phoenix.mapreduce.PhoenixInputFormat.getSplits(PhoenixInputFormat.java:80)
> 	at org.apache.spark.rdd.NewHadoopRDD.getPartitions(NewHadoopRDD.scala:95)
> 	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:219)
> 	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:217)
> 	at scala.Option.getOrElse(Option.scala:120)
> 	at org.apache.spark.rdd.RDD.partitions(RDD.scala:217)
> 	at org.apache.phoenix.spark.PhoenixRDD.getPartitions(PhoenixRDD.scala:47)
> 	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:219)
> 	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:217)
> 	at scala.Option.getOrElse(Option.scala:120)
> {color:red}
> I also tried:
> {code}
> sqlContext.sql("select * from temp where \"col_1\"='200' ")  --> EMPTY result, no
exception
> {code}
> {code}
> sqlContext.sql("select * from temp where \"cf1\".\"col_1\"='200' ")  --> exception,
cannot recognize SQL
> {code}
> {color}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message