phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Soldatov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-4222) PhoenixStorageHandler for Hive is not working
Date Thu, 21 Sep 2017 18:17:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-4222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175230#comment-16175230
] 

Sergey Soldatov commented on PHOENIX-4222:
------------------------------------------

[~elserj] Well, all this column mapping is really pain in the neck. The main problem is that
we need not only map it from hive to phoenix, but also do the reverse during the query execution.
Keeping all column maps all the time was looking promising, but I recently run into the case
when it was not possible to map all columns because Hive has limit of 4000 chars for properties.
Need to think how to better handle that.

> PhoenixStorageHandler for Hive is not working
> ---------------------------------------------
>
>                 Key: PHOENIX-4222
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-4222
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.11.0
>            Reporter: JeongMin Ju
>            Assignee: Sergey Soldatov
>              Labels: hive
>         Attachments: PhoenixQueryBuilder-buildExpressionStringFrom.jpeg, PhoenixQueryBuilder-getSelectColumns.jpeg
>
>
> {panel}
> ERROR CliDriver: Failed with exception java.io.IOException:java.lang.RuntimeException:
org.apache.phoenix.schema.ColumnNotFoundException: ERROR 504 (42703): Undefined column. columnName=PHOENIX.MOVIES.movie_id
> java.io.IOException: java.lang.RuntimeException: org.apache.phoenix.schema.ColumnNotFoundException:
ERROR 504 (42703): Undefined column. columnName=PHOENIX.MOVIES.movie_id
>   at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:508)
>   at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:415)
>   at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:138)
>   at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1986)
>   at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:240)
>   at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:172)
>   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:383)
>   at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:775)
>   at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:693)
>   at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:628)
> Caused by: java.lang.RuntimeException: org.apache.phoenix.schema.ColumnNotFoundException:
ERROR 504 (42703): Undefined column. columnName=PHOENIX.MOVIES.movie_id
>   at org.apache.phoenix.hive.mapreduce.PhoenixInputFormat.getQueryPlan(PhoenixInputFormat.java:266)
>   at org.apache.phoenix.hive.mapreduce.PhoenixInputFormat.getSplits(PhoenixInputFormat.java:131)
>   at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextSplits(FetchOperator.java:363)
>   at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:295)
>   at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:446)
>   ... 9 more
> Caused by: org.apache.phoenix.schema.ColumnNotFoundException: ERROR 504 (42703): Undefined
column. columnName=PHOENIX.MOVIES.movie_id
>   at org.apache.phoenix.schema.PTableImpl.getColumnForColumnName(PTableImpl.java:818)
>   at org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.resolveColumn(FromCompiler.java:478)
>   at org.apache.phoenix.compile.TupleProjectionCompiler$ColumnRefVisitor.visit(TupleProjectionCompiler.java:208)
>   at org.apache.phoenix.compile.TupleProjectionCompiler$ColumnRefVisitor.visit(TupleProjectionCompiler.java:1)
>   at org.apache.phoenix.parse.ColumnParseNode.accept(ColumnParseNode.java:56)
>   at org.apache.phoenix.compile.TupleProjectionCompiler.createProjectedTable(TupleProjectionCompiler.java:109)
>   at org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:528)
>   at org.apache.phoenix.compile.QueryCompiler.compileSingleQuery(QueryCompiler.java:507)
>   at org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:202)
>   at org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:157)
>   at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:475)
>   at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:1)
>   at org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1648)
>   at org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1641)
>   at org.apache.phoenix.jdbc.PhoenixStatement.optimizeQuery(PhoenixStatement.java:1635)
>   at org.apache.phoenix.hive.mapreduce.PhoenixInputFormat.getQueryPlan(PhoenixInputFormat.java:260)
>   ... 13 more
> {panel}
> {noformat}
> : jdbc:phoenix:juke-cdh2> !columns phoenix.movies
> +------------+--------------+-------------+--------------+------------+------------+--------------+----------------+-----------------+-----------------+-----------+----------+-------------+
> | TABLE_CAT  | TABLE_SCHEM  | TABLE_NAME  | COLUMN_NAME  | DATA_TYPE  | TYPE_NAME  |
COLUMN_SIZE  | BUFFER_LENGTH  | DECIMAL_DIGITS  | NUM_PREC_RADIX  | NULLABLE  | REMARKS  |
COLUMN_DEF  |
> +------------+--------------+-------------+--------------+------------+------------+--------------+----------------+-----------------+-----------------+-----------+----------+-------------+
> |            | PHOENIX      | MOVIES      | MOVIE_ID     | 4          | INTEGER    |
null         | null           | null            | null            | 0         |          |
            |
> |            | PHOENIX      | MOVIES      | TITLE        | 12         | VARCHAR    |
null         | null           | null            | null            | 1         |          |
            |
> |            | PHOENIX      | MOVIES      | GENRES       | 12         | VARCHAR    |
null         | null           | null            | null            | 1         |          |
            |
> +------------+--------------+-------------+--------------+------------+------------+--------------+----------------+-----------------+-----------------+-----------+----------+-------------+
> {noformat}
> {noformat}
> hive> show create table movies;
> CREATE EXTERNAL TABLE `movies`(
>   `movie_id` int COMMENT 'from deserializer', 
>   `title` string COMMENT 'from deserializer', 
>   `genres` string COMMENT 'from deserializer')
> ROW FORMAT SERDE 
>   'org.apache.phoenix.hive.PhoenixSerDe' 
> STORED BY 
>   'org.apache.phoenix.hive.PhoenixStorageHandler' 
> WITH SERDEPROPERTIES ( 
>   'serialization.format'='1')
> LOCATION
>   'hdfs://nameservice1/user/hive/warehouse/movies'
> TBLPROPERTIES (
>   'COLUMN_STATS_ACCURATE'='false', 
>   'numFiles'='0', 
>   'numRows'='-1', 
>   'phoenix.table.name'='phoenix.movies', 
>   'phoenix.zookeeper.quorum'='xxx', 
>   'rawDataSize'='-1', 
>   'totalSize'='0', 
>   'transient_lastDdlTime'='1503021635')
> {noformat}
> I ran the query "select * from movies" and
> The query created by the phoenix handler is:
> select /*+ NO_CACHE  */ "movie_id","title","genres" from phoenix.movies
> But do not enter double quotes like this.
> I looked up where to put double quotes.
> !https://issues.apache.org/jira/secure/attachment/12888228/PhoenixQueryBuilder-getSelectColumns.jpeg!
> !https://issues.apache.org/jira/secure/attachment/12888229/PhoenixQueryBuilder-buildExpressionStringFrom.jpeg!
> Why did you change it to double quotes?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message