kylin-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KYLIN-3695) Error while creating hive table through Kylin build cube with mysql imported tables
Date Wed, 05 Dec 2018 05:28:00 GMT

    [ https://issues.apache.org/jira/browse/KYLIN-3695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16709619#comment-16709619
] 

ASF GitHub Bot commented on KYLIN-3695:
---------------------------------------

Emiya0306 opened a new pull request #371: KYLIN-3695, fix lose decimal scale value in column
type decimal(a, b)
URL: https://github.com/apache/kylin/pull/371
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Error while creating hive table through Kylin build cube with mysql imported tables
> -----------------------------------------------------------------------------------
>
>                 Key: KYLIN-3695
>                 URL: https://issues.apache.org/jira/browse/KYLIN-3695
>             Project: Kylin
>          Issue Type: Bug
>          Components: Integration
>    Affects Versions: v2.5.0
>            Reporter: Gaurav Rawat
>            Assignee: XiaoXiang Yu
>            Priority: Minor
>         Attachments: image-2018-12-04-15-53-49-983.png
>
>
> HI I am trying to build a cube with Kylin, the data gets souced fine from sqoop but the
next step for creating hive tables fails . Looking at the command being fired it looks weird
as the create statement looks good to me .
> I think the issue is with DOUBLE types as when I remove the same the create statement
works fine . Can someone please help .
> I am using the stack in AWS EMR, kylin 2.5 hive 2.3.0
> The errors logs with commands as as below, the table is a msql table which had columns
with DOUBLE type
> Command
> {code:java}
> hive -e "USE default; DROP TABLE IF EXISTS kylin_intermediate_fm_inv_holdings_8a1c33df_d12b_3609_13ee_39e169169368;
CREATE EXTERNAL TABLE IF NOT EXISTS kylin_intermediate_fm_inv_holdings_8a1c33df_d12b_3609_13ee_39e169169368
( HOLDINGS_STOCK_INVESTOR_ID string ,STOCK_INVESTORS_CHANNEL string ,STOCK_STOCK_ID string
,STOCK_DOMICILE string ,STOCK_STOCK_NM string ,STOCK_APPROACH string ,STOCK_STOCK_TYP string
,INVESTOR_ID string ,INVESTOR_NM string ,INVESTOR_DOMICILE_CNTRY string ,CLIENT_NM string
,INVESTOR_HOLDINGS_GROSS_ASSETS_USD double(22) ,INVESTOR_HOLDINGS_NET_ASSETS_USD double(22)
) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' STORED AS TEXTFILE LOCATION 's3://wfg1tst-models/kylin/kylin_metadata/kylin-4ae3b18b-831b-da66-eb8c-7318245c4448/kylin_intermediate_fm_inv_holdings_8a1c33df_d12b_3609_13ee_39e169169368';
ALTER TABLE kylin_intermediate_fm_inv_holdings_8a1c33df_d12b_3609_13ee_39e169169368 SET TBLPROPERTIES('auto.purge'='true');
" --hiveconf hive.merge.mapredfiles=false --hiveconf hive.auto.convert.join=true --hiveconf
dfs.replication=2 --hiveconf hive.exec.compress.output=true --hiveconf hive.auto.convert.join.noconditionaltask=true
--hiveconf mapreduce.job.split.metainfo.maxsize=-1 --hiveconf hive.merge.mapfiles=false --hiveconf
hive.auto.convert.join.noconditionaltask.size=100000000 --hiveconf hive.stats.autogather=true{code}
> Error is as below
> {code:java}
> OK Time taken: 1.315 seconds OK Time taken: 0.09 seconds MismatchedTokenException(334!=347)
at org.antlr.runtime.BaseRecognizer.recoverFromMismatchedToken(BaseRecognizer.java:617) at
org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:115) at org.apache.hadoop.hive.ql.parse.HiveParser.createTableStatement(HiveParser.java:6179)
at org.apache.hadoop.hive.ql.parse.HiveParser.ddlStatement(HiveParser.java:3808) at org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:2382)
at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1333) at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:204)
at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:77) at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:70)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:468) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1316)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1456) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1236)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1226) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:336) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:787)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at
java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136) FAILED: ParseException line 15:42 mismatched
input '(' expecting ) near 'double' in create table statement{code}
> [hadoop|https://stackoverflow.com/questions/tagged/hadoop] [hive|https://stackoverflow.com/questions/tagged/hive] [kylin|https://stackoverflow.com/questions/tagged/kylin]
>  
> More details here  [https://stackoverflow.com/questions/53377623/error-while-creating-hive-table-through-kylin-build-cube] .
Though the same got solved when I used DECIMAL type with precision .
>  
> Also observer that if you use DECIMAL(10) it does not work unless it has a precision
while building mode for the same it gives error as tries to use precision as undefined ie
. DECIMAL(10,undefined)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message