kylin-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KYLIN-3500) kylin 2.4 use jdbc datasource :Unknown column 'A.A.CRT_DATE' in 'where clause'
Date Mon, 20 Aug 2018 08:02:00 GMT

    [ https://issues.apache.org/jira/browse/KYLIN-3500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16585578#comment-16585578
] 

ASF GitHub Bot commented on KYLIN-3500:
---------------------------------------

shaofengshi closed pull request #203: KYLIN-3500 fix duplicated tablename at createSqoopToFlatHiveStep
 wh…
URL: https://github.com/apache/kylin/pull/203
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/source-hive/src/main/java/org/apache/kylin/source/jdbc/JdbcHiveMRInput.java b/source-hive/src/main/java/org/apache/kylin/source/jdbc/JdbcHiveMRInput.java
index 74d95cf0ee..b8bc56faaf 100644
--- a/source-hive/src/main/java/org/apache/kylin/source/jdbc/JdbcHiveMRInput.java
+++ b/source-hive/src/main/java/org/apache/kylin/source/jdbc/JdbcHiveMRInput.java
@@ -184,15 +184,11 @@ private AbstractExecutable createSqoopToFlatHiveStep(String jobWorkingDir,
Strin
                 }
             }
 
-            //related to "kylin.engine.mr.config-override.mapreduce.job.queuename"
-            String queueName = getSqoopJobQueueName(config);
-            String cmd = String.format("%s/sqoop import -Dorg.apache.sqoop.splitter.allow_text_splitter=true
"
-                    + generateSqoopConfigArgString()
+            String cmd = String.format("%s/sqoop import" + generateSqoopConfigArgString()
                     + "--connect \"%s\" --driver %s --username %s --password %s --query \"%s
AND \\$CONDITIONS\" "
-                    + "--target-dir %s/%s --split-by %s.%s --boundary-query \"%s\" --null-string
'' "
+                    + "--target-dir %s/%s --split-by %s --boundary-query \"%s\" --null-string
'' "
                     + "--fields-terminated-by '%s' --num-mappers %d", sqoopHome, connectionUrl,
driverClass, jdbcUser,
-                    jdbcPass, selectSql, jobWorkingDir, hiveTable, splitTable, splitColumn,
bquery, filedDelimiter,
-                    mapperNum);
+                    jdbcPass, selectSql, jobWorkingDir, hiveTable, splitColumn, bquery, filedDelimiter,
mapperNum);
             logger.debug(String.format("sqoop cmd:%s", cmd));
             CmdStep step = new CmdStep();
             step.setCmd(cmd);
@@ -212,7 +208,7 @@ protected String generateSqoopConfigArgString() {
             config.putAll(SourceConfigurationUtil.loadSqoopConfiguration());
             config.putAll(kylinConfig.getSqoopConfigOverride());
 
-            StringBuilder args = new StringBuilder();
+            StringBuilder args = new StringBuilder(" -Dorg.apache.sqoop.splitter.allow_text_splitter=true
");
             for (Map.Entry<String, String> entry : config.entrySet()) {
                 args.append(" -D" + entry.getKey() + "=" + entry.getValue() + " ");
             }


 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> kylin 2.4 use jdbc datasource :Unknown column 'A.A.CRT_DATE' in 'where clause'
> ------------------------------------------------------------------------------
>
>                 Key: KYLIN-3500
>                 URL: https://issues.apache.org/jira/browse/KYLIN-3500
>             Project: Kylin
>          Issue Type: Bug
>          Components: RDBMS Source
>    Affects Versions: v2.3.0, v2.4.0
>         Environment: Hadoop2.7.6+Hive-1.2.2+HBase1.2.6+Sqoop1.4.7
>            Reporter: cqli
>            Assignee: Shaoxiong Zhan
>            Priority: Major
>             Fix For: v2.4.1, v2.5.0
>
>         Attachments: TIM图片20180815175301.png, TIM图片20180815175342.png, TIM图片20180815175407.png
>
>
> Kylin2.4 使用JDBC数据源,构建时通过Sqoop从mysql抽取数据到hive报错,split-by字段出现名重复:tablename.tablename.field
> sqoop-1.4.6/bin/sqoop import -Dorg.apache.sqoop.splitter.allow_text_splitter=true -Dmapreduce.job.queuename=default
--connect "jdbc:mysql://192.168.31.108:3306/kylin_jdbcresource_test" --driver com.mysql.jdbc.Driver
--username root --password root1324 --query "SELECT A.NAME as A_NAME ,A.CRT_DATE FROM KYLIN_JDBCRESOURCE_TEST.A
as A  WHERE 1=1 AND (A.CRT_DATE >= '2018-08-01 00:00:00' AND A.CRT_DATE < '2018-08-16
00:00:00')  AND \$CONDITIONS" --target-dir hdfs://localhost:9000/kylin/kylin_metadata/kylin-6bbe475b-ad11-4501-b3db-a56ee0222c90/kylin_intermediate_jdbc_resource_cube_7a4bd639_38ac_41be_9fba_6c64e040b175{color:#ff0000}
--split-by A.A.CRT_DATE{color} --boundary-query "SELECT min(A.CRT_DATE), max(A.CRT_DATE) FROM
"KYLIN_JDBCRESOURCE_TEST".A as A WHERE A.CRT_DATE >= '2018-08-01 00:00:00' AND A.CRT_DATE
< '2018-08-16 00:00:00'" --null-string '' --fields-terminated-by '|' --num-mappers 1
> Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException:{color:#ff0000}
Unknown column 'A.A.CRT_DATE' in 'where clause'{color}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message