hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lefty Leverenz (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-8588) sqoop REST endpoint fails to send appropriate JDBC driver to the cluster
Date Sun, 26 Oct 2014 01:19:34 GMT

    [ https://issues.apache.org/jira/browse/HIVE-8588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14184347#comment-14184347
] 

Lefty Leverenz commented on HIVE-8588:
--------------------------------------

Doc note:  This adds WebHCat configuration parameters *templeton.sqoop.archive* and *templeton.sqoop.home*,
and changes the default value of *templeton.sqoop.path* (which isn't documented in the wiki
yet -- see HIVE-6940 and comments on HIVE-4725).

They need to be documented here:

* [WebHCat Configuration -- Configuration Variables | https://cwiki.apache.org/confluence/display/Hive/WebHCat+Configure#WebHCatConfigure-ConfigurationVariables]

Will that be sufficient, or should there be a new section providing guidance and examples?

> sqoop REST endpoint fails to send appropriate JDBC driver to the cluster
> ------------------------------------------------------------------------
>
>                 Key: HIVE-8588
>                 URL: https://issues.apache.org/jira/browse/HIVE-8588
>             Project: Hive
>          Issue Type: Bug
>          Components: WebHCat
>    Affects Versions: 0.14.0
>            Reporter: Eugene Koifman
>            Assignee: Eugene Koifman
>            Priority: Critical
>              Labels: TODOC14
>             Fix For: 0.14.0, 0.15.0
>
>         Attachments: HIVE-8588.1.patch, HIVE-8588.2.patch
>
>
> This is originally discovered by [~deepesh]
> When running a Sqoop integration test from WebHCat
> {noformat}
> curl --show-error -d command="export -libjars hdfs:///tmp/mysql-connector-java.jar --connect
jdbc:mysql://deepesh-c6-1.cs1cloud.internal/sqooptest --username sqoop --password passwd --export-dir
/tmp/templeton_test_data/sqoop --table person" -d statusdir=sqoop.output -X POST "http://deepesh-c6-1.cs1cloud.internal:50111/templeton/v1/sqoop?user.name=hrt_qa"
> {noformat}
> the job is failing with the following error:
> {noformat}
> $ hadoop fs -cat /user/hrt_qa/sqoop.output/stderr
> 14/10/15 23:52:53 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5.2.2.0.0-897
> 14/10/15 23:52:53 WARN tool.BaseSqoopTool: Setting your password on the command-line
is insecure. Consider using -P instead.
> 14/10/15 23:52:54 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
> 14/10/15 23:52:54 INFO tool.CodeGenTool: Beginning code generation
> 14/10/15 23:52:54 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException:
Could not load db driver class: com.mysql.jdbc.Driver
> java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver
> 	at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:848)
> 	at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
> 	at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:736)
> 	at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:759)
> 	at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:269)
> 	at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:240)
> 	at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:226)
> 	at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295)
> 	at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1773)
> 	at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1578)
> 	at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96)
> 	at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:64)
> 	at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:100)
> 	at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
> 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> 	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
> 	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
> 	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
> 	at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
> {noformat}
> Note that the Sqoop tar bundle does not contain the JDBC connector jar. I think the problem
here maybe that the mysql connector jar added to libjars isn't available to the Sqoop tool
which first connects to the database through JDBC driver to collect some table information
before running the MR job. libjars will only add the connector jar for the MR job and not
the local one.
> NO PRECOMMIT TESTS



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message