hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Question about sqoop command error
Date Mon, 28 Jul 2014 07:45:04 GMT
This appears to be an JDBC end issue (not Sqoop's issue). You likely
have a mix of different Oracle JDBC (non-DMS and DMS ones?) jars under
$SQOOP_HOME/lib or other locations. This is causing a class loading
conflict.

On Mon, Jul 28, 2014 at 11:09 AM, R J <rj201244@yahoo.com> wrote:
> Thanks a lot for the reply. I put it now under the lib directroy:
>
> $ pwd
> /home/catfish/scoop/sqoop-1.4.4.bin__hadoop-0.20/lib
> $ ls /home/catfish/scoop/sqoop-1.4.4.bin__hadoop-0.20/lib/ojdbc6.jar
> ojdbc6.jar
> $cd /home/catfish/scoop/sqoop-1.4.4.bin__hadoop-0.20/bin/
> $sqoop import --driver oracle.jdbc.driver.OracleDriver --connect
> jdbc:oracle:thin:@server123.mycot.com:1521/instance1 --username scott
> --password tiger --table tab1
>
> Now I get different error:
>
> Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.
> Please set $HCAT_HOME to the root of your HCatalog installation.
> 14/07/28 05:33:02 WARN tool.BaseSqoopTool: Setting your password on the
> command-line is insecure. Consider using -P instead.
> 14/07/28 05:33:03 WARN sqoop.ConnFactory: Parameter --driver is set to an
> explicit driver however appropriate connection manager is nanager). Sqoop is
> going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please
> specify explicitly which connection manag
> 14/07/28 05:33:03 INFO manager.SqlManager: Using default fetchSize of 1000
> 14/07/28 05:33:03 INFO tool.CodeGenTool: Beginning code generation
> Exception in thread "main" java.lang.NoClassDefFoundError:
> oracle/dms/console/DMSConsole
>         at oracle.jdbc.driver.DMSFactory.<clinit>(DMSFactory.java:46)
>         at
> oracle.jdbc.driver.PhysicalConnection.createDMSSensors(PhysicalConnection.java:4771)
>         at
> oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:651)
>         at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:233)
>         at
> oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:29)
>         at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:566)
>         at java.sql.DriverManager.getConnection(DriverManager.java:582)
>         at java.sql.DriverManager.getConnection(DriverManager.java:185)
>         at
> org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:801)
>
>         at
> org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
>         at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:660)
>         at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:683)
>         at
> org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:240)
>         at
> org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:223)
>         at
> org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:347)
>         at
> org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1277)
>         at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1089)
>         at
> org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96)
>         at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:396)
>         at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
>         at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>         at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>         at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
> Caused by: java.lang.ClassNotFoundException: oracle.dms.console.DMSConsole
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>         ... 26 more
>
>
>
>
> On Sunday, July 27, 2014 10:17 PM, Harsh J <harsh@cloudera.com> wrote:
>
>
> The jar must be placed under the $SQOOP_HOME/lib/ directory (the
> libraries location), not the $SQOOP_HOME/bin/ directory (which is for
> executable scripts/binaries only).
>
> On Mon, Jul 28, 2014 at 10:39 AM, R J <rj201244@yahoo.com> wrote:
>> Hi Gordon,
>>
>> Thank you for the reply. I downloaded the jdbc driver from
>>
>> http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-112010-090769.html
>> . I downloaded the file ojdbc6.jar and put it in the directory
>> /home/catfish/scoop/sqoop-1.4.4.bin__hadoop-0.20/bin/
>> $cd /home/catfish/scoop/sqoop-1.4.4.bin__hadoop-0.20/bin/
>> $ls ojdbc6.jar
>> ojdbc6.jar
>> $sqoop import --driver oracle.jdbc.driver.OracleDriver --connect
>> jdbc:oracle:thin:@server123.mycot.com:1521/instance1 --username scott
>> --password tiger --table tab1
>>
>> I still get the same error:
>> Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.
>> Please set $HCAT_HOME to the root of your HCatalog installation.
>> 14/07/28 05:08:20 WARN tool.BaseSqoopTool: Setting your password on the
>> command-line is insecure. Consider using -P instead.
>> 14/07/28 05:08:20 WARN sqoop.ConnFactory: Parameter --driver is set to an
>> explicit driver however appropriate connection manager is not being set
>> (via
>> --connection-manager). Sqoop is going to fall back to
>> org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly
>> which
>> connection manager should be used next time.
>> 14/07/28 05:08:20 INFO manager.SqlManager: Using default fetchSize of 1000
>> 14/07/28 05:08:20 INFO tool.CodeGenTool: Beginning code generation
>> 14/07/28 05:08:20 ERROR sqoop.Sqoop: Got exception running Sqoop:
>> java.lang.RuntimeException: Could not load db driver class:
>> oracle.jdbc.driver.OracleDriver
>>
>> java.lang.RuntimeException: Could not load db driver class:
>> oracle.jdbc.driver.OracleDriver
>>        at
>> org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:772)
>>        at
>>
>> org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
>>        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:660)
>>        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:683)
>>        at
>>
>> org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:240)
>>        at
>> org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:223)
>>        at
>> org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:347)
>>        at
>> org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1277)
>>        at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1089)
>>        at
>> org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96)
>>        at
>> org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:396)
>>        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
>>        at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>>        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>>        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>>        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>>        at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
>>
>>
>>
>>
>> On Sunday, July 27, 2014 8:49 PM, Gordon Wang <gwang@pivotal.io> wrote:
>>
>>
>> You did not download Oracle JDBC driver jar.
>>
>> Please download the JDBC driver from the vendor site. and follow this page
>>
>> http://sqoop.apache.org/docs/1.4.4/SqoopUserGuide.html#_connecting_to_a_database_server
>> to make sqoop work.
>>
>>
>> On Mon, Jul 28, 2014 at 9:35 AM, R J <rj201244@yahoo.com> wrote:
>>
>> Hi All,
>>
>> I installed sqoop and wanted to try the sqoop command on my linux server
>> where I have Hadoop environment running.
>>
>> I unzipped the scoop executable at:
>> /home/catfish/scoop/sqoop-1.4.4.bin__hadoop-0.20/bin/
>>
>> $sqoop import --driver oracle.jdbc.driver.OracleDriver --connect
>> jdbc:oracle:thin:@server123.mycot.com:1521/instance1 --username scott
>> --password tiger --table tab1
>> I get the following errors:
>> Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.
>> Please set $HCAT_HOME to the root of your HCatalog installation.
>> 14/07/28 01:31:26 WARN tool.BaseSqoopTool: Setting your password on the
>> command-line is insecure. Consider using -P instead.
>> 14/07/28 01:31:26 WARN sqoop.ConnFactory: Parameter --driver is set to an
>> explicit driver however appropriate connection manager is not being set
>> (via
>> --connection-manager). Sqoop is going to fall back to
>> org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly
>> which
>> connection manager should be used next time.
>> 14/07/28 01:31:26 INFO manager.SqlManager: Using default fetchSize of 1000
>> 14/07/28 01:31:26 INFO tool.CodeGenTool: Beginning code generation
>> 14/07/28 01:31:26 ERROR sqoop.Sqoop: Got exception running Sqoop:
>> java.lang.RuntimeException: Could not load db driver class:
>> oracle.jdbc.driver.OracleDriver
>> java.lang.RuntimeException: Could not load db driver class:
>> oracle.jdbc.driver.OracleDriver
>>        at
>> org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:772)
>>        at
>>
>> org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
>>        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:660)
>>        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:683)
>>        at
>>
>> org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:240)
>>        at
>> org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:223)
>>        at
>> org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:347)
>>        at
>> org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1277)
>>        at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1089)
>>        at
>> org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96)
>>        at
>> org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:396)
>>        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
>>        at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>>        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>>        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>>        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>>        at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
>>
>> Could you please suggest how I could make the sqoop command work? Thanks a
>> lot.
>>
>> Shu
>>
>>
>>
>>
>>
>> --
>> Regards
>> Gordon Wang
>
>>
>>
>
>
>
> --
> Harsh J
>
>
>



-- 
Harsh J

Mime
View raw message