hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From R J <rj201...@yahoo.com>
Subject Re: Question about sqoop command error
Date Mon, 28 Jul 2014 16:23:17 GMT
Thanks a lot. I removed other jdbc jar files except for the one ojdbc6.jar under 
/home/catfish/scoop/sqoop-1.4.4.bin__hadoop-0.20/lib/.

Now I ran the scoop command again:
$/home/catfish/scoop/sqoop-1.4.4.bin__hadoop-0.20/bin/sqoop import --driver oracle.jdbc.driver.OracleDriver
--connect jdbc:oracle:thin:@server123.mycot.com:1521/instance1 --username scott --password
tiger --table tab1


This time I get a different error as below. Not sure why it is trying to execute the query
"SELECT t.* FROM SOAK AS t WHERE 1=0". I checked the syntax of the sqoop command. Is there
an alternate command/syntax to obtain data from a table. 

Error I got:
Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
14/07/28 16:09:28 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure.
Consider using -P instead.
14/07/28 16:09:28 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver
however appropriate connection manager is not being set (via --connection-manager). Sqoop
is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly
which connection manager should be used next time.
14/07/28 16:09:28 INFO manager.SqlManager: Using default fetchSize of 1000
14/07/28 16:09:28 INFO tool.CodeGenTool: Beginning code generation
14/07/28 16:09:29 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM SOAK AS
t WHERE 1=0
14/07/28 16:09:29 ERROR manager.SqlManager: Error executing statement: java.sql.SQLSyntaxErrorException:
ORA-00933: SQL command not properly ended

java.sql.SQLSyntaxErrorException: ORA-00933: SQL command not properly ended

        at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
        at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
        at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
        at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
        at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
        at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
        at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
        at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
        at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
        at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
        at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
        at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
        at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:674)
        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:683)
        at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:240)
        at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:223)
        at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:347)
        at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1277)
        at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1089)
        at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96)
        at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:396)
        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
14/07/28 16:09:29 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException:
No columns to generate for ClassWriter
        at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1095)
        at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96)
        at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:396)
        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:238)





On Monday, July 28, 2014 12:45 AM, Harsh J <harsh@cloudera.com> wrote:
 


This appears to be an JDBC end issue (not Sqoop's issue). You likely
have a mix of different Oracle JDBC (non-DMS and DMS ones?) jars under
$SQOOP_HOME/lib or other locations. This is causing a class loading
conflict.


On Mon, Jul 28, 2014 at 11:09 AM, R J <rj201244@yahoo.com> wrote:
> Thanks a lot for the reply. I put it now under the lib directroy:
>
> $ pwd
> /home/catfish/scoop/sqoop-1.4.4.bin__hadoop-0.20/lib
> $ ls /home/catfish/scoop/sqoop-1.4.4.bin__hadoop-0.20/lib/ojdbc6.jar
> ojdbc6.jar
> $cd /home/catfish/scoop/sqoop-1.4.4.bin__hadoop-0.20/bin/
> $sqoop import --driver oracle.jdbc.driver.OracleDriver --connect
> jdbc:oracle:thin:@server123.mycot.com:1521/instance1 --username scott
> --password tiger --table tab1
>
> Now I get different error:
>
> Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.
> Please set $HCAT_HOME to the root of your HCatalog installation.
> 14/07/28 05:33:02 WARN tool.BaseSqoopTool: Setting your password on the
> command-line is insecure. Consider using -P instead.
> 14/07/28 05:33:03 WARN sqoop.ConnFactory: Parameter --driver is set to an
> explicit driver however appropriate connection manager is nanager). Sqoop is
> going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please
> specify explicitly which connection manag
> 14/07/28 05:33:03 INFO manager.SqlManager: Using default fetchSize of 1000
> 14/07/28 05:33:03 INFO tool.CodeGenTool: Beginning code generation
> Exception in thread "main" java.lang.NoClassDefFoundError:
> oracle/dms/console/DMSConsole
>         at oracle.jdbc.driver.DMSFactory.<clinit>(DMSFactory.java:46)
>         at
> oracle.jdbc.driver.PhysicalConnection.createDMSSensors(PhysicalConnection.java:4771)
>         at
> oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:651)
>         at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:233)
>         at
> oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:29)
>         at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:566)
>         at java.sql.DriverManager.getConnection(DriverManager.java:582)
>         at java.sql.DriverManager.getConnection(DriverManager.java:185)
>         at
> org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:801)
>
>         at
> org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
>         at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:660)
>         at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:683)
>         at
> org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:240)
>         at
> org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:223)
>         at
> org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:347)
>         at
> org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1277)
>         at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1089)
>         at
> org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96)
>         at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:396)
>         at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
>         at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>         at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>         at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
> Caused by: java.lang.ClassNotFoundException: oracle.dms.console.DMSConsole
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>         ... 26 more
>
>
>
>
> On Sunday, July 27, 2014 10:17 PM, Harsh J <harsh@cloudera.com> wrote:
>
>
> The jar must be placed under the $SQOOP_HOME/lib/ directory (the
> libraries location), not the $SQOOP_HOME/bin/ directory (which is for
> executable scripts/binaries only).
>
> On Mon, Jul 28, 2014 at 10:39 AM, R J <rj201244@yahoo.com> wrote:
>> Hi Gordon,
>>
>> Thank you for the reply. I downloaded the jdbc driver from
>>
>> http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-112010-090769.html
>> . I downloaded the file ojdbc6.jar and put it in the directory
>> /home/catfish/scoop/sqoop-1.4.4.bin__hadoop-0.20/bin/
>> $cd /home/catfish/scoop/sqoop-1.4.4.bin__hadoop-0.20/bin/
>> $ls ojdbc6.jar
>> ojdbc6.jar
>> $sqoop import --driver oracle.jdbc.driver.OracleDriver --connect
>> jdbc:oracle:thin:@server123.mycot.com:1521/instance1 --username scott
>> --password tiger --table tab1
>>
>> I still get the same error:
>> Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.
>> Please set $HCAT_HOME to the root of your HCatalog installation.
>> 14/07/28 05:08:20 WARN tool.BaseSqoopTool: Setting your password on the
>> command-line is insecure. Consider using -P instead.
>> 14/07/28 05:08:20 WARN sqoop.ConnFactory: Parameter --driver is set to an
>> explicit driver however appropriate connection manager is not being set
>> (via
>> --connection-manager). Sqoop is going to fall back to
>> org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly
>> which
>> connection manager should be used next time.
>> 14/07/28 05:08:20 INFO manager.SqlManager: Using default fetchSize of 1000
>> 14/07/28 05:08:20 INFO tool.CodeGenTool: Beginning code generation
>> 14/07/28 05:08:20 ERROR sqoop.Sqoop: Got exception running Sqoop:
>> java.lang.RuntimeException: Could not load db driver class:
>> oracle.jdbc.driver.OracleDriver
>>
>> java.lang.RuntimeException: Could not load db driver class:
>> oracle.jdbc.driver.OracleDriver
>>        at
>> org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:772)
>>        at
>>
>> org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
>>        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:660)
>>        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:683)
>>        at
>>
>> org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:240)
>>        at
>> org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:223)
>>        at
>> org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:347)
>>        at
>> org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1277)
>>        at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1089)
>>        at
>> org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96)
>>        at
>> org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:396)
>>        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
>>        at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>>        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>>        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>>        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>>        at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
>>
>>
>>
>>
>> On Sunday, July 27, 2014 8:49 PM, Gordon Wang <gwang@pivotal.io> wrote:
>>
>>
>> You did not download Oracle JDBC driver jar.
>>
>> Please download the JDBC driver from the vendor site. and follow this page
>>
>> http://sqoop.apache.org/docs/1.4.4/SqoopUserGuide.html#_connecting_to_a_database_server
>> to make sqoop work.
>>
>>
>> On Mon, Jul 28, 2014 at 9:35 AM, R J <rj201244@yahoo.com> wrote:
>>
>> Hi All,
>>
>> I installed sqoop and wanted to try the sqoop command on my linux server
>> where I have Hadoop environment running.
>>
>> I unzipped the scoop executable at:
>> /home/catfish/scoop/sqoop-1.4.4.bin__hadoop-0.20/bin/
>>
>> $sqoop import --driver oracle.jdbc.driver.OracleDriver --connect
>> jdbc:oracle:thin:@server123.mycot.com:1521/instance1 --username scott
>> --password tiger --table tab1
>> I get the following errors:
>> Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.
>> Please set $HCAT_HOME to the root of your HCatalog installation.
>> 14/07/28 01:31:26 WARN tool.BaseSqoopTool: Setting your password on the
>> command-line is insecure. Consider using -P instead.
>> 14/07/28 01:31:26 WARN sqoop.ConnFactory: Parameter --driver is set to an
>> explicit driver however appropriate connection manager is not being set
>> (via
>> --connection-manager). Sqoop is going to fall back to
>> org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly
>> which
>> connection manager should be used next time.
>> 14/07/28 01:31:26 INFO manager.SqlManager: Using default fetchSize of 1000
>> 14/07/28 01:31:26 INFO tool.CodeGenTool: Beginning code generation
>> 14/07/28 01:31:26 ERROR sqoop.Sqoop: Got exception running Sqoop:
>> java.lang.RuntimeException: Could not load db driver class:
>> oracle.jdbc.driver.OracleDriver
>> java.lang.RuntimeException: Could not load db driver class:
>> oracle.jdbc.driver.OracleDriver
>>        at
>> org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:772)
>>        at
>>
>> org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
>>        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:660)
>>        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:683)
>>        at
>>
>> org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:240)
>>        at
>> org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:223)
>>        at
>> org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:347)
>>        at
>> org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1277)
>>        at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1089)
>>        at
>> org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96)
>>        at
>> org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:396)
>>        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
>>        at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>>        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>>        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>>        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>>        at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
>>
>> Could you please suggest how I could make the sqoop command work? Thanks a
>> lot.
>>
>> Shu
>>
>>
>>
>>
>>
>> --
>> Regards
>> Gordon Wang
>
>>
>>
>
>
>
> --
> Harsh J
>
>
>



-- 
Harsh J
Mime
View raw message