sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josef Kafka <clojurian...@gmail.com>
Subject Re: Sqoop 1.4.4 targeting Hadoop 0.20.203.0 Fails
Date Mon, 06 Oct 2014 22:30:25 GMT
Thank you so much for your help,

It ran find with 1.3.0-cdh3u6, but we're getting the following error when
running the job against sqoop-1.4.5.jar. Notice the mismatchings of the
jars between apache and Cloudera namespaces. Could this be a bug in Sqoop?

Exception in thread "main" java.lang.IncompatibleClassChangeError: Found
class org.apache.hadoop.mapreduce.JobContext, but interface was expected
        at
org.apache.sqoop.config.ConfigurationHelper.getJobNumMaps(ConfigurationHelper.java:53)
        at *com.cloudera.sqoop.config.*
ConfigurationHelper.getJobNumMaps(ConfigurationHelper.java:36)
        at
org.apache.sqoop.mapreduce.db.DataDrivenDBInputFormat.getSplits(DataDrivenDBInputFormat.java:121)
        at
org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:902)
        at
org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:919)
        at org.apache.hadoop.mapred.JobClient.access$500(JobClient.java:170)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:838)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:791)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
        at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:791)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:465)
        at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:494)
        at
org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:186)
        at
org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:159)
        at
org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:247)
        at
org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:665)
        at
org.apache.sqoop.manager.OracleManager.importTable(OracleManager.java:444)
        at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:497)
        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:601)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
        at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:53)
        at
org.troposphere.mapreduce.bft.mrjobs.SqoopHDFSImportJob.runJob(SqoopHDFSImportJob.java:58)
        at
org.troposphere.mapreduce.commons.AbstractRunnable.run(AbstractRunnable.java:29)
        at org.troposphere.mapreduce.runner.JobRunner.run(JobRunner.java:62)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at
org.troposphere.mapreduce.runner.JobRunner.main(JobRunner.java:41)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)


It worked with 1.3.0-cdh3u6.

On Mon, Oct 6, 2014 at 2:46 PM, Abraham Elmahrek <abe@cloudera.com> wrote:

> Hey there,
>
> A couple of things stand out:
> 1. you need to provide "hadoopversion" for your changes to take effect.
> e.g. ant -Dhadoopversion=20 tar.
> 2. Why not use a 0.20 release from
> http://www.apache.org/dyn/closer.cgi/sqoop/1.4.5 ?
>
> -Abe
>
> On Mon, Oct 6, 2014 at 11:35 AM, Josef Kafka <clojurian123@gmail.com>
> wrote:
>
> > Hi,
> >
> > We are trying to find a version of Sqoop that runs against Hadoop
> > 0.20.203.0. I discovered the following on StackOverflow:
> >
> > "Sqoop have currently two main branches. Sqoop 1 is older fully
> > functional and mature project supporting Hadoop 0.20, 1.x, 0.23 and
> > 2.0.x You can download the bits from here. Please make sure that you
> > download file ending with "_hadoop-0.20", otherwise you will be
> > getting weird exceptions."
> >
> >
> >
> > What we did is to download the 1.4.4 branch of Sqoop from here:
> > https://github.com/apache/sqoop/tree/branch-1.4.4
> >
> >
> >
> > and change the build.xml file of Sqoop to target 0.20.203.0 before "ant
> > mvn-install":
> >
> >
> >   <!-- Set default Hadoop version if not set -->
> >   <if>
> >     <isset property="hadoopversion" />
> >     <then>
> >       <echo message="Use Hadoop ${hadoopversion}" />
> >     </then>
> >     <else>
> >       <echo message="Use Hadoop 2.x by default" />
> >       <property name="hadoopversion" value="*20*" />
> >     </else>
> >   </if>
> >
> >     <!-- Set variables per target hadoop version -->
> >     <if>
> >       <equals arg1="${hadoopversion}" arg2="*20*" />
> >       <then>
> >             <property name="hadoop.version" value="*0.20.203.0*" />
> >             <property name="hbase.version" value="0.94.2" />
> >             <property name="zookeeper.version" value="3.4.2" />
> >             <property name="hadoop.version.full" value="0.20.203.0" />
> >             <property name="hcatalog.version" value="0.11.0" />
> >       </then>
> >
> >     <elseif>
> >
> >
> >
> >
> > But when we try to run the job, we get the following error:
> >
> >
> > Exception in thread "main" java.lang.IncompatibleClassChangeError: Found
> > class org.apache.hadoop.mapreduce.JobContext, but interface was expected
> >         at
> >
> >
> org.apache.sqoop.config.ConfigurationHelper.getJobNumMaps(ConfigurationHelper.java:53)
> >         at
> >
> >
> com.cloudera.sqoop.config.ConfigurationHelper.getJobNumMaps(ConfigurationHelper.java:36)
> >         at
> >
> >
> org.apache.sqoop.mapreduce.db.DataDrivenDBInputFormat.getSplits(DataDrivenDBInputFormat.java:121)
> >         at
> > org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:902)
> >         at
> > org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:919)
> >         at
> > org.apache.hadoop.mapred.JobClient.access$500(JobClient.java:170)
> >         at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:838)
> >         at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:791)
> >         at java.security.AccessController.doPrivileged(Native Method)
> >         at javax.security.auth.Subject.doAs(Subject.java:415)
> >         at
> >
> >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> >         at
> > org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:791)
> >         at org.apache.hadoop.mapreduce.Job.submit(Job.java:465)
> >         at
> org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:494)
> >         at
> >
> >
> org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:186)
> >         at
> > org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:159)
> >         at
> >
> org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:239)
> >         at
> > org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:600)
> >         at
> >
> org.apache.sqoop.manager.OracleManager.importTable(OracleManager.java:384)
> >         at
> > org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:413)
> >         at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
> >         at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
> >         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> >         at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
> >         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
> >         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
> >         at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:53)
> >         at
> >
> >
> org.troposphere.mapreduce.bft.mrjobs.SqoopHDFSImportJob.runJob(SqoopHDFSImportJob.java:58)
> >         at
> >
> >
> org.troposphere.mapreduce.commons.AbstractRunnable.run(AbstractRunnable.java:29)
> >         at
> > org.troposphere.mapreduce.runner.JobRunner.run(JobRunner.java:62)
> >         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> >         at
> > org.troposphere.mapreduce.runner.JobRunner.main(JobRunner.java:41)
> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >         at
> >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >         at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >         at java.lang.reflect.Method.invoke(Method.java:606)
> >         at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message