giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ping Jin <jinpi...@gmail.com>
Subject Re: giraph-hive having problem with cdh-4.4 (hadoop2.0)
Date Tue, 19 Nov 2013 17:49:27 GMT
hi Roman,
Thank you.
Do you mean the hive cmd line I use to ran the job?
I uses the Hive downloaded from same CDH version (CDH4.4). And I have no
problem running hive queries against the hadoop cluster.

If you mean the hive dependency in Giraph source, it's pulling 0.11:
    <dep.hive.version>0.11.0</dep.hive.version>
I check hive release, it says 0.11 support Hadoop 2.x.y

I'm suspecting the Hive-IO might be the cause. Hive-IO was compiled against
hadoop 0.20.
But I checked Hive-IO page and can't find a version that support newer
Hadoop.

Any suggestion?

Thanks,
-Ping


2013/11/18 Roman Shaposhnik <roman@shaposhnik.org>

> I could bet this is because the default version of Hive you're pulling
> is compiled
> against Hadoop 1, not Hadoop from CDH.
>
> If you want to run against a CDH cluster -- you have to make sure you
> change
> versions of all dependecies to be CDH ones (take a look at the properties
> section of the Giraph's root pom file).
>
> Thanks,
> Roman.
>
> On Mon, Nov 18, 2013 at 6:11 PM, Ping Jin <jinpingv@gmail.com> wrote:
> > hi,
> > I'm a new user of giraph. I'm trying to setup giraph-hive to work with
> the
> > cdh4.4 Hive and Hadoop.
> > I successfully built and run the SimpleShortestPath job on cdh-4.4 Hadoop
> > cluster.
> > However when I try to setup Giraph-Hive and run a job through
> > GiraphHiveRunner, I got following exceptions:
> >
> > Exception in thread "main" java.lang.IncompatibleClassChangeError: Found
> > interface org.apache.hadoop.mapreduce.JobContext, but class was expected
> >
> > at
> >
> com.facebook.hiveio.output.HiveApiOutputFormat.checkOutputSpecs(HiveApiOutputFormat.java:247)
> >
> > at
> >
> org.apache.giraph.hive.output.HiveVertexOutputFormat.checkOutputSpecs(HiveVertexOutputFormat.java:108)
> >
> > at
> >
> org.apache.giraph.io.internal.WrappedVertexOutputFormat.checkOutputSpecs(WrappedVertexOutputFormat.java:104)
> >
> > at
> >
> org.apache.giraph.bsp.BspOutputFormat.checkOutputSpecs(BspOutputFormat.java:52)
> >
> > at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:984)
> >
> > at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:945)
> >
> > at java.security.AccessController.doPrivileged(Native Method)
> >
> > at javax.security.auth.Subject.doAs(Subject.java:394)
> >
> > at
> >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
> >
> > at
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:945)
> >
> > at org.apache.hadoop.mapreduce.Job.submit(Job.java:566)
> >
> > at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:246)
> >
> > at org.apache.giraph.hive.HiveGiraphRunner.run(HiveGiraphRunner.java:275)
> >
> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> >
> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> >
> > at
> org.apache.giraph.hive.HiveGiraphRunner.main(HiveGiraphRunner.java:246)
> >
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >
> > at java.lang.reflect.Method.invoke(Method.java:597)
> >
> > at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> >
> >
> > Any one can help me figure out what's going wrong?
> >
> >
> > Thanks,
> >
> > -Ping
>

Mime
View raw message