giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eli Reisman <apache.mail...@gmail.com>
Subject Re: I'm stucked with this error: Error starting MRAppMaster java.lang.VerifyError: class org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto overrides final method getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet;
Date Wed, 01 Aug 2018 10:11:43 GMT
hi francesco,

I think in the mailing list you might find some tips for running on hadoop
2.5.x but IIRC hadoop 2.3.x plus appropriate Giraph build works. Again,
given the error msg I'd also carefully review your command line options and
config file just in case.

Also for context as far as I know no one is actively maintaining Giraph on
YARN compatibility for newer versions right now so taking a look at the
mailing list backscroll will probably yield the best info on which
combinations work best

On Mon, Jul 30, 2018, 11:52 AM Francesco Sclano <sclano.francesco@gmail.com>
wrote:

> Hi Eli,
> ok, thanks for you suggestion. I'll try it and let you know. In order to
> be sure, before try to use an earlier version of yarn/hadoop2, I'm using
> giraph-dist-1.2.0-hadoop2-bin.tar.gz. Does this giraph version officially
> support hadoop-2.5.1 with yarn? If not how I have to configure
> hadoop-2.5.1? Thanks
>
> 2018-07-29 9:47 GMT+02:00 Eli Reisman <apache.mailbox@gmail.com>:
>
>> hey francesco sorry im on my phone and have not been running giraph on
>> yarn for some time but if i had to guess this looks like a version
>> compatibility issue with the version of yarn the giraph is built against
>> compared to what you ran on ec2 or perhaps the version of protobuf dep in
>> the giraph build and the yarn cluster? either way id try an earlier version
>> of yarn/hadoop2 for ec2 and your giraph build, as giraph hasnt been keeping
>> up with all the yarn side changes.
>>
>> alternate theory is i have seen startup argument and config errors show
>> up with ambiguous traces similar to that. not able to review your conf file
>> from phone but you might review configs and args against giraph and yarn
>> code directly to ensure all those are good since docs might be outdated on
>> some of those.
>>
>> Good luck!
>>
>> On Sat, Jul 28, 2018, 9:42 PM Francesco Sclano <
>> sclano.francesco@gmail.com> wrote:
>>
>>> Hi,
>>> I'm using giraph-1.2 for my master thesis in computer science. I
>>> developed in giraph the calculus of 4-profiles like
>>> eelenberg.github.io/Elenberg4profileWWW16.pdf
>>> I succesfully configured and runned giraph-1.2 and hadoop-2.5.1 in
>>> pseudo distributed mode on my local pc with this configuration
>>> <https://we.tl/dB0PmGhUPX>.
>>>
>>> In pseudo distributed mode I launch my giraph program with following
>>> parameters:
>>> giraph MY_JAR.jar SUPERSTEP0_CLASS -ca
>>> giraph.master.observers=MY_OBSERVER_CLASS -mc MY_MASTER_CLASS -eif
>>> MY_CUSTOM_INPUT_FORMAT -eip input.txt -vof
>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op output -w 1
>>> -ca giraph.SplitMasterWorker=false -ca io.edge.reverse.duplicator=true
>>>
>>> Then I tryed to use amazon ec2 with a simple cluster of 3 nodes with
>>> this other configuration <https://we.tl/Gbv9q0nQ7h> but I obtain the
>>> error below. I also re-tryed pseudo-distributed mode in amazon ec2 with
>>> only 1 node and I obtain the same error below, but on my local pc it
>>> works!!!
>>> I'm using ubuntu 16.04 both in local and in ec2.
>>>
>>> Obviously I tried an hadoop example on my ec2 cluster and it works.
>>> Please help me because I'm blocked with my master this for this error! I
>>> red many forums about this error and I checked many times that I have only
>>> protobuf-java-2.5.0.jar inside hadoop-2.5.2 and giraph-1.2 both in ec2 and
>>> in my local pc.
>>>
>>>
>>>
>>> FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error
>>> starting MRAppMaster
>>> java.lang.VerifyError: class
>>> org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto overrides final
>>> method getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet;
>>>         at java.lang.ClassLoader.defineClass1(Native Method)
>>>         at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
>>>         at
>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>>>         at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
>>>         at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>         at java.lang.Class.getDeclaredConstructors0(Native Method)
>>>         at
>>> java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
>>>         at java.lang.Class.getConstructor0(Class.java:3075)
>>>         at java.lang.Class.getConstructor(Class.java:1825)
>>>         at
>>> org.apache.hadoop.yarn.factories.impl.pb.RecordFactoryPBImpl.newRecordInstance(RecordFactoryPBImpl.java:62)
>>>         at org.apache.hadoop.yarn.util.Records.newRecord(Records.java:36)
>>>         at
>>> org.apache.hadoop.yarn.api.records.ApplicationId.newInstance(ApplicationId.java:49)
>>>         at
>>> org.apache.hadoop.yarn.util.ConverterUtils.toApplicationAttemptId(ConverterUtils.java:137)
>>>         at
>>> org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:177)
>>>         at
>>> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1391)
>>>
>>> INFO [main] org.apache.hadoop.util.ExitUtil: Exiting with status 1
>>>
>>>
>>> Links of configurafion files works until 4 August 2018 but I can re-send
>>> them.
>>>
>>> Many Thanks
>>>
>>> Best Regards
>>>
>>> --
>>> Francesco Sclano
>>>
>>
>
>
> --
> Francesco Sclano
>

Mime
View raw message