hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shekhar Sharma <shekhar2...@gmail.com>
Subject Re: InvalidProtocolBufferException while submitting crunch job to cluster
Date Sat, 31 Aug 2013 14:44:46 GMT
Can you please check whether are you able to access HDFS using java
API..and also able to run MR Job.
Regards,
Som Shekhar Sharma
+91-8197243810


On Sat, Aug 31, 2013 at 7:08 PM, Narlin M <hpnole@gmail.com> wrote:
> The <server_address> that was mentioned in my original post is not
> pointing to bdatadev. I should have mentioned this in my original post,
> sorry I missed that.
>
> On 8/31/13 8:32 AM, "Narlin M" <hpnole@gmail.com> wrote:
>
>>I would, but bdatadev is not one of my servers, it seems like a random
>>host name. I can't figure out how or where this name got generated. That's
>>what puzzling me.
>>
>>On 8/31/13 5:43 AM, "Shekhar Sharma" <shekhar2581@gmail.com> wrote:
>>
>>>: java.net.UnknownHostException: bdatadev
>>>
>>>
>>>edit your /etc/hosts file
>>>Regards,
>>>Som Shekhar Sharma
>>>+91-8197243810
>>>
>>>
>>>On Sat, Aug 31, 2013 at 2:05 AM, Narlin M <hpnole@gmail.com> wrote:
>>>> Looks like I was pointing to incorrect ports. After correcting the port
>>>> numbers,
>>>>
>>>> conf.set("fs.defaultFS", "hdfs://<server_address>:8020");
>>>> conf.set("mapred.job.tracker", "<server_address>:8021");
>>>>
>>>> I am now getting the following exception:
>>>>
>>>> 2880 [Thread-15] INFO
>>>> org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob
>>>>-
>>>> java.lang.IllegalArgumentException: java.net.UnknownHostException:
>>>>bdatadev
>>>> at
>>>>
>>>>org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.j
>>>>a
>>>>va:414)
>>>> at
>>>>
>>>>org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.
>>>>j
>>>>ava:164)
>>>> at
>>>>
>>>>org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:
>>>>1
>>>>29)
>>>> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:389)
>>>> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:356)
>>>> at
>>>>
>>>>org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileS
>>>>y
>>>>stem.java:124)
>>>> at
>>>>org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2218)
>>>> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:80)
>>>> at
>>>>org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2252)
>>>> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2234)
>>>> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:300)
>>>> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:194)
>>>> at
>>>>
>>>>org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissi
>>>>o
>>>>nFiles.java:103)
>>>> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:902)
>>>> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:896)
>>>> at java.security.AccessController.doPrivileged(Native Method)
>>>> at javax.security.auth.Subject.doAs(Subject.java:396)
>>>> at
>>>>
>>>>org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformatio
>>>>n
>>>>.java:1332)
>>>> at
>>>>org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:896)
>>>> at org.apache.hadoop.mapreduce.Job.submit(Job.java:531)
>>>> at
>>>>
>>>>org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob.su
>>>>b
>>>>mit(CrunchControlledJob.java:305)
>>>> at
>>>>
>>>>org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchJobControl.start
>>>>R
>>>>eadyJobs(CrunchJobControl.java:180)
>>>> at
>>>>
>>>>org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchJobControl.pollJ
>>>>o
>>>>bStatusAndStartNewOnes(CrunchJobControl.java:209)
>>>> at
>>>>
>>>>org.apache.crunch.impl.mr.exec.MRExecutor.monitorLoop(MRExecutor.java:10
>>>>0
>>>>)
>>>> at
>>>>org.apache.crunch.impl.mr.exec.MRExecutor.access$000(MRExecutor.java:51)
>>>> at org.apache.crunch.impl.mr.exec.MRExecutor$1.run(MRExecutor.java:75)
>>>> at java.lang.Thread.run(Thread.java:680)
>>>> Caused by: java.net.UnknownHostException: bdatadev
>>>> ... 27 more
>>>>
>>>> However nowhere in my code a host named "bdatadev" is mentioned, and I
>>>> cannot ping this host.
>>>>
>>>> Thanks for the help.
>>>>
>>>>
>>>> On Fri, Aug 30, 2013 at 3:04 PM, Narlin M <hpnole@gmail.com> wrote:
>>>>>
>>>>> I am getting following exception while trying to submit a crunch
>>>>>pipeline
>>>>> job to a remote hadoop cluster:
>>>>>
>>>>> Exception in thread "main" java.lang.RuntimeException: Cannot create
>>>>>job
>>>>> output directory /tmp/crunch-324987940
>>>>> at
>>>>>
>>>>>org.apache.crunch.impl.mr.MRPipeline.createTempDirectory(MRPipeline.jav
>>>>>a
>>>>>:344)
>>>>> at org.apache.crunch.impl.mr.MRPipeline.<init>(MRPipeline.java:125)
>>>>> at test.CrunchTest.setup(CrunchTest.java:98)
>>>>> at test.CrunchTest.main(CrunchTest.java:367)
>>>>> Caused by: java.io.IOException: Failed on local exception:
>>>>> com.google.protobuf.InvalidProtocolBufferException: Protocol message
>>>>> end-group tag did not match expected tag.; Host Details : local host
>>>>>is:
>>>>> "NARLIN/127.0.0.1"; destination host is: "<server_address>":50070;
>>>>> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:759)
>>>>> at org.apache.hadoop.ipc.Client.call(Client.java:1164)
>>>>> at
>>>>>
>>>>>org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngin
>>>>>e
>>>>>.java:202)
>>>>> at com.sun.proxy.$Proxy11.mkdirs(Unknown Source)
>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>> at
>>>>>
>>>>>sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja
>>>>>v
>>>>>a:39)
>>>>> at
>>>>>
>>>>>sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso
>>>>>r
>>>>>Impl.java:25)
>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>> at
>>>>>
>>>>>org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInv
>>>>>o
>>>>>cationHandler.java:164)
>>>>> at
>>>>>
>>>>>org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocatio
>>>>>n
>>>>>Handler.java:83)
>>>>> at com.sun.proxy.$Proxy11.mkdirs(Unknown Source)
>>>>> at
>>>>>
>>>>>org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mk
>>>>>d
>>>>>irs(ClientNamenodeProtocolTranslatorPB.java:425)
>>>>> at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:1943)
>>>>> at
>>>>>
>>>>>org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSyst
>>>>>e
>>>>>m.java:523)
>>>>> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1799)
>>>>> at
>>>>>
>>>>>org.apache.crunch.impl.mr.MRPipeline.createTempDirectory(MRPipeline.jav
>>>>>a
>>>>>:342)
>>>>> ... 3 more
>>>>> Caused by: com.google.protobuf.InvalidProtocolBufferException:
>>>>>Protocol
>>>>> message end-group tag did not match expected tag.
>>>>> at
>>>>>
>>>>>com.google.protobuf.InvalidProtocolBufferException.invalidEndTag(Invali
>>>>>d
>>>>>ProtocolBufferException.java:73)
>>>>> at
>>>>>
>>>>>com.google.protobuf.CodedInputStream.checkLastTagWas(CodedInputStream.j
>>>>>a
>>>>>va:124)
>>>>> at
>>>>>
>>>>>com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessa
>>>>>g
>>>>>eLite.java:213)
>>>>> at
>>>>>
>>>>>com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.j
>>>>>a
>>>>>va:746)
>>>>> at
>>>>>
>>>>>com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.j
>>>>>a
>>>>>va:238)
>>>>> at
>>>>>
>>>>>com.google.protobuf.AbstractMessageLite$Builder.mergeDelimitedFrom(Abst
>>>>>r
>>>>>actMessageLite.java:282)
>>>>> at
>>>>>
>>>>>com.google.protobuf.AbstractMessage$Builder.mergeDelimitedFrom(Abstract
>>>>>M
>>>>>essage.java:760)
>>>>> at
>>>>>
>>>>>com.google.protobuf.AbstractMessageLite$Builder.mergeDelimitedFrom(Abst
>>>>>r
>>>>>actMessageLite.java:288)
>>>>> at
>>>>>
>>>>>com.google.protobuf.AbstractMessage$Builder.mergeDelimitedFrom(Abstract
>>>>>M
>>>>>essage.java:752)
>>>>> at
>>>>>
>>>>>org.apache.hadoop.ipc.protobuf.RpcPayloadHeaderProtos$RpcResponseHeader
>>>>>P
>>>>>roto.parseDelimitedFrom(RpcPayloadHeaderProtos.java:985)
>>>>> at
>>>>>
>>>>>org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:882
>>>>>)
>>>>> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:813)
>>>>> 0    [Thread-3] WARN  org.apache.hadoop.util.ShutdownHookManager  -
>>>>> ShutdownHook 'ClientFinalizer' failed, java.lang.NoSuchMethodError:
>>>>> com.google.common.collect.LinkedListMultimap.values()Ljava/util/List;
>>>>> java.lang.NoSuchMethodError:
>>>>> com.google.common.collect.LinkedListMultimap.values()Ljava/util/List;
>>>>> at org.apache.hadoop.hdfs.SocketCache.clear(SocketCache.java:135)
>>>>> at org.apache.hadoop.hdfs.DFSClient.close(DFSClient.java:672)
>>>>> at
>>>>>
>>>>>org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSyste
>>>>>m
>>>>>.java:539)
>>>>> at
>>>>>org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2308)
>>>>> at
>>>>>
>>>>>org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.ja
>>>>>v
>>>>>a:2324)
>>>>> at
>>>>>
>>>>>org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.ja
>>>>>v
>>>>>a:54)
>>>>>
>>>>> Google search on this error yielded solutions that asked to confirm
>>>>>that
>>>>> /etc/hosts file contained the entry for NARLIN which it does in my
>>>>>case.
>>>>>
>>>>> Here's the code that I am using to set up the MRPipeline:
>>>>>
>>>>> Configuration conf = HBaseConfiguration.create();
>>>>>
>>>>> conf.set("fs.defaultFS", "hdfs://<server_address>:50070");
>>>>> conf.set("mapred.job.tracker", "<server_address>:50030");
>>>>>
>>>>> System.out.println("Hadoop configuration created.");
>>>>> System.out.println("Initializing crunch pipeline ...");
>>>>>
>>>>> conf.set("mapred.jar", "<path_to_jar_file>");
>>>>>
>>>>> pipeline = new MRPipeline(getClass(), "crunchjobtest", conf);
>>>>>
>>>>> Has anyone faced this issue before and knows how to resolve it/point
>>>>>out
>>>>> if I am missing anything?
>>>>>
>>>>> Thanks for the help.
>>>>
>>>>
>>
>>
>
>

Mime
View raw message