Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5130D17C47 for ; Tue, 3 Feb 2015 11:44:37 +0000 (UTC) Received: (qmail 76750 invoked by uid 500); 3 Feb 2015 11:44:25 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 76550 invoked by uid 500); 3 Feb 2015 11:44:24 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 76475 invoked by uid 99); 3 Feb 2015 11:44:08 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Feb 2015 11:44:08 +0000 X-ASF-Spam-Status: No, hits=2.5 required=5.0 tests=FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of wget.null@gmail.com designates 74.125.82.44 as permitted sender) Received: from [74.125.82.44] (HELO mail-wg0-f44.google.com) (74.125.82.44) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Feb 2015 11:44:02 +0000 Received: by mail-wg0-f44.google.com with SMTP id z12so44143327wgg.3 for ; Tue, 03 Feb 2015 03:43:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:message-id:mime-version:subject:date:references :to:in-reply-to; bh=n7KUB5d7749/M58M1i4FoOBwqXHnjo8CYxhonwiJRjk=; b=uiBqioAcJQPN7bMxJaMaJa8c+XciRyZvSgSnIIM0erXWtvrUyjbXjW3j6wEddw9a3C 1Dk37kpIyMNlrU7FA2O6yrBXrNl4krPU9vBgVVuRpEXsCiLZW8Un5Cgjhj7Duprb/DvD bFD1yPcg0TkVErIeanrcAv6I5ITWtzQqlcjuzZG9BjLazzpXsyrOBwW7F5oNgcu1HRnS chAhZ0izV5hWSWZ/hOlIdQ6BuFpM0sYoX9nDpmiMzKGN0orpLjQRBfAw15Etj9SPBBFg y90xdVIi5170gD6CwhhL8xBkYFLacb2RknLTXEbkF9bk3ukg1DjiUb3Nvo3l1aoDSAx3 ePkw== X-Received: by 10.194.2.147 with SMTP id 19mr26611772wju.51.1422963820897; Tue, 03 Feb 2015 03:43:40 -0800 (PST) Received: from [172.27.1.67] ([194.127.8.150]) by mx.google.com with ESMTPSA id ej10sm20289341wib.1.2015.02.03.03.43.38 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 03 Feb 2015 03:43:39 -0800 (PST) From: Alexander Alten-Lorenz Content-Type: multipart/alternative; boundary="Apple-Mail=_F63A124A-03DA-4A3B-B2E4-60FE11760EE4" Message-Id: <90E7BA82-441F-4508-9D3C-96210323D8F4@gmail.com> Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: tools.DistCp: Invalid arguments Date: Tue, 3 Feb 2015 12:43:37 +0100 References: <54CFD576.7020106@gmail.com> <56B55994-0CD1-4793-8A90-35D06EA0310C@gmail.com> <54D0AB86.2020808@gmail.com> To: user@hadoop.apache.org In-Reply-To: <54D0AB86.2020808@gmail.com> X-Mailer: Apple Mail (2.2070.6) X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail=_F63A124A-03DA-4A3B-B2E4-60FE11760EE4 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi, Can you please try webhdfs instead hdfs? - Alexander=20 > On 03 Feb 2015, at 12:05, xeonmailinglist = wrote: >=20 > Maybe this has to do with this error=E2=80=A6 I can=E2=80=99t do ls to = my own machine using the command below. Can this be related to the other = problem? Shouldn't I list the files with this command? > vagrant@hadoop-coc-1:~$ hdfs dfs -ls hdfs://192.168.56.100/ > ls: Call =46rom hadoop-coc-1/192.168.56.100 to hadoop-coc-1:8020 = failed on connection exception: java.net.ConnectException: Connection = refused; For more details see: = http://wiki.apache.org/hadoop/ConnectionRefused = > On 02-02-2015 19:59, Alexander Alten-Lorenz wrote: >=20 >=20 >=20 >> Have a closer look: >>=20 >>> hdfs://hadoop-coc-2:50070/ >> No Path is given. >>=20 >>=20 >>> On 02 Feb 2015, at 20:52, xeonmailinglist > wrote: >>>=20 >>> Hi, >>>=20 >>> I am trying to copy data using distcp but I get this error. Both = hadoop runtime are working properly. Why is this happening? >>>=20 >>>=20 >>> vagrant@hadoop-coc-1:~/Programs/hadoop$ hadoop distcp = hdfs://hadoop-coc-1:50070/input1 = hdfs://hadoop-coc-2:50070/ >>> 15/02/02 19:46:37 ERROR tools.DistCp: Invalid arguments:=20 >>> java.io.IOException: Failed on local exception: = com.google.protobuf.InvalidProtocolBufferException: Protocol message = end-group tag did not match expected tag.; Host Details : local host is: = "hadoop-coc-1/127.0.1.1"; destination host is: "hadoop-coc-2":50070;=20 >>> at = org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772) >>> at org.apache.hadoop.ipc.Client.call(Client.java:1472) >>> at org.apache.hadoop.ipc.Client.call(Client.java:1399) >>> at = org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.j= ava:232) >>> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source) >>> at = org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFi= leInfo(ClientNamenodeProtocolTranslatorPB.java:752) >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at = sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:= 57) >>> at = sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm= pl.java:43) >>> at java.lang.reflect.Method.invoke(Method.java:606) >>> at = org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvoca= tionHandler.java:187) >>> at = org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHa= ndler.java:102) >>> at com.sun.proxy.$Proxy10.getFileInfo(Unknown Source) >>> at = org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1988) >>> at = org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSyst= em.java:1118) >>> at = org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSyst= em.java:1114) >>> at = org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver= .java:81) >>> at = org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFile= System.java:1114) >>> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1400) >>> at = org.apache.hadoop.tools.DistCp.setTargetPathExists(DistCp.java:188) >>> at org.apache.hadoop.tools.DistCp.run(DistCp.java:111) >>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >>> at org.apache.hadoop.tools.DistCp.main(DistCp.java:401) >>> Caused by: com.google.protobuf.InvalidProtocolBufferException: = Protocol message end-group tag did not match expected tag. >>> at = com.google.protobuf.InvalidProtocolBufferException.invalidEndTag(InvalidPr= otocolBufferException.java:94) >>> at = com.google.protobuf.CodedInputStream.checkLastTagWas(CodedInputStream.java= :124) >>> at = com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:20= 2) >>> at = com.google.protobuf.AbstractParser.parsePartialDelimitedFrom(AbstractParse= r.java:241) >>> at = com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:= 253) >>> at = com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:= 259) >>> at = com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:= 49) >>> at = org.apache.hadoop.ipc.protobuf.RpcHeaderProtos$RpcResponseHeaderProto.pars= eDelimitedFrom(RpcHeaderProtos.java:3167) >>> at = org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:107= 2) >>> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:966) >>> Invalid arguments: Failed on local exception: = com.google.protobuf.InvalidProtocolBufferException: Protocol message = end-group tag did not match expected tag.; Host Details : local host is: = "hadoop-coc-1/127.0.1.1"; destination host is: "hadoop-coc-2":50070;=20 >>> usage: distcp OPTIONS [source_path...] >>> Thanks, >>>=20 >>=20 >=20 >=20 --Apple-Mail=_F63A124A-03DA-4A3B-B2E4-60FE11760EE4 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
Hi,

Can you please try webhdfs instead = hdfs?

- = Alexander 

On 03 Feb 2015, at 12:05, xeonmailinglist = <xeonmailinglist@gmail.com> wrote:

=20 =20

Maybe this has to do with this error=E2=80=A6 I can=E2=80=99t do ls= to my own machine using the command below. Can this be related to the other problem? Shouldn't I list the files with this command?

vagrant@hadoop-coc-1:~$ hdfs dfs -ls hdfs://192.168.56.100/
ls: Call =46rom hadoop-coc-1/192.168.56.100 to hadoop-coc-1:8020 failed =
on connection exception: java.net.ConnectException: Connection refused; =
For more details see:  http://wiki.apach=
e.org/hadoop/ConnectionRefused

On = 02-02-2015 19:59, Alexander Alten-Lorenz wrote:



Have a closer look:

No Path is given.


On 02 Feb 2015, at 20:52, xeonmailinglist <xeonmailinglist@gmail.com> wrote:

Hi,

I= am trying to copy data using distcp but I get this error. Both hadoop runtime are working properly. Why is this happening?


vagrant@hadoop-coc-1:~/Programs/hadoop$ hadoop distcp hdfs://hadoop-coc-1:50070/input1 hdfs://hadoop-coc-2:50070/
15/02/02 19:46:37 ERROR tools.DistCp: Invalid arguments:=20
java.io.IOException: Failed on local exception: =
com.google.protobuf.InvalidProtocolBufferException: Protocol message =
end-group tag did not match expected tag.; Host Details : local host is: =
"hadoop-coc-1/127.0.1.1"; destination host is: "hadoop-coc-2":50070;=20
    at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
    at org.apache.hadoop.ipc.Client.call(Client.java:1472)
    at org.apache.hadoop.ipc.Client.call(Client.java:1399)
    at =
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.j=
ava:232)
    at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
    at =
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFi=
leInfo(ClientNamenodeProtocolTranslatorPB.java:752)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at =
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:=
57)
    at =
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm=
pl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at =
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvoca=
tionHandler.java:187)
    at =
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHa=
ndler.java:102)
    at com.sun.proxy.$Proxy10.getFileInfo(Unknown Source)
    at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1988)
    at =
org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSyst=
em.java:1118)
    at =
org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSyst=
em.java:1114)
    at =
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver=
.java:81)
    at =
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFile=
System.java:1114)
    at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1400)
    at =
org.apache.hadoop.tools.DistCp.setTargetPathExists(DistCp.java:188)
    at org.apache.hadoop.tools.DistCp.run(DistCp.java:111)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.hadoop.tools.DistCp.main(DistCp.java:401)
Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol =
message end-group tag did not match expected tag.
    at =
com.google.protobuf.InvalidProtocolBufferException.invalidEndTag(InvalidPr=
otocolBufferException.java:94)
    at =
com.google.protobuf.CodedInputStream.checkLastTagWas(CodedInputStream.java=
:124)
    at =
com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:20=
2)
    at =
com.google.protobuf.AbstractParser.parsePartialDelimitedFrom(AbstractParse=
r.java:241)
    at =
com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:=
253)
    at =
com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:=
259)
    at =
com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:=
49)
    at =
org.apache.hadoop.ipc.protobuf.RpcHeaderProtos$RpcResponseHeaderProto.pars=
eDelimitedFrom(RpcHeaderProtos.java:3167)
    at =
org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:107=
2)
    at org.apache.hadoop.ipc.Client$Connection.run(Client.java:966)
Invalid arguments: Failed on local exception: =
com.google.protobuf.InvalidProtocolBufferException: Protocol message =
end-group tag did not match expected tag.; Host Details : local host is: =
"hadoop-coc-1/127.0.1.1"; destination host is: "hadoop-coc-2":50070;=20
usage: distcp OPTIONS [source_path...] <target_path>

Thanks,

=E2=80=8B



=E2=80=8B

= --Apple-Mail=_F63A124A-03DA-4A3B-B2E4-60FE11760EE4--