From core-dev-return-45365-apmail-hadoop-core-dev-archive=hadoop.apache.org@hadoop.apache.org Mon Aug 04 16:39:18 2008 Return-Path: Delivered-To: apmail-hadoop-core-dev-archive@www.apache.org Received: (qmail 82283 invoked from network); 4 Aug 2008 16:39:17 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 4 Aug 2008 16:39:17 -0000 Received: (qmail 89156 invoked by uid 500); 4 Aug 2008 16:39:14 -0000 Delivered-To: apmail-hadoop-core-dev-archive@hadoop.apache.org Received: (qmail 89055 invoked by uid 500); 4 Aug 2008 16:39:14 -0000 Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: core-dev@hadoop.apache.org Delivered-To: mailing list core-dev@hadoop.apache.org Received: (qmail 89043 invoked by uid 99); 4 Aug 2008 16:39:14 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 04 Aug 2008 09:39:14 -0700 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.140] (HELO brutus.apache.org) (140.211.11.140) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 04 Aug 2008 16:38:19 +0000 Received: from brutus (localhost [127.0.0.1]) by brutus.apache.org (Postfix) with ESMTP id 3689E234C18C for ; Mon, 4 Aug 2008 09:38:46 -0700 (PDT) Message-ID: <1996218382.1217867926222.JavaMail.jira@brutus> Date: Mon, 4 Aug 2008 09:38:46 -0700 (PDT) From: =?utf-8?Q?Alberto_Forc=C3=A9n_=28JIRA=29?= To: core-dev@hadoop.apache.org Subject: [jira] Updated: (HADOOP-3781) Unable to access data from non hadoop application (Version mismatch in DataNode) In-Reply-To: <1498993325.1216311691768.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HADOOP-3781?page=3Dcom.atlassi= an.jira.plugin.system.issuetabpanels:all-tabpanel ] Alberto Forc=C3=A9n updated HADOOP-3781: ----------------------------------- Release Note:=20 I'm in a very same situation: I have a server (linux) eith a HDFS installed= and running correctly. I'm trying to access HDFS remotely from a java appl= ication running in eclipse (3.3 europa) under Windows (XP Prof). First of all, I declare a DistributedFileSystem object. Then, I set configu= ration parameters ({URI}, {user_name, group_list}) and then I call Distribu= tedFileSystem.initialize(uri, conf). In that moment the exception is raised= . [code] =09=09String ipStr =3D "X.X.X.X"; //ip dir =09=09String portStr =3D "N"; // port number =09=09String uriStr =3D "http://" + ipStr + ":" + portStr; =09=09Configuration conf =3D new Configuration(); =09=09conf.set("fs.default.name", uriStr); =09=09conf.set("dfs.http.address", ipStr + ":" + portStr); =09=09conf.set("hadoop.job.ugi", "username,group"); =09=09conf.set("fs.hdfs.impl", uriStr); =09=09 =09=09DistributedFileSystem dfs =3D new DistributedFileSystem(); =09=09URI uri =3D new URI(uriStr); =09=09dfs.initialize(uri, conf); [/code] [trace] Exception in thread "main" java.net.SocketTimeoutException: timed out waiti= ng for rpc response =09at org.apache.hadoop.ipc.Client.call(Client.java:559) =09at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212) =09at org.apache.hadoop.dfs.$Proxy0.getProtocolVersion(Unknown Source) =09at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:313) =09at org.apache.hadoop.dfs.DFSClient.createRPCNamenode(DFSClient.java:102) =09at org.apache.hadoop.dfs.DFSClient.(DFSClient.java:178) =09at org.apache.hadoop.dfs.DistributedFileSystem.initialize(DistributedFil= eSystem.java:68) =09at examples.HadoopAccess.main(HadoopAccess.java:154) [/trace] > Unable to access data from non hadoop application (Version mismatch in Da= taNode) > -------------------------------------------------------------------------= ------- > > Key: HADOOP-3781 > URL: https://issues.apache.org/jira/browse/HADOOP-3781 > Project: Hadoop Core > Issue Type: Bug > Components: dfs > Affects Versions: 0.17.1 > Environment: Client Eclipse, server windows/cygwin > Reporter: Thibaut > > Hi, I'm trying to access the hdfs of my hadoop cluster in a non hadoop ap= plication. Hadoop 0.17.1 is running on standart ports (The same error also = occured on earlier verisons). The code however will fail, as there is a ver= sion conflict. > This is the code I use: > FileSystem fileSystem =3D null; > String hdfsurl =3D "hdfs://localhost:50010"; > fileSystem =3D new DistributedFileSystem(); > try { > fileSystem.initialize(new URI(hdfsurl), new Confi= guration()); > } catch (Exception e) { > e.printStackTrace(); > System.out.println("init error:"); > System.exit(1); > } > which fails with the exception: > java.net.SocketTimeoutException: timed out waiting for rpc response > at org.apache.hadoop.ipc.Client.call(Client.java:559) > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212) > at org.apache.hadoop.dfs.$Proxy0.getProtocolVersion(Unknown Sourc= e) > at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:313) > at org.apache.hadoop.dfs.DFSClient.createRPCNamenode(DFSClient.ja= va:102) > at org.apache.hadoop.dfs.DFSClient.(DFSClient.java:178) > at org.apache.hadoop.dfs.DistributedFileSystem.initialize(Distrib= utedFileSystem.java:68) > at com.iterend.spider.conf.Config.getRemoteFileSystem(Config.java= :72) > at tests.RemoteFileSystemTest.main(RemoteFileSystemTest.java:22) > init error: > The haddop logfile contains the following error: > STARTUP_MSG: Starting DataNode > STARTUP_MSG: host =3D bluelu-PC/192.168.1.130 > STARTUP_MSG: args =3D [] > STARTUP_MSG: version =3D 0.17.1 > STARTUP_MSG: build =3D http://svn.apache.org/repos/asf/hadoop/core/bran= ches/branch-0.17 -r 669344; compiled by 'hadoopqa' on Thu Jun 19 01:18:25 U= TC 2008=20 > 2008-07-10 23:05:47,840 INFO org.apache.hadoop.dfs.Storage: Storage direc= tory \hadoop\tmp\hadoop-sshd_server\dfs\data is not formatted. > 2008-07-10 23:05:47,840 INFO org.apache.hadoop.dfs.Storage: Formatting ..= . > 2008-07-10 23:05:47,928 INFO org.apache.hadoop.dfs.DataNode: Registered F= SDatasetStatusMBean > 2008-07-10 23:05:47,929 INFO org.apache.hadoop.dfs.DataNode: Opened serve= r at 50010 > 2008-07-10 23:05:47,933 INFO org.apache.hadoop.dfs.DataNode: Balancing ba= ndwith is 1048576 bytes/s > 2008-07-10 23:05:48,128 INFO org.mortbay.util.Credential: Checking Resour= ce aliases > 2008-07-10 23:05:48,344 INFO org.mortbay.http.HttpServer: Version Jetty/5= .1.4 > 2008-07-10 23:05:48,346 INFO org.mortbay.util.Container: Started HttpCont= ext[/static,/static] > 2008-07-10 23:05:48,346 INFO org.mortbay.util.Container: Started HttpCont= ext[/logs,/logs] > 2008-07-10 23:05:49,047 INFO org.mortbay.util.Container: Started org.mort= bay.jetty.servlet.WebApplicationHandler@15bc6c8 > 2008-07-10 23:05:49,244 INFO org.mortbay.util.Container: Started WebAppli= cationContext[/,/] > 2008-07-10 23:05:49,247 INFO org.mortbay.http.SocketListener: Started Soc= ketListener on 0.0.0.0:50075 > 2008-07-10 23:05:49,247 INFO org.mortbay.util.Container: Started org.mort= bay.jetty.Server@47a0d4 > 2008-07-10 23:05:49,257 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: In= itializing JVM Metrics with processName=3DDataNode, sessionId=3Dnull > 2008-07-10 23:05:49,535 INFO org.apache.hadoop.dfs.DataNode: New storage = id DS-2117780943-192.168.1.130-50010-1215723949510 is assigned to data-node= 127.0.0.1:50010 > 2008-07-10 23:05:49,586 INFO org.apache.hadoop.dfs.DataNode: 127.0.0.1:50= 010In DataNode.run, data =3D FSDataset{dirpath=3D'c:\hadoop\tmp\hadoop-sshd= _server\dfs\data\current'} > 2008-07-10 23:05:49,586 INFO org.apache.hadoop.dfs.DataNode: using BLOCKR= EPORT_INTERVAL of 3600000msec Initial delay: 60000msec > 2008-07-10 23:06:04,636 INFO org.apache.hadoop.dfs.DataNode: BlockReport = of 0 blocks got processed in 11 msecs > 2008-07-10 23:19:54,512 ERROR org.apache.hadoop.dfs.DataNode: 127.0.0.1:5= 0010:DataXceiver: java.io.IOException: Version Mismatch > at org.apache.hadoop.dfs.DataNode$DataXceiver.run(DataNode.java:9= 61) > at java.lang.Thread.run(Thread.java:619) > When compiling my own jar from the 0.17-1, I see that the distributed ver= sion has the revision number compiled into version number, instead of using= the one from the source code (26738 vs 9). Skipping this check triggers an= other exception: > 2008-07-17 17:28:51,268 ERROR org.apache.hadoop.dfs.DataNode: 127.0.0.1:5= 0010:DataXceiver: java.io.IOException: Unknown opcode 112 in data stream > =09at org.apache.hadoop.dfs.DataNode$DataXceiver.run(DataNode.java:1002) > =09at java.lang.Thread.run(Thread.java:619) > What do I do different from a hadoop application accessing hdfs? --=20 This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.