hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "mogrob.sanit@libero.it" <mogrob.sa...@libero.it>
Subject libhdfs doesn't connect to hdfs filesystem
Date Tue, 24 Apr 2012 16:54:56 GMT
Hi all,

I wanted to write a c++ code using libhdfs API. To perform a simple test, I 
compiled and ran the sample program at http://hadoop.apache.org/common/docs/r0.
20.2/libhdfs.html. It worked ed exited correctly, but instead of creating 
/tmp/textfile.txt in hdfs it created it on the local filesystem.
That seemed to be a very strange issue, so I tried the code below:

#include "hdfs.h"
#include <stdio.h>

int main(int argc, char **argv) {
    hdfsFS hdfs = hdfsConnect("default", 0);
    int num;
    hdfsFileInfo* list = hdfsListDirectory(hdfs, "/", &num);
    for (int i=0; i<num; i++)
       printf("%s\n", list[i].mName);
    hdfsDisconnect(hdfs);
}

The output was, as I expected, the list of file in / [root directory of the 
local filesystem]. So there's a problem: hdfsConnect() chooses as default the 
local filesystem instead of hdfs' one. 

To help understanding the problem here are some more details. 

I'm working with a 64-bit machine, running debian, kernel 2.6.32-5 with hadoop-
0.20. The content of core-site.xml is: 

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
<property>
  <name>fs.default.name</name>
  <value>hdfs://hydra1:54310/</value>
  <description>The name of the default file system.  A URI whose
  scheme and authority determine the FileSystem implementation.  The
  uri's scheme determines the config property (fs.SCHEME.impl) naming
  the FileSystem implementation class.  The uri's authority is used to
  determine the host, port, etc. for a filesystem.</description>
 </property>
</configuration>

and the output of echo $CLASSPATH is

/usr/lib/hadoop-0.20/lib/ant-contrib-1.0b3.jar:/usr/lib/hadoop-0.20
/lib/aspectjrt-1.6.5.jar:/usr/lib/hadoop-0.20/lib/aspectjtools-1.6.5.jar:
/usr/lib/hadoop-0.20/lib/commons-cli-1.2.jar:/usr/lib/hadoop-0.20/lib/commons-
codec-1.4.jar:/usr/lib/hadoop-0.20/lib/commons-daemon-1.0.1.jar:/usr/lib/hadoop-
0.20/lib/commons-el-1.0.jar:/usr/lib/hadoop-0.20/lib/commons-httpclient-3.1.jar:
/usr/lib/hadoop-0.20/lib/commons-lang-2.4.jar:/usr/lib/hadoop-0.20/lib/commons-
logging-1.0.4.jar:/usr/lib/hadoop-0.20/lib/commons-logging-api-1.0.4.jar:
/usr/lib/hadoop-0.20/lib/commons-net-1.4.1.jar:/usr/lib/hadoop-0.20/lib/core-
3.1.1.jar:/usr/lib/hadoop-0.20/lib/guava-r09-jar:/usr/lib/hadoop-0.20
/lib/hadoop-fairscheduler-0.20.2-cdh3u3.jar:/usr/lib/hadoop-0.20/lib/hsqldb-
1.8.0.10.jar:/usr/lib/hadoop-0.20/lib/jackson-core-asl-1.5.2.jar:
/usr/lib/hadoop-0.20/lib/jackson-mapper-asl-1.5.2.jar:/usr/lib/hadoop-0.20
/lib/jasper-compiler-5.5.12.jar:/usr/lib/hadoop-0.20/lib/jasper-runtime-5.5.12.
jar:/usr/lib/hadoop-0.20/lib/jets3t-0.6.1.jar:/usr/lib/hadoop-0.20/lib/jetty-
6.1.26.cloudera.1.jar:/usr/lib/hadoop-0.20/lib/jetty-servlet-tester-6.1.26.
cloudera.1.jar:/usr/lib/hadoop-0.20/lib/jetty-util-6.1.26.cloudera.1.jar:
/usr/lib/hadoop-0.20/lib/jsch-0.1.42.jar:/usr/lib/hadoop-0.20/lib/junit-4.5.jar:
/usr/lib/hadoop-0.20/lib/kfs-0.2.2.jar:/usr/lib/hadoop-0.20/lib/log4j-1.2.15.
jar:/usr/lib/hadoop-0.20/lib/mockito-all-1.8.2.jar:/usr/lib/hadoop-0.20/lib/oro-
2.0.8.jar:/usr/lib/hadoop-0.20/lib/servlet-api-2.5-20081211.jar:/usr/lib/hadoop-
0.20/lib/servlet-api-2.5-6.1.14.jar:/usr/lib/hadoop-0.20/lib/slf4j-api-1.4.3.
jar:/usr/lib/hadoop-0.20/lib/slf4j-log4j12-1.4.3.jar:/usr/lib/hadoop-0.20
/lib/xmlenc-0.52.jar:/usr/lib/hadoop-0.20/hadoop-0.20.2-cdh3u3-ant.jar:
/usr/lib/hadoop-0.20/hadoop-0.20.2-cdh3u3-core.jar:/usr/lib/hadoop-0.20/hadoop-
0.20.2-cdh3u3-examples.jar:/usr/lib/hadoop-0.20/hadoop-0.20.2-cdh3u3-test.jar:
/usr/lib/hadoop-0.20/hadoop-0.20.2-cdh3u3-tools.jar:/usr/lib/hadoop-0.20/hadoop-
ant-0.20.2-cdh3u3.jar:/usr/lib/hadoop-0.20/hadoop-ant.jar:/usr/lib/hadoop-0.20
/hadoop-core-0.20.2-cdh3u3.jar:/usr/lib/hadoop-0.20/hadoop-core.jar:
/usr/lib/hadoop-0.20/hadoop-examples-0.20.2-cdh3u3.jar:/usr/lib/hadoop-0.20
/hadoop-examples.jar:/usr/lib/hadoop-0.20/hadoop-test-0.20.2-cdh3u3.jar:
/usr/lib/hadoop-0.20/hadoop-test.jar:/usr/lib/hadoop-0.20/hadoop-tools-0.20.2-
cdh3u3.jar:/usr/lib/hadoop-0.20/hadoop-tools.jar:/usr/lib/hadoop-0.20/conf/core-
site.xml:/usr/lib/hadoop-0.20/conf/hdfs-site.xml

Has anybody an idea of what could cause such kind of issue, and how could I 
fix it?

Thank you,

Mogrob

Mime
View raw message