hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexandru Calin <alexandrucali...@gmail.com>
Subject File is not written on HDFS after running libhdfs C API
Date Thu, 05 Mar 2015 07:52:26 GMT
I am trying to run the basic libhdfs example, it compiles ok, and actually
runs ok, and executes the whole program, but I cannot see the file on the
HDFS.

It is said  here <http://hadoop.apache.org/docs/r1.2.1/libhdfs.html>, that
you have to include *the right configuration directory containing
hdfs-site.xml*

My hdfs-site.xml:

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
    <property>
      <name>dfs.namenode.name.dir</name>
      <value>file:///usr/local/hadoop/hadoop_data/hdfs/namenode</value>
    </property>
    <property>
      <name>dfs.datanode.data.dir</name>
      <value>file:///usr/local/hadoop/hadoop_store/hdfs/datanode</value>
    </property></configuration>

I generate my classpath with this:

#!/bin/bashexport CLASSPATH=/usr/local/hadoop/
declare -a subdirs=("hdfs" "tools" "common" "yarn" "mapreduce")for
subdir in "${subdirs[@]}"do
        for file in $(find /usr/local/hadoop/share/hadoop/$subdir -name *.jar)
        do
                export CLASSPATH=$CLASSPATH:$file
        donedone

and I also add export CLASSPATH=$CLASSPATH:/usr/local/hadoop/etc/hadoop ,
where my *hdfs-site.xml* reside.

MY LD_LIBRARY_PATH =
/usr/local/hadoop/lib/native:/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server
Code:

#include "hdfs.h"#include <stdio.h>#include <string.h>#include
<stdio.h>#include <stdlib.h>
int main(int argc, char **argv) {

    hdfsFS fs = hdfsConnect("default", 0);
    const char* writePath = "/tmp/testfile.txt";
    hdfsFile writeFile = hdfsOpenFile(fs, writePath, O_WRONLY|O_CREAT, 0, 0, 0);
    if(!writeFile) {
          printf("Failed to open %s for writing!\n", writePath);
          exit(-1);
    }
    printf("\nfile opened\n");
    char* buffer = "Hello, World!";
    tSize num_written_bytes = hdfsWrite(fs, writeFile, (void*)buffer,
strlen(buffer)+1);
    printf("\nWrote %d bytes\n", (int)num_written_bytes);
    if (hdfsFlush(fs, writeFile)) {
           printf("Failed to 'flush' %s\n", writePath);
          exit(-1);
    }
   hdfsCloseFile(fs, writeFile);
   hdfsDisconnect(fs);
   return 0;}

It compiles and runs without error, but I cannot see the file on HDFS.

I have Hadoop 2.6.0 on Ubuntu 14.04 64bit.

Any ideas on this ?

Mime
View raw message