hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Azuryy Yu <azury...@gmail.com>
Subject Re: File is not written on HDFS after running libhdfs C API
Date Thu, 05 Mar 2015 08:04:14 GMT
you need to include core-site.xml as well. and I think you can find
'/tmp/testfile.txt' on your local disk, instead of HDFS.

if so,  My guess is right.  because you don't include core-site.xml, then
your Filesystem schema is file:// by default, not hdfs://.



On Thu, Mar 5, 2015 at 3:52 PM, Alexandru Calin <alexandrucalin29@gmail.com>
wrote:

> I am trying to run the basic libhdfs example, it compiles ok, and actually
> runs ok, and executes the whole program, but I cannot see the file on the
> HDFS.
>
> It is said  here <http://hadoop.apache.org/docs/r1.2.1/libhdfs.html>,
> that you have to include *the right configuration directory containing
> hdfs-site.xml*
>
> My hdfs-site.xml:
>
> <configuration>
>     <property>
>         <name>dfs.replication</name>
>         <value>1</value>
>     </property>
>     <property>
>       <name>dfs.namenode.name.dir</name>
>       <value>file:///usr/local/hadoop/hadoop_data/hdfs/namenode</value>
>     </property>
>     <property>
>       <name>dfs.datanode.data.dir</name>
>       <value>file:///usr/local/hadoop/hadoop_store/hdfs/datanode</value>
>     </property></configuration>
>
> I generate my classpath with this:
>
> #!/bin/bashexport CLASSPATH=/usr/local/hadoop/
> declare -a subdirs=("hdfs" "tools" "common" "yarn" "mapreduce")for subdir in "${subdirs[@]}"do
>         for file in $(find /usr/local/hadoop/share/hadoop/$subdir -name *.jar)
>         do
>                 export CLASSPATH=$CLASSPATH:$file
>         donedone
>
> and I also add export CLASSPATH=$CLASSPATH:/usr/local/hadoop/etc/hadoop ,
> where my *hdfs-site.xml* reside.
>
> MY LD_LIBRARY_PATH =
> /usr/local/hadoop/lib/native:/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server
> Code:
>
> #include "hdfs.h"#include <stdio.h>#include <string.h>#include <stdio.h>#include
<stdlib.h>
> int main(int argc, char **argv) {
>
>     hdfsFS fs = hdfsConnect("default", 0);
>     const char* writePath = "/tmp/testfile.txt";
>     hdfsFile writeFile = hdfsOpenFile(fs, writePath, O_WRONLY|O_CREAT, 0, 0, 0);
>     if(!writeFile) {
>           printf("Failed to open %s for writing!\n", writePath);
>           exit(-1);
>     }
>     printf("\nfile opened\n");
>     char* buffer = "Hello, World!";
>     tSize num_written_bytes = hdfsWrite(fs, writeFile, (void*)buffer, strlen(buffer)+1);
>     printf("\nWrote %d bytes\n", (int)num_written_bytes);
>     if (hdfsFlush(fs, writeFile)) {
>            printf("Failed to 'flush' %s\n", writePath);
>           exit(-1);
>     }
>    hdfsCloseFile(fs, writeFile);
>    hdfsDisconnect(fs);
>    return 0;}
>
> It compiles and runs without error, but I cannot see the file on HDFS.
>
> I have Hadoop 2.6.0 on Ubuntu 14.04 64bit.
>
> Any ideas on this ?
>
>
>

Mime
View raw message