hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mapred Learn <mapred.le...@gmail.com>
Subject Upload files directly to hdfs from standard out using pipes
Date Wed, 12 Jan 2011 10:07:52 GMT
Hi,
I found out that :
https://github.com/cloudera/hue/blob/master/desktop/libs/hadoop/src/hadoop/fs/hadoopfs.py

can be used to write data directly to HDFS without writing to a local
filesystem but I am not able to understand how.

Could somebody tell me how can I do this ? Or is there some other best way to do it ?

Basiclly my use case is to pipe standard out from a program directly
to HDFS so that it does not have to go through the disk write.


Thanks in advance !


Mime
View raw message