hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hemanth Yamijala <yhema...@thoughtworks.com>
Subject Re: map reduce and sync
Date Sat, 23 Feb 2013 07:37:01 GMT
Could you please clarify, are you opening the file in your mapper code and
reading from there ?

Thanks
Hemanth

On Friday, February 22, 2013, Lucas Bernardi wrote:

> Hello there, I'm trying to use hadoop map reduce to process an open file. The
> writing process, writes a line to the file and syncs the file to readers.
> (org.apache.hadoop.fs.FSDataOutputStream.sync()).
>
> If I try to read the file from another process, it works fine, at least
> using
> org.apache.hadoop.fs.FSDataInputStream.
>
> hadoop -fs -tail also works just fine
>
> But it looks like map reduce doesn't read any data. I tried using the word
> count example, same thing, it is like if the file were empty for the map
> reduce framework.
>
> I'm using hadoop 1.0.3. and pig 0.10.0
>
> I need some help around this.
>
> Thanks!
>
> Lucas
>

Mime
View raw message