hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From maha <m...@umail.ucsb.edu>
Subject Re: Needs a simple answer
Date Fri, 17 Dec 2010 22:20:10 GMT
Hi Ted, thanks for the hint , but I'm using only one machine now .. so the file is definitely
in there. 

So, How can I open an hdfs- file in my InputFormat class (in GetSplits) so I can pass the
contents to the mappers ?

Now, the splits are splits of files-paths from the input directory ( eg. split1: /tmp/f1,
split2:/tmp/f2 .... split4: /tmp/f4)  instead I want ----->  ( split1: content of file1
, ....).

Thank you,

Maha


On Dec 16, 2010, at 2:49 PM, Ted Dunning wrote:

> Maha,
> 
> Remember that the mapper is not running on the same machine as the main
> class.  Thus local files aren't where you think.
> 
> On Thu, Dec 16, 2010 at 1:06 PM, maha <maha@umail.ucsb.edu> wrote:
> 
>> Hi all,
>> 
>>  Why the following lines would work in the main class (WordCount) and not
>> in Mapper ? even though " myconf " is set in WordCount to point to the
>> getConf() returned object.
>> 
>>            try{
>>                   FileSystem hdfs = FileSystem.get(wc.WordCount.myconf);
>>                   hdfs.copyFromLocalFile(new Path("/Users/file"), new
>> Path("/tmp/file"));
>>                  }catch(Exception e) { System.err.print("\nError");}
>> 
>> 
>> Also, the print statement will never print on console unless it's in my
>> run function..
>> 
>>     Appreciate it :)
>> 
>>       Maha
>> 
>> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message