hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aishwarya Venkataraman <avenk...@cs.ucsd.edu>
Subject Re: Viewing hadoop mapper output
Date Thu, 06 Oct 2011 20:30:31 GMT
I ran the following (I am using IdentityReducer) :

./hadoop jar ../contrib/streaming/hadoop-0.20.2-streaming.jar -file
~/mapper.sh -mapper ~/mapper.sh -input ../foo.txt -output output

When I do
./hadoop dfs -cat output/* I do not see any output on screen. Is this how I
view the output of mapper ?

Thanks,
AIshwarya

On Thu, Oct 6, 2011 at 12:37 PM, Robert Evans <evans@yahoo-inc.com> wrote:

> A streaming jobs stderr is logged for the task, but its stdout is what is
> sent to the reducer.  The simplest way to get it is to turn off the
> reducers, and then look at the output in HDFS.
>
> --Bobby Evans
>
> On 10/6/11 1:16 PM, "Aishwarya Venkataraman" <avenkata@cs.ucsd.edu> wrote:
>
> Hello,
>
> I want to view the mapper output for a given hadoop streaming jobs (that
> runs a shell script). However I am not able to find this in any log files.
> Where should I look for this ?
>
> Thanks,
> Aishwarya
>
>


-- 
Thanks,
Aishwarya Venkataraman
avenkata@cs.ucsd.edu
Graduate Student | Department of Computer Science
University of California, San Diego

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message