hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Rosenstrauch <dar...@darose.net>
Subject Re: Three Questions
Date Tue, 03 May 2011 22:08:50 GMT
On 05/03/2011 05:49 PM, Geoffry Roberts wrote:
> David,
>
> Thanks for the response.
>
> Last thing first:
>
> I am using org.apache.hadoop.mapreduce.lib.output.MultipleOutputs
>
> which is differs from what your link points to
> org.apache.hadoop.mapred.lib.MultipleOutputs. Using the class you propose,
> requires me to use a number of other classes from the same package.  These
> used to be deprecated, but apparently are not any more.
>
> Question: Does my package even work?  Must I use the other?

My apologies, I sent you the wrong link.  Should be this:

http://hadoop.apache.org/mapreduce/docs/r0.21.0/api/org/apache/hadoop/mapreduce/lib/output/MultipleOutputs.html#addNamedOutput%28org.apache.hadoop.mapreduce.Job,%20java.lang.String,%20java.lang.Class,%20java.lang.Class,%20java.lang.Class%29

Point remains the same though:  you need to call 
MultipleOutputs.addNamedOutput() to configure the named output before 
you can start writing to it.

> So far as the logging goes, I didn't quite follow your response.  You say
> "You can find an individual map or reduce task's logs here:"  but there is
> no link.
>
> I am familiar with the drill down that starts by clicking NameNode
> Logs/userlogs/job*/attempt*r*/stdout.  Are you recommending something
> different?

I am.  I wasn't recommending a specific link, but rather for you to 
click on links in one of your own M/R jobs in your Hadoop GUI.  I.e.:

* go to http:<your job tracker>:50030
* click on a job
* click on (e.g.) the word "reduce" in the UI, which brings you to the 
"All Tasks" page
...
etc.

> btw,
>
> In my Reduce class, I have a System.out statement in the setup() method that
> works (i.e. I get output.), but similar statements in the reduce() method
> yield nada.

System.out.println won't output to the job log like I'm describing 
above.  (Plus you'll have no control over the logging level.)  Using the 
logging framework is much preferred.

HTH,

DR

Mime
View raw message