Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; c=nofws; d=darose.net; h=message-id:date
	:from:mime-version:to:subject:references:in-reply-to
	:content-type:content-transfer-encoding; q=dns; s=darose.net; b=
	NdfA++J4+aiB678For+LsmKyIJ33INxYs+KJyh6NQ9v8BPh3L/jBceu6HHYLhBdo
	c9hQ7bWXpvYEiVjvTm56k7DFCsjrJf8RPDICwfHjaBlaqVAygL3Ffq024zh53bct
	5sSKSn7ryPa7difiLDyKqMENdYJrxkd7MSMfwonfkm0=
Message-ID: <4DC07CF2.7050106@darose.net>
Date: Tue, 03 May 2011 18:08:50 -0400
From: David Rosenstrauch <darose@darose.net>
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
 rv:1.9.2.17) Gecko/20110429 Thunderbird/3.1.10
MIME-Version: 1.0
To: mapreduce-user@hadoop.apache.org
Subject: Re: Three Questions
References: <BANLkTikofhTSknGdV3PAuZzQhY58NHMFDQ@mail.gmail.com>
	<4DC067E8.8010101@darose.net>
 <BANLkTimpHSp7rFLNj0w3hhn_Ne=oE+V2rQ@mail.gmail.com>
In-Reply-To: <BANLkTimpHSp7rFLNj0w3hhn_Ne=oE+V2rQ@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

On 05/03/2011 05:49 PM, Geoffry Roberts wrote:
> David,
>
> Thanks for the response.
>
> Last thing first:
>
> I am using org.apache.hadoop.mapreduce.lib.output.MultipleOutputs
>
> which is differs from what your link points to
> org.apache.hadoop.mapred.lib.MultipleOutputs. Using the class you propose,
> requires me to use a number of other classes from the same package.  These
> used to be deprecated, but apparently are not any more.
>
> Question: Does my package even work?  Must I use the other?

My apologies, I sent you the wrong link.  Should be this:

http://hadoop.apache.org/mapreduce/docs/r0.21.0/api/org/apache/hadoop/mapreduce/lib/output/MultipleOutputs.html#addNamedOutput%28org.apache.hadoop.mapreduce.Job,%20java.lang.String,%20java.lang.Class,%20java.lang.Class,%20java.lang.Class%29

Point remains the same though:  you need to call 
MultipleOutputs.addNamedOutput() to configure the named output before 
you can start writing to it.

> So far as the logging goes, I didn't quite follow your response.  You say
> "You can find an individual map or reduce task's logs here:"  but there is
> no link.
>
> I am familiar with the drill down that starts by clicking NameNode
> Logs/userlogs/job*/attempt*r*/stdout.  Are you recommending something
> different?

I am.  I wasn't recommending a specific link, but rather for you to 
click on links in one of your own M/R jobs in your Hadoop GUI.  I.e.:

* go to http:<your job tracker>:50030
* click on a job
* click on (e.g.) the word "reduce" in the UI, which brings you to the 
"All Tasks" page
...
etc.

> btw,
>
> In my Reduce class, I have a System.out statement in the setup() method that
> works (i.e. I get output.), but similar statements in the reduce() method
> yield nada.

System.out.println won't output to the job log like I'm describing 
above.  (Plus you'll have no control over the logging level.)  Using the 
logging framework is much preferred.

HTH,

DR