hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tsuyoshi OZAWA <ozawa.tsuyo...@gmail.com>
Subject Re: "attempt*" directories in user logs
Date Mon, 10 Dec 2012 08:18:10 GMT
Hi Oleg,

Speculative tasks can be launched as TaskAttempt in MR jobs.
And, if no reducer class is set, MR launches default Reducer
class(IdentityReducer).

Thanks,
Tsuyoshi


On Sun, Dec 9, 2012 at 11:53 PM, Oleg Zhurakousky <
oleg.zhurakousky@gmail.com> wrote:

> I studying user logs on the two node cluster that I have setup and I was
> wondering if anyone can shed some light on these "attempt*' directories
> >$ ls
> attempt_201212051224_0021_m_000000_0  attempt_201212051224_0021_m_000003_0
>  job-acls.xml
> attempt_201212051224_0021_m_000002_0  attempt_201212051224_0021_r_000000_0
>
> I mean its obvious that its talking about 3 attempts for Map task and 1
> attempt for reduce task. However my current MR job only results in some
> output written to "attempt_201212051224_0021_m_000000_0". Nothing is the
> reduce part (understandably since I don't even have a reducer, so my
> question is:
>
> 1. The two more M attempts. . . what are they?
> 2. Why was there an attempt to do a Reduce when no reducer was
> provided.implemented
> 3. Why my master node only had 1 attempt for M task but the slave had all
> that's displayed and questioned above (the 'ls' output above is from the
> slave node)
>
> Thanks
> Oleg
>




-- 
OZAWA Tsuyoshi

Mime
View raw message