hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Siddu <siddu.s...@gmail.com>
Subject Re: common reasons a map task would fail on a distributed cluster but not locally?
Date Sun, 15 Nov 2009 13:18:15 GMT
On Sun, Nov 15, 2009 at 1:03 AM, Mike Kendall <mkendall@justin.tv> wrote:

> for some reason i never tried lowering my number of map and reduce tasks
> until now.  looks like i need to reconfigure my cluster since it runs fine
> with only 3 map tasks and 3 reduce tasks.
>
> :X
>
> On Sat, Nov 14, 2009 at 11:22 AM, Mike Kendall <mkendall@justin.tv> wrote:
>
> > so if i run my task as:
> >
> > cat input | ./map.py | ./sum.py > output
> >
> > it works just fine.  however, running it on my cluster as:
> >
> > hadoop jar /usr/local/hadoop/contrib/streaming/hadoop-*-streaming.jar
> -file
> > map.py -mapper map.py -file cat.py -reducer cat.py -input input -output
> > output
> >
> > it fails.  i'm really confused as to why this script would fail while my
>

If fail is followed by any error . please paste it here !


> > others that were written with the same methodology would work.
> >
> > is there a "common reasons map tasks fail" list somewhere?  any ideas?
> >
>




-- 
Regards,
~Sid~
I have never met a man so ignorant that i couldn't learn something from him

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message