hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeremy Lewi <jer...@lewi.us>
Subject Re: debug mapreduce basics
Date Thu, 28 Apr 2011 23:58:34 GMT

Check out this wiki

It shows how if your running in stand alone mode you can run a job in
debug mode so that you can then start a remote debugging session with
Eclipse. You can then step through your code.

I've found it to be quite helpful.

On Thu, 2011-04-28 at 14:59 -0400, Mike Spreitzer wrote:
> So I wrote my first org.apache.hadoop.madpreduce.Job
> (not ...mapred.Job).  Oddly enough, when the reducer is invoked, the
> "Iterable values" parameter actually iterates over just one value, not
> all the corresponding values that were output by my map tasks.  I know
> this from adding calls to System.out.println to my map and reduce
> code.  I also know that my combiner was not invoked (also from println
> debugging).  So now I am trying to determine why things are not being
> wired up correctly.  What is the right way to debug situations like
> this? 
> Thanks, 
> Mike Spreitzer
> SMTP: mspreitz@us.ibm.com, Lotus Notes: Mike Spreitzer/Watson/IBM
> Office phone: +1-914-784-6424 (IBM T/L 863-)
> AOL Instant Messaging: M1k3Sprtzr

View raw message