hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rakesh Davanum <rakesh...@gmail.com>
Subject Re: Running a Job in a Local Job Runner:Windows 7 64-bit
Date Fri, 07 Mar 2014 03:14:55 GMT
Hi RR,

You don't need to have the actual Hadoop daemons running on windows
machince. Just install Cygwin and ensure that you have all the required
Hadoop jars in the class path of your program. You can test/debug directly
from the IDE itself just by saying "Run As" -> "Java Application" on the
driver class. This will run the program in the Local Job Runner mode. You
can use this to verify the basic logic of your MR code. We you start using
advanced features of MR, the behavior/output on Local Job Runner may be
different from when running the program on a real distributed cluster.


On Thu, Mar 6, 2014 at 2:36 PM, Radhe Radhe <radhe.krishna.radhe@live.com>wrote:

> Hi All,
> I'm trying to get some hands-on on the Map Reduce programming.
> I downloaded the code examples from Hadoop-The definitive guide, 3rd
> edition and build it using Maven:
> *mvn package -DskipTests -Dhadoop.distro=apache-2*
> Next I imported the maven projects into Eclipse. Using Eclipse now I can
> develop my own Map Reduce jobs.
> Bu how do I test\run the job locally using the "*Local Job Runner*"?
> The book excerpt says:
> *Now we can run this application against some local files. Hadoop comes
> with a localjob runner, a cut-down version of the MapReduce execution
> engine for running Map-Reduce jobs in a single JVM. It's designed for
> testing, and is very convenient for use inan IDE, since you can run it in a
> debugger to step through the code in your mapper andreducer.*
> Do I also need to install Hadoop locally on Windows for that?
> Thanks,
> -RR

View raw message