hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Zhurakousky <oleg.zhurakou...@gmail.com>
Subject Re: What configuration parameters cause a Hadoop 2.x job to run on the cluster
Date Fri, 25 Apr 2014 16:00:03 GMT
What version of Hadoop you are using? (YARN or no YARN)
To answer your question; Yes its possible and simple. All you need to to is
to have Hadoop JARs on the classpath with relevant configuration files on
the same  classpath pointing to the Hadoop cluster. Most often people
simply copy core-site.xml, yarn-site.xml etc from the actual cluster to the
application classpath and then you can run it straight from IDE.

Not a windows user so not sure about that second part of the question.

Cheers
Oleg


On Fri, Apr 25, 2014 at 11:46 AM, Steve Lewis <lordjoe2000@gmail.com> wrote:

> Assume I have a machine on the same network as a hadoop 2 cluster but
> separate from it.
>
> My understanding is that by setting certain elements of the config file or
> local xml files to point to the cluster I can launch a job without having
> to log into the cluster, move my jar to hdfs and start the job from the
> cluster's hadoop machine.
>
> Does this work?
> What Parameters need I sat?
> Where is the jar file?
> What issues would I see if the machine is running Windows with cygwin
> installed?
>
>

Mime
View raw message