hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Zhurakousky <oleg.zhurakou...@gmail.com>
Subject Re: What configuration parameters cause a Hadoop 2.x job to run on the cluster
Date Fri, 25 Apr 2014 16:52:34 GMT
Yes, if you are running MR


On Fri, Apr 25, 2014 at 12:48 PM, Steve Lewis <lordjoe2000@gmail.com> wrote:

> Thank you for your answer
>
> 1) I am using YARN
> 2) So presumably dropping  core-site.xml, yarn-site into user.dir works
> do I need mapred-site.xml as well?
>
>
>
> On Fri, Apr 25, 2014 at 9:00 AM, Oleg Zhurakousky <
> oleg.zhurakousky@gmail.com> wrote:
>
>> What version of Hadoop you are using? (YARN or no YARN)
>> To answer your question; Yes its possible and simple. All you need to to
>> is to have Hadoop JARs on the classpath with relevant configuration files
>> on the same  classpath pointing to the Hadoop cluster. Most often people
>> simply copy core-site.xml, yarn-site.xml etc from the actual cluster to the
>> application classpath and then you can run it straight from IDE.
>>
>> Not a windows user so not sure about that second part of the question.
>>
>> Cheers
>> Oleg
>>
>>
>> On Fri, Apr 25, 2014 at 11:46 AM, Steve Lewis <lordjoe2000@gmail.com>wrote:
>>
>>> Assume I have a machine on the same network as a hadoop 2 cluster but
>>> separate from it.
>>>
>>> My understanding is that by setting certain elements of the config file
>>> or local xml files to point to the cluster I can launch a job without
>>> having to log into the cluster, move my jar to hdfs and start the job from
>>> the cluster's hadoop machine.
>>>
>>> Does this work?
>>> What Parameters need I sat?
>>> Where is the jar file?
>>> What issues would I see if the machine is running Windows with cygwin
>>> installed?
>>>
>>>
>>
>
>
> --
> Steven M. Lewis PhD
> 4221 105th Ave NE
> Kirkland, WA 98033
> 206-384-1340 (cell)
> Skype lordjoe_com
>
>

Mime
View raw message