hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 杨浩 <yangha...@gmail.com>
Subject Re: Can we run mapreduce job from eclipse IDE on fully distributed mode hadoop cluster?
Date Sun, 12 Apr 2015 02:27:44 GMT
I think you can have a try at http://hdt.incubator.apache.org/

2015-04-12 2:04 GMT+08:00 Answer Agrawal <yrsna.tset01@gmail.com>:

> Thanks Jonathan
>
> I have installed and configured my own hadoop cluster with one master node
> and 7 slave nodes. Now I just want to make sure that job running through
> Eclipse is internally same as running through jar file. Also job history
> server and web ui 8088 shows only list of those jobs which are submitted
> through jar using terminal.
>
>
>
>
> On Sat, Apr 11, 2015 at 12:25 PM, Jonathan Aquilina <
> jaquilina@eagleeyet.net> wrote:
>
>>  I could be wrong here, but the way I understand things I do not think
>> that is even possible to run the JAR file from your PC. There are two
>> things that you need to realize.
>>
>> 1) How is the JAR file going to connect to the cluster
>>
>> 2) How is the JAR file going to be distributed to the cluster.
>>
>> Again I could be wrong here in my response, so anyone else on the list
>> feel free to correct me. I am still a novice to Hadoop and have only worked
>> with it on amazon EMR.
>>
>> ---
>> Regards,
>> Jonathan Aquilina
>> Founder Eagle Eye T
>>
>>  On 2015-04-11 08:23, Answer Agrawal wrote:
>>
>>  A mapreduce job can be run as jar file from terminal or directly from
>> eclipse IDE. When a job run as jar file from terminal it uses multiple jvm
>> and all resources of cluster. Does the same thing happen when we run from
>> IDE. I have run a job on both and it takes less time on IDE than jar file
>> on terminal.
>>
>> Thanks
>>
>>
>>
>

Mime
View raw message