hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stuti Awasthi <stutiawas...@hcl.com>
Subject RE: Basic question on HDFS - MR
Date Tue, 18 Oct 2011 11:26:41 GMT
Thanks Bejoy, Uma
This clears my doubt .

From: Bejoy KS [mailto:bejoy.hadoop@gmail.com]
Sent: Tuesday, October 18, 2011 4:47 PM
To: hdfs-user@hadoop.apache.org
Subject: Re: Basic question on HDFS - MR

Hi Stuti
        You don't need anything manually to do the distribution of your jar across Task Trackers(DN).
You place your jar in some dir in LFS specify your jar path in the hadoop jar command then
hadoop internally copies the jar to all the required task trackers. Also you can place the
jar in any location job tracker would distribute it across TT.

Hope it helps

On Tue, Oct 18, 2011 at 4:33 PM, Stuti Awasthi <stutiawasthi@hcl.com<mailto:stutiawasthi@hcl.com>>
I have a very basic question on MR jobs. Suppose I have a cluster 3 nodes out of which 1 NN
and 3 DN.
I wanted to run an MR job so I placed the jar on 1 of the cluster machine and executed, it
runs fine.
Now my question is do I need to copy my MR job to every DN for distributed processing?
Placing MR jar on any of the machine does not make any difference , MR job will run in distributed



The contents of this e-mail and any attachment(s) are confidential and intended for the named
recipient(s) only.
It shall not attach any liability on the originator or HCL or its affiliates. Any views or
opinions presented in
this email are solely those of the author and may not necessarily reflect the opinions of
HCL or its affiliates.
Any form of reproduction, dissemination, copying, disclosure, modification, distribution and
/ or publication of
this message without the prior written consent of the author of this e-mail is strictly prohibited.
If you have
received this email in error please delete it and notify the sender immediately. Before opening
any mail and
attachments please check them for viruses and defect.


View raw message