hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Yang <lin.yang.ja...@gmail.com>
Subject Re: How to connect to a cluster by using eclipse
Date Thu, 05 Jul 2012 03:53:13 GMT
all right, thanks~

在 2012年7月5日星期四,Marcos Ortiz 写道:

>  Jason,
> Ramon is right.
> The best way to debug a MapReduce job is mounting a local cluster, and
> then, when you have tested enough your code, then, you can
> deploy it in a real distributed cluster.
> On 07/04/2012 10:00 PM, Jason Yang wrote:
>
> ramon,
>
>  Thank for your reply very much.
>
>  However, I was still wonder whether I could debug a MR application in
> this way.
>
>  I have read some posts talking about using NAT to redirect all the
> packets to the network card which connect to the local LAN, but it does not
> work as I tried to redirect by using iptables :(
>
> 在 2012年7月4日星期三, 写道:
>
>>  Jason,
>>
>>
>>   the easiest way to debug a MapRedupe program with eclipse is working on
>> hadoop local.
>> http://hadoop.apache.org/common/docs/r0.20.2/quickstart.html#Local In
>> this mode all the components run locally on the same VM and can be easily
>> debugged using Eclipse. Hope this will be useful.
>>
>>
>>
>> *From:* Jason Yang [mailto:lin.yang.jason@gmail.com]
>> *Sent:* miércoles, 04 de julio de 2012 11:25
>> *To:* mapreduce-user
>> *Subject:* How to connect to a cluster by using eclipse
>>
>>
>>
>> Hi, all
>>
>>
>>
>> I have a hadoop cluster with 3 nodes, the network topology is like this:
>>
>> 1. For each DataNode, its IP address is like :192.168.0.XXX;
>>
>> 2. For the NameNode, it has two network cards: one is connect with the
>> DataNodes as a local LAN with IP address 192.168.0.110, while the other one
>> is connect to the company network(which eventually connect to the Internet);
>>
>> --
>>
>> now I'm trying to debug a MapReduce program on a computer which is in the
>> company network. Since the jobtracker in this scenario is
>> 192.168.0.110:9001, I was wondering how could I connect to the cluster
>> by using eclipse?
>>
>>
>>
>> --
>>
>> YANG, Lin
>>
>>
>>
>> ------------------------------
>> Subject to local law, communications with Accenture and its affiliates
>> including telephone calls and emails (including content), may be monitored
>> by our systems for the purposes of security and the assessment of internal
>> compliance with Accenture policy.
>>
>> ______________________________________________________________________________________
>>
>> www.accenture.com
>>
>
>
> --
> YANG, Lin
>
>
> --
> **
>
> Marcos Luis Ortíz Valmaseda
> *Data Engineer && Sr. System Administrator at UCI*
> about.me/marcosortiz
> My Blog <http://marcosluis2186.posterous.com>
> @marcosluis2186 <http://twitter.com/marcosluis2186>
>  **
>
>
>
>   <http://www.uci.cu/>
>
>

-- 
YANG, Lin

Mime
View raw message