hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From xu cheng <xcheng....@gmail.com>
Subject Re: how to debug hadoop
Date Wed, 15 Sep 2010 05:54:00 GMT
(of course, if you suspend the datanode daemon for a long time, the cluster
might treated it as a dead node  ^_^ for me)

maybe there is some options in the config files that can make the time
longer?any suggestion?

2010/9/15 xu cheng <xcheng.222@gmail.com>

> hello:
>   thanks for replying!
>   I'm not sure which specific area I'm interested in. I just wanna to
> understand how the wonderful system works. and triggling break point in the
> source code and watch the state of the system might be a useful way to do
> that(or I might be wrong  ^_^ ).
>   I've found that remote debugging with the JDWP protocal in Eclipse IDEmight work.
>   for an example, If I wanna debug  the datanode, I can set the last line
> of the HADOOP_HOME/hdfs/bin/hdfs.sh
> exec "$JAVA"  $JAVA_HEAP_MAX $HADOOP_OPTS $CLASS "$@"
>
> to like this:
>
> exec "$JAVA"  -Xdebug -Xrunjdwp:transport=dt_socket,address=9999
> ,server=y,suspend=y  $JAVA_HEAP_MAX $HADOOP_OPTS $CLASS "$@"
>
> and then in the eclipse IDE (of course ,with the source code of the hadoop
> imported ^_^  )   click run->debug configurations
> ->remote java application. set the ip(or the name if the hosts file is
> set) to the machine that running the datanode daemon you wanna debug and the
> port which is 9999 that I set before.
> then apply and debug.
>
> ok,if you don't set any breakpoints ,then the system will work normally,and once any
breakpoints are set, the daemon will be
> suspended, and you can watch anything at this point(of course, if you
> suspend the datanode daemon for a long time, the cluster might treated it as
> a dead node  ^_^ for me)
>
> I've wrote  more  specific  words on this(but in Chineses ^_^)anyone
> interested in it ,just let me know).
> and thanks for your suggestion, I will read the documents you suggested to
> see an easy way
>
> thanks again.
>
> xu
>
> 2010/9/15 Arun C Murthy <acm@yahoo-inc.com>
>
> Xu,
>>
>>  Welcome!
>>
>>  Is there a specific area you are interested in? If so we can point you to
>> relevant classes etc. In fact, as you understand them I'd encourage you to
>> start documenting them to help others too.
>>
>>  One way to start is to take up simpler jiras, in the beginning, and start
>> fixing them:
>>  https://issues.apache.org/jira/browse/MAPREDUCE
>>
>>  Also look at the 'Developer Documentation' section in the hadoop wiki:
>> http://wiki.apache.org/hadoop/.
>>  This link might be useful http://wiki.apache.org/hadoop/HowToContribute.
>>
>> Arun
>>
>>
>> On Sep 14, 2010, at 8:02 PM, xu cheng wrote:
>>
>>  hello:
>>>  I've learn hadoop for some time, and I'm interesting in the
>>> implementation
>>> of the mapreduce framework of the Hadoop project. I think reading the
>>> mapreduce framework source code would be a good way to do that and
>>> debugging
>>> the system to see how everything works would make this progress better.
>>>  however I have no experience debugging distributed system. and I have no
>>> idea on how to debug the mapreduce framework source code. so could you
>>> please help me?
>>>  any suggestion or reference would be appreciated! ^_^ thanks
>>>
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message