hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From xu cheng <xcheng....@gmail.com>
Subject Re: how to debug hadoop
Date Wed, 15 Sep 2010 05:41:01 GMT
  thanks for replying!
  I'm not sure which specific area I'm interested in. I just wanna to
understand how the wonderful system works. and triggling break point in the
source code and watch the state of the system might be a useful way to do
that(or I might be wrong  ^_^ ).
  I've found that remote debugging with the JDWP protocal in Eclipse
IDEmight work.
  for an example, If I wanna debug  the datanode, I can set the last line of
the HADOOP_HOME/hdfs/bin/hdfs.sh

to like this:

exec "$JAVA"  -Xdebug -Xrunjdwp:transport=dt_socket,address=9999
,server=y,suspend=y  $JAVA_HEAP_MAX $HADOOP_OPTS $CLASS "$@"

and then in the eclipse IDE (of course ,with the source code of the hadoop
imported ^_^  )   click run->debug configurations
->remote java application. set the ip(or the name if the hosts file is set)
to the machine that running the datanode daemon you wanna debug and the port
which is 9999 that I set before.
then apply and debug.

ok,if you don't set any breakpoints ,then the system will work normally, and
once any breakpoints are set, the daemon will be suspended, and you can
watch anything at this point(of course, if you suspend the datanode daemon
for a long time, the cluster might treated it as a dead node  ^_^ for me)

I've wrote  more  specific  words on this(but in Chineses ^_^)anyone
interested in it ,just let me know).
and thanks for your suggestion, I will read the documents you suggested to
see an easy way

thanks again.


2010/9/15 Arun C Murthy <acm@yahoo-inc.com>

> Xu,
>  Welcome!
>  Is there a specific area you are interested in? If so we can point you to
> relevant classes etc. In fact, as you understand them I'd encourage you to
> start documenting them to help others too.
>  One way to start is to take up simpler jiras, in the beginning, and start
> fixing them:
>  https://issues.apache.org/jira/browse/MAPREDUCE
>  Also look at the 'Developer Documentation' section in the hadoop wiki:
> http://wiki.apache.org/hadoop/.
>  This link might be useful http://wiki.apache.org/hadoop/HowToContribute.
> Arun
> On Sep 14, 2010, at 8:02 PM, xu cheng wrote:
>  hello:
>>  I've learn hadoop for some time, and I'm interesting in the
>> implementation
>> of the mapreduce framework of the Hadoop project. I think reading the
>> mapreduce framework source code would be a good way to do that and
>> debugging
>> the system to see how everything works would make this progress better.
>>  however I have no experience debugging distributed system. and I have no
>> idea on how to debug the mapreduce framework source code. so could you
>> please help me?
>>  any suggestion or reference would be appreciated! ^_^ thanks

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message