hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tanvir Rahman <tanvir9982...@gmail.com>
Subject How to attach Intellij debugger to a running process to debug yarn mapreduce Application Master source code?
Date Thu, 09 Feb 2017 19:52:02 GMT
Hello everyone,
I am currently working on a research project where i need to understand the
yarn mapreduce Application Master code in hadoop 2.7.3. I have downloaded
hadoop-2.7.3 source code, built it, I have a local Hadoop configuration on
my machine. I can successfully run wordcount application in my hadoop setup
using intelliJ IDE.

However, I can not attach Intellij debugger to a running process to debug
Application Master code.
I can debug the wordcount application source code but I have trouble
attaching the the application master (MRAppMaster.java). I tried to follow
suggestions that I found on the web about setting HADOOP_OPTS and/or
JAVA_OPTS to something like

export JAVA_OPTS="-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005"

but I ran into problems, since the same port can only be used once. Hadoop
starts however multiple jvms, and by setting these options generally all
jvm try to bind  to that port.

Does anybody have experience on how to specifically attach to the mapreduce
application master ?

Thanks in advance

View raw message