hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tanvir Rahman <tanvir9982...@gmail.com>
Subject Re: How to attach Intellij debugger to a running process to debug yarn mapreduce Application Master source code?
Date Thu, 09 Feb 2017 21:26:56 GMT
Thanks you Steven for your quick feedback.
Setting yarn.app.mapreduce.am.command-opts works in a cluster, but it is
not working in my local Hadoop setup.
I am trying to find out the reason.

On Thu, Feb 9, 2017 at 2:00 PM, Steven Rand <stevenjrand@gmail.com> wrote:

> Have you tried setting yarn.app.mapreduce.am.command-opts? That should
> allow you to set those Java options on only the Application Master and no
> other processes.
> On Thu, Feb 9, 2017 at 11:52 AM, Tanvir Rahman <tanvir9982000@gmail.com>
> wrote:
>> Hello everyone,
>> I am currently working on a research project where i need to understand
>> the yarn mapreduce Application Master code in hadoop 2.7.3. I have
>> downloaded hadoop-2.7.3 source code, built it, I have a local Hadoop
>> configuration on my machine. I can successfully run wordcount application
>> in my hadoop setup using intelliJ IDE.
>> However, I can not attach Intellij debugger to a running process to debug
>> Application Master code.
>> I can debug the wordcount application source code but I have trouble
>> attaching the the application master (MRAppMaster.java). I tried to follow
>> suggestions that I found on the web about setting HADOOP_OPTS and/or
>> JAVA_OPTS to something like
>> export JAVA_OPTS="-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005"
>> but I ran into problems, since the same port can only be used once.
>> Hadoop starts however multiple jvms, and by setting these options generally
>> all jvm try to bind  to that port.
>> Does anybody have experience on how to specifically attach to the
>> mapreduce application master ?
>> Thanks in advance
>> Tanvir

View raw message