hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Zhang <zjf...@gmail.com>
Subject Re: How to debug hadoop(or YARN) locally?
Date Tue, 22 Dec 2015 00:55:01 GMT
+1 for Chris, remote debug will help you.

On Tue, Dec 22, 2015 at 1:54 AM, Chris Nauroth <cnauroth@hortonworks.com>
wrote:

> If you're running the Hadoop daemons in pseudo-distributed mode (all the
> daemons running as separate processes, but on a single dev host), then
> another option is to launch the daemon's JVM with the JDWP arguments and
> attach a "remote" debugger.  This can be either the jdb CLI debugger that
> ships with the JDK or a fancier IDE like Eclipse or IntelliJ.
>
> Each daemon's JVM arguments are controlled with an environment variable
> suffixed with "_OPTS" defined in files named *-env.sh.  For example, in
> hadoop-env.sh, you could set something like this to enable remote
> debugging for the NameNode process:
>
> export
> HADOOP_NAMENODE_OPTS="-agentlib:jdwp=transport=dt_socket,server=y,address=8
> 000,suspend=n $HADOOP_NAMENODE_OPTS"
>
>
> Then, you can run "jdb -attach localhost:8000" to attach the debugger, or
> do the equivalent in your IDE of choice.
>
> --Chris Nauroth
>
>
>
>
> On 12/21/15, 7:25 AM, "Daniel Templeton" <daniel@cloudera.com> wrote:
>
> >Your best bet is to find a test that includes all the bits you want and
> >execute that test in debug mode.  (You can also change an existing test
> >to include what you want, but in most cases it is easier to start with
> >an existing test than to start from scratch.)
> >
> >Daniel
> >
> >On 12/20/15 6:01 PM, Allen Zhang wrote:
> >> Hi all,
> >>
> >> I am reading hadoop-2.6.0 source code, mainly focusing on hadoop yarn.
> >> However i have some problems in reading or debugging the source
> >>code,can I debug it locally(I mean in my laptop locally with this source
> >>code I've downloaded, not remotely debug),
> >> because I need to track it execution flow stey by stey, and then I want
> >>to add a new feature or enhancement.
> >>
> >>
> >> So can anyone give some good suggestions or share your method or any
> >>wiki page?  Really appreciate!!
> >>
> >>
> >> Thanks,
> >> Allen
> >
> >
>
>


-- 
Best Regards

Jeff Zhang

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message