Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4E3021990B for ; Tue, 19 Apr 2016 18:33:16 +0000 (UTC) Received: (qmail 10876 invoked by uid 500); 19 Apr 2016 18:33:10 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 10754 invoked by uid 500); 19 Apr 2016 18:33:10 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 10737 invoked by uid 99); 19 Apr 2016 18:33:10 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 19 Apr 2016 18:33:10 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 19EAAC1313 for ; Tue, 19 Apr 2016 18:33:10 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.18 X-Spam-Level: * X-Spam-Status: No, score=1.18 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, WEIRD_PORT=0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx2-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id CaK0CeQV2dNd for ; Tue, 19 Apr 2016 18:33:08 +0000 (UTC) Received: from mail-oi0-f45.google.com (mail-oi0-f45.google.com [209.85.218.45]) by mx2-lw-eu.apache.org (ASF Mail Server at mx2-lw-eu.apache.org) with ESMTPS id 41F625FAF4 for ; Tue, 19 Apr 2016 18:33:07 +0000 (UTC) Received: by mail-oi0-f45.google.com with SMTP id p188so20086550oih.2 for ; Tue, 19 Apr 2016 11:33:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc; bh=tYk2ni8yuHSuF3vmQO3vW2787zWxxI+Os97JmZSARRo=; b=tPa3vRszMvijiRd2WkGzj0rqJEMUO/xpi39oOjXDtNZXocnbnAJeuu5AFkX31QmcOb 5MmQtfTQfrfyQrSki9lPEfnrKvwGmEFd5P3PLunjtAm0BOVrdFRK3ED0tWkJ1eZZxxJa +0uO/h4sOcFhOWBUd7jSmIyMTuZ9fqr4xTeCEs0RfDN8QiU15hB8EApumyxxOM7DiaC/ M9a+r3Mie2trRTdt1owKqQ/eUVx7ypuLGBF3zpOddR8XzkMz+d1WOYNM4M4Yf9XtAQrx XlOJWtUETiYHKWefhcbBF7dlPetDs/NVvgZhqWZtktzsNWxkQDalW7qg65J8rpNCq3L9 r4Rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc; bh=tYk2ni8yuHSuF3vmQO3vW2787zWxxI+Os97JmZSARRo=; b=iDCWCKf1WGhTE+ORkb782BMvozBW++X7ycWE/GENetVi5Yl5+u6JABxChzCs4QiC1g SYbF2qbv0AjqLd59EXYkivcEI424+guVRF1KjdknBvH0f4U+UrhKZ7yxu5XLr/Vj8rX/ Bm7tyAQd3HdrXpeStWZfYdjFMUPvnTXtAbhvfxgOhlNlJWrRNhObTPNYG950WbUped10 wZXzXbr7c7b7AHcWIuExfz346n+NjOnWdgXIrnf4tkfhNP8iS8wAcTDi3RZk9SZlbJK5 baODFBaSiKGcoFn+eJ170kZbATx3MYPoXD4NTLxb38/xyw4pywvzxeJdl1sEKg7vWndr Z0kw== X-Gm-Message-State: AOPr4FWg1lfagFOHHK+bZl4YYHkmjjJ221ndM1fylISIsrgJJLiazkr7lIcdf0MhvQ/Y/MLs0xxrSOKidxBzQA== MIME-Version: 1.0 X-Received: by 10.202.240.68 with SMTP id o65mr1888299oih.0.1461090780256; Tue, 19 Apr 2016 11:33:00 -0700 (PDT) Received: by 10.157.3.51 with HTTP; Tue, 19 Apr 2016 11:33:00 -0700 (PDT) In-Reply-To: References: Date: Tue, 19 Apr 2016 14:33:00 -0400 Message-ID: Subject: Re: Eclipse debug HDFS server side code From: Kun Ren To: Vinayakumar B Cc: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=94eb2c08fa66f123ee0530dab43b --94eb2c08fa66f123ee0530dab43b Content-Type: text/plain; charset=UTF-8 Hi Vinay, Thanks a lot for your quick reply. My current configuration is: I run the pseudo-distributed mode in my one Linux machine. Yes, I changed the code in eclipse first, and then re-compiled the code, and then restart HDFS using the new binary, and re-debug, but still can not see any output? When you said "the console of remote process", do you mean the console of the terminal that I run the command? If I want to change some code, Could you please explain a little more about how to debug/run my new modified code? Thanks so much. On Tue, Apr 19, 2016 at 2:17 PM, Vinayakumar B wrote: > > -Vinay > > ---------- Forwarded message ---------- > From: Vinayakumar B > Date: Tue, Apr 19, 2016 at 11:47 PM > Subject: Re: Eclipse debug HDFS server side code > To: Kun Ren > > > 1. Since you are debugging remote code, you can't change the code > dynamically during debugging. If you want to change, put the compiled code > in remote process, and restart, reconnect and debug again. > 2. All the console messages will be printed on the console of remote > process, not in eclipse. > 3. Though you cannot change the code, but you can change/inspect the value > of value of an variable in 'display' window. > > You can explore more related to these. > > -Vinay > > On Tue, Apr 19, 2016 at 11:42 PM, Kun Ren wrote: > >> Hi Vinay, >> >> Thanks a lot, it works:) >> >> I have another question: >> 1. When I add the following lines into the public boolean mkdirs(String >> src, FsPermission masked, boolean createParent) in the >> NameNodeRpcServer.java, >> >> LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn"); >> System.out.println("In NameNodeRpcServer.java:mkdirs, before do >> actual work"); >> >> But I can not see any output either from my terminal or Eclipse console >> when I step-by-step debug this function, do you know why? >> >> 2. When I debug the client side code, for example FsShell, the default >> log level is INFO, I changed the level to DEBUG using the >> http://localhost:50070/logLevel, however, it still can not print out >> LOG.debug(), but I can see LOG.info(), I don't know why it can not print >> out debug information even I set the log level to DEBUG. >> >> >> Thanks a lot for your help, >> >> >> >> >> >> On Tue, Apr 19, 2016 at 12:21 PM, Vinayakumar B >> wrote: >> >>> Hi Kun Ren, >>> >>> You can follow the below steps. >>> 1. configure HADOOP_NAMENODE_OPTS="-Xdebug >>> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in >>> hadoop-env.sh >>> 2. Start Namenode >>> 3. Now Namenode will start debug port in 3988. >>> 4. Configure Remote debug application to connect to :3988 >>> in eclipse >>> 5. Now add a breakpoint in NameNodeRpcServer.java's any method you want >>> to debug. NameNodeRpcServer.java is the gateway between NameNode and IPC >>> Server Layer. >>> 6. Now you can do step-by-step execution to observe the detailed >>> executions. >>> >>> Happy Debugging ;) >>> >>> -Vinay >>> >>> On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren wrote: >>> >>>> Hi All, >>>> >>>> Currently I used eclipse to compile/debug the source code, and I >>>> configured the "remote Java application" to debug the source code in >>>> eclipse, for example, I can debug the client side code when I run the >>>> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell---> >>>> DistributedFileSystem---->DFSClient ....->send request to the server side. >>>> Then it finish debugging, but I can not see how the server side handle the >>>> request. >>>> >>>> >>>> However, I want to debug and see how the server side(HDFS) handle the >>>> request? Do you know how I can do this? Thanks so much. Also when I try to >>>> add some debug information(such as print out something), but I can not see >>>> anything printed out in eclipse, I don't know why. Thanks again. >>>> >>>> Best, >>>> Kun >>>> >>> >>> >> > > --94eb2c08fa66f123ee0530dab43b Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi Vinay,

Thanks a lot for you= r quick reply.

My current configuration is: I run the pseudo-d= istributed mode in my one Linux machine.
Yes, I changed the code i= n eclipse first, and then re-compiled the code, and then restart HDFS using= the new binary, and re-debug,=C2=A0 but still can not see any output?=C2= =A0 When you said "the console of remote process", do you mean th= e console of the terminal that I run the command?

If I want to chan= ge some code, Could you please explain a little more about how to debug/run= my new modified code? Thanks so much.



On Tue, Apr 19, 2016 at 2= :17 PM, Vinayakumar B <vinayakumarb@apache.org> wrote:=

=
-Vinay

---------- Forwarded message ----------
F= rom: Vinayakumar B <= vinayakumarb@a= pache.org>
Date: Tue, Apr 19, 2016 at 11:47 PM
Subject:= Re: Eclipse debug HDFS server side code
To: Kun Ren <ren.hdfs@gmail.com>

=
1. Since you are debugging remote code, you can't = change the code dynamically during debugging. If you want to change, put th= e compiled code in remote process, and restart, reconnect and debug again.<= div>2. All the console messages will be printed on the console of remote pr= ocess, not in eclipse.
3. Though you cannot change the code, but = you can change/inspect the value of value of an variable in 'display= 9; window.=C2=A0

You can explore more related to t= hese.

<= div dir=3D"ltr">
-Vin= ay

On Tue, Apr 19, 2016 at 11:42 PM, Kun Ren <ren.hdfs@gmail.com> wrote:
Hi Vinay,

Thanks a l= ot, it works:)

I have another question:
1. When I add= the following lines into the public boolean mkdirs(String src, FsPermissio= n masked, boolean createParent)=C2=A0 in the NameNodeRpcServer.java,
=C2=A0=C2=A0=C2=A0 =C2=A0 LOG.warn("NameNodeRpcServer.java:mkdi= rs, log level=3Dwarn");
=C2=A0=C2=A0=C2=A0 =C2=A0 System.out.printl= n("In NameNodeRpcServer.java:mkdirs, before do actual work");
=
But I can not see any output either from my terminal or Eclipse c= onsole when I step-by-step debug this function, do you know why?

2.=C2=A0 When I debug the client side code, for example FsShell, the def= ault log level is INFO, I changed the level to DEBUG using the http://localhost:50070/lo= gLevel, however, it still can not print out LOG.debug(), but I can see = LOG.info(),=C2=A0 I don't know why it can not print out debug informati= on even I set the log level to DEBUG.


Thanks a lot for yo= ur help,




<= /div>

On Tue, Apr 19, 2016 at 12:21 PM, V= inayakumar B <vinayakumarb@apache.org> wrote:
Hi Kun Ren,

Y= ou can follow the below steps.
1. configure HADOOP_NAMENODE_OPTS=3D&qu= ot;-Xdebug -Xrunjdwp:transport=3Ddt_socket,server=3Dy,suspend=3Dn,address= =3D3988" in hadoop-env.sh
2. Start Namenode
3. Now Namen= ode will start debug port in 3988.
4. Configure Remote debug appl= ication to connect to <namenode_host>:3988 in eclipse
5. No= w add a breakpoint in NameNodeRpcServer.java's any method you want to d= ebug. NameNodeRpcServer.java is the gateway between NameNode and IPC Server= Layer.
6. Now you can do step-by-step execution to observe the d= etailed executions.

Happy Debugging ;)

-Vinay

On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <= ren.hdfs@gmail.com> wrote:
Hi All,

Currently I used eclipse to=20 compile/debug the source code, and I configured the "remote Java=20 application" to debug the source code in eclipse, for example, I can= =20 debug the=C2=A0 client side code when I run the command "./bin/hdfs df= s=20 -mkdir test", it goes through "FsShell--->=20 DistributedFileSystem---->DFSClient ....->send request to the server side. Then it finish debugging, but I can not see how the server side handle the request.


However, I want to debug and=20 see how the server side(HDFS) handle the request? Do you know how I can=20 do this?=C2=A0 Thanks so much. Also when I try to add some debug=20 information(such as print out something), but I can not see anything=20 printed out in eclipse, I don't know why. Thanks again.

Be= st,
Kun





--94eb2c08fa66f123ee0530dab43b--