hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karim Awara <karim.aw...@kaust.edu.sa>
Subject Re: Debug Hadoop Junit Test in Eclipse
Date Mon, 16 Dec 2013 15:32:01 GMT
Useful post, however, I am not trying to  debug mapreduce programs with its
associated VMs. I want to modify HDFS source code on how it uploads files.
So I am only looking to trace fs commands through the DFS shell. I believe
this should be require less work in debugging than actually going to mapred
VMs!




--
Best Regards,
Karim Ahmed Awara


On Mon, Dec 16, 2013 at 5:57 PM, Jay Vyas <jayunit100@gmail.com> wrote:

> Excellent question.  its not trivial to debug a distributed app in
> eclipse, but it is totally doable using javaagents .   We've written it up
> here:
>
>
> http://jayunit100.blogspot.com/2013/07/deep-dive-into-hadoop-with-bigtop-and.html
>
> FYI cc Brad childs (https://github.com/childsb) at red hat has helped me
> with the tutorial, he might have some extra advice also (cc'd on this
> email), I've written up one way to do this using the bigtop VMs here.
>
>
>
>
> On Mon, Dec 16, 2013 at 8:07 AM, Karim Awara <karim.awara@kaust.edu.sa>wrote:
>
>> Hi,
>>
>> I want to trace how a file upload (-put) happens in hadoop. So Im junit
>> testing TestDFSShell.java. When I try to debug the test, It fails due to
>> test timed out exception. I believe this is because I am trying to stop one
>> thread while the rest are working. I have changed the breakpoint property
>> to suspend VM, but still same problem.
>>
>>
>> How can I trace calls made by datanode/namenode when running
>> TestDFSShell.java Junit test through eclipse?
>>
>> I am using hadoop 2.2.0
>>
>> --
>> Best Regards,
>> Karim Ahmed Awara
>>
>> ------------------------------
>> This message and its contents, including attachments are intended solely
>> for the original recipient. If you are not the intended recipient or have
>> received this message in error, please notify me immediately and delete
>> this message from your computer system. Any unauthorized use or
>> distribution is prohibited. Please consider the environment before printing
>> this email.
>
>
>
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com
>

-- 

------------------------------
This message and its contents, including attachments are intended solely 
for the original recipient. If you are not the intended recipient or have 
received this message in error, please notify me immediately and delete 
this message from your computer system. Any unauthorized use or 
distribution is prohibited. Please consider the environment before printing 
this email.

Mime
View raw message