hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karim Awara <karim.aw...@kaust.edu.sa>
Subject Re: Hadoop 2.x with Eclipse
Date Mon, 04 Nov 2013 04:22:47 GMT
Hi guys,

Can I just install the HDFS project and debug it? (assuming I am running a
<put> command through the command line). If so, which project should I
download (hadoop project that has hdfs)?

--
Best Regards,
Karim Ahmed Awara


On Sun, Oct 6, 2013 at 5:40 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> Karim:
> If you want to debug unit tests, using Eclipse is a viable approach.
> Here is what I did the past week debugging certain part of hadoop
> (JobSubmitter in particular) through an HBase unit test.
>
> Run 'mvn install -DskipTests' to install hadoop locally
> Open the class you want to debug and place breakpoint at proper location
> Open unit test which depends on the class above and select Debug As ->
> JUnit Test
> When breakpoint hits, associate the sources.jar file in local maven repo
> with the class. In my case, the sources jar file is located
> under ~/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-core/2.1.2-SNAPSHOT
>
> You should be able to step through hadoop code as usual at this point.
>
> Cheers
>
>
> On Sun, Oct 6, 2013 at 6:14 AM, John Lilley <john.lilley@redpoint.net>wrote:
>
>>  Karim,
>>
>>
>>
>> I am not an experienced Hadoop programmer, but what I found was that
>> building and debugging Hadoop under Eclipse was very difficult, and I was
>> never to make it work correctly.  I suggest using the well documented
>> command-line Maven build, installing Hadoop from that build, and running it
>> normally.  Once you have that working, run your namemode or datanode daemon
>> so as to wait for a remote debugger attach before starting.  You should
>> also get comfortable with log4j, the logging framework used by Hadoop, as
>> those log files are often your best friend when trying to debug a
>> collection of services.
>>
>>
>>
>> john
>>
>>
>>
>> *From:* Karim Awara [mailto:karim.awara@kaust.edu.sa]
>> *Sent:* Sunday, October 06, 2013 5:41 AM
>> *To:* user
>> *Subject:* Hadoop 2.x with Eclipse
>>
>>
>>
>> Hi,
>>
>> I followed the instructions on how to import hadoop files to Eclipse (I
>> am using hadoop 2.1 beta).
>>
>> Currently on my machine, I have hadoop 2.1 installed.. and its source
>> code is imported on Eclipse. What I can't grasp is   how to proceed from
>> there?
>>
>> I want to modify HDFS code (blockplacement strategy).. Now building hdfs
>> project via generates errors to me (unresolved types in hadoop common).
>> and if i built successfully, how to test my modified code?
>>
>>
>>   --
>> Best Regards,
>> Karim Ahmed Awara
>>
>>
>>  ------------------------------
>>
>> This message and its contents, including attachments are intended solely
>> for the original recipient. If you are not the intended recipient or have
>> received this message in error, please notify me immediately and delete
>> this message from your computer system. Any unauthorized use or
>> distribution is prohibited. Please consider the environment before printing
>> this email.
>>
>
>

-- 

------------------------------
This message and its contents, including attachments are intended solely 
for the original recipient. If you are not the intended recipient or have 
received this message in error, please notify me immediately and delete 
this message from your computer system. Any unauthorized use or 
distribution is prohibited. Please consider the environment before printing 
this email.

Mime
View raw message