hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Krahn <danielkr...@gmail.com>
Subject Debugging Hadoop Environment
Date Sat, 06 Jul 2013 02:25:26 GMT

I would like to ask a few questions on how to debug Hadoop.
First I'll explain what I'm trying to do.

I'm doing my graduation thesis.
Are group is starting an implementation on a voluntary environment. For
this it is very important to have all the results signed so that we can
guarantee that the information has not been altered by someone in the
middle. But for this we could just cipher the channel, but we need to be
able to verify that the user is giving the correct answer. If he does not,
we have to be able to blame the right node.
So for this, I'm signing the results of each phase. I'm starting by the
intermediate files between the MAP and REDUCE.
But I'm having trouble in debugging the part that implements this. I'm
using YARN. So I was able to debug remotely each node. I also was able to
debug the NodeManager inside Eclipse IDE from source code. But I was not
able to run the WordCount application with the NodeManager being debugged
from source inside Eclipse IDE. I gives me an error that it doesn't find
MRAppMaster. So I went to find it and wasn't able to debug the MRAppMaster,
for it doesn't find the AM_CONTAINER_ID_ENV in the environment.

What could I be doing wrong?

Note: I got the project from GIT and it is version 2.0.4

Daniel Krahn

View raw message