hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From hmar...@umbc.edu
Subject Re: Is it possible ....!!! COOL!
Date Thu, 10 Jun 2010 11:47:59 GMT

This is a really neat idea.... if anyone has a way to do this, could you
I'll bet this could be very interesting! Thanks...


> Hi,
>     I wanted to ask if it is possible to intercept every communication
> that
> takes place between hadoop's map reduce task i.e between JobTracker and
> TaskTracker and make it pass through my own communication library.
> So, if JobTracker and TaskTracker talk through http or rpc, i would like
> to
> intercept the call and let it pass through my communication library. If it
> is possible can anyone tell me that which set of classes i need to look at
> hadoop's distribution.
> Similarly, for the hdfs, is it possible to let all the communication that
> is
> happening between namenode and datanode to pass through my communication
> library.
> Reason for doing that is that i want all the communication to happen
> through
> a communication library that resolves every communication problem that we
> can have e.g firewalls, NAT, non routed paths, multi homing etc etc. By
> using that library all the headache of communication will be gone. So, we
> will be able to use hadoop quite easily and there will be no communication
> problems.
> Thats my master's project. So, i want to know how to start and where to
> look
> for.
> I would really appreciate a reply.
> Regards,
> Ahmad Shahzad

View raw message