hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Smith <ryan.justin.sm...@gmail.com>
Subject Re: Is it possible ....!!! COOL!
Date Thu, 10 Jun 2010 11:50:40 GMT
Sounds like it could be a SPOF.

On Thu, Jun 10, 2010 at 7:47 AM, <hmarti2@umbc.edu> wrote:

> Hey,
>
> This is a really neat idea.... if anyone has a way to do this, could you
> share?
> I'll bet this could be very interesting! Thanks...
>
> Best,
> HAL
>
>
> > Hi,
> >     I wanted to ask if it is possible to intercept every communication
> > that
> > takes place between hadoop's map reduce task i.e between JobTracker and
> > TaskTracker and make it pass through my own communication library.
> > So, if JobTracker and TaskTracker talk through http or rpc, i would like
> > to
> > intercept the call and let it pass through my communication library. If
> it
> > is possible can anyone tell me that which set of classes i need to look
> at
> > hadoop's distribution.
> >
> > Similarly, for the hdfs, is it possible to let all the communication that
> > is
> > happening between namenode and datanode to pass through my communication
> > library.
> >
> > Reason for doing that is that i want all the communication to happen
> > through
> > a communication library that resolves every communication problem that we
> > can have e.g firewalls, NAT, non routed paths, multi homing etc etc. By
> > using that library all the headache of communication will be gone. So, we
> > will be able to use hadoop quite easily and there will be no
> communication
> > problems.
> >
> > Thats my master's project. So, i want to know how to start and where to
> > look
> > for.
> >
> > I would really appreciate a reply.
> >
> > Regards,
> > Ahmad Shahzad
> >
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message