hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grandl Robert <rgra...@yahoo.com>
Subject Re: non-local map task input
Date Mon, 22 Jul 2013 17:06:12 GMT
Can anyone help me with this please ?


 From: Grandl Robert <rgrandl@yahoo.com>
To: "hdfs-dev@hadoop.apache.org" <hdfs-dev@hadoop.apache.org> 
Sent: Sunday, July 21, 2013 8:41 PM
Subject: non-local map task input

Hi guys,

I am trying to figure out all the points in hdfs code where hdfs traffic is read/written.
As far as I can tell, it seems most of the traffic goes through BlockSender/BlockReceiver,
right ?

However, when a client do a copyFromLocal, or read a file, or for a map task whose input is
not local, it seems the DFSClient is invoked. I understand that with DFSClient, it gets the
dananodes locations from namenode and then directly open a socket and read/writes. Anyway,
I am not very sure where that happens. Can someone point me out where in the code I can find
the exact calls to read/write from other datanodes with DFSClient ?

Thanks in advance,
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message