hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Bockelman <bbock...@cse.unl.edu>
Subject Re: Hdfs over RDMA
Date Tue, 22 Feb 2011 13:34:17 GMT

On Feb 18, 2011, at 12:32 PM, Rajat Sharma wrote:

> Hi Guys
> 
> I am trying to work on HDFS to improve its performance by adding RDMA
> functionality to its code. But i cannot find any sort of documentation or
> help regarding this topic except some information about Socket Direct
> Protocol or Allocated buffers. since RDMA is the premier protocol which is
> highly preferred for Distributed File Systems , why is it not preferred in
> case of HDFS or what are the factors which makes it tough. Any suggestions
> or papers or ideas regarding the ways we can get some way to work on it. I
> am using OFED Distribution , RDMA for this.
> 
> Thanks

Hi,

It's an overused analogy, but HDFS is a freight train (high throughput) while the file systems
you are thinking of are more like race cars (high performance).

While HDFS provides decent latency, it's not optimized for latency.  RDMA would provide little
benefit for the type of hardware it is designed for; it would introduce a plethora of headaches.

Brian
Mime
View raw message