hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raghu Angadi <rang...@yahoo-inc.com>
Subject Re: Storing data packets using Hadoop
Date Fri, 05 Oct 2007 17:39:07 GMT

Sure. It is possible, as long as you exclude Hadoop traffic itself and 
you are not expecting to saturate the network interfaces with combined 
traffic. Also because of the way data is written to the DFS, you need a 
large buffer between capture and the client writing to DFS, other wise 
you might drop packets. I would say the buffer should be at least 2 to 3 
times the block size (default block size is 64MB).

Raghu.

Dhn wrote:
> Hi all ,
> 
> 
> Is it possible to store the 10000 data packets comming from sockets using
> Hadoop if so please reply me .
> 
> 
> Thanks in advance
> 
> 
> Dhayalan.G


Mime
View raw message