hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marco Nicosia <ma...@yahoo-inc.com>
Subject Re: Client Scaling
Date Fri, 13 Jul 2007 16:15:12 GMT
Hadoop is built from the ground up to be scalable. You might want to look at
the project's PoweredBy page to get a sense of the grids that have deployed
using Hadoop already:
    http://wiki.apache.org/lucene-hadoop/PoweredBy

In your case remember that the NameNode is only a single point of constraint
for metadata operations. The actual flow of bytes is more a factor of your
network design and the number of DataNodes available in the HDFS.

In general, I bet 100 clients should be no problem if you have a reasonable
number of dataNodes, especially if the client operations are not
simultaneous.

-- Marco

On 7/13/07 08:44, "Stu Hood" <stuhood@webmail.us> wrote:

> 
> 
> Hello,
> 
> How well does Hadoop scale for multiple client inputs? For instance, could a
> reasonably powerful namenode handle 100 client machines copying in 10 MB every
> 10 minutes? Assume all of the clients would be running a wrapper around the
> "copyFromLocalFile" method.
> 
> Thanks,
> 
> Stu Hood
> Webmail.us
> "You manage your business. We'll manage your email."®

-- 
   Marco Nicosia - Kryptonite Grid
   Systems, Tools, and Services Group



Mime
View raw message