hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marco Nicosia <ma...@yahoo-inc.com>
Subject Re: Client Scaling
Date Fri, 13 Jul 2007 16:15:12 GMT
Hadoop is built from the ground up to be scalable. You might want to look at
the project's PoweredBy page to get a sense of the grids that have deployed
using Hadoop already:

In your case remember that the NameNode is only a single point of constraint
for metadata operations. The actual flow of bytes is more a factor of your
network design and the number of DataNodes available in the HDFS.

In general, I bet 100 clients should be no problem if you have a reasonable
number of dataNodes, especially if the client operations are not

-- Marco

On 7/13/07 08:44, "Stu Hood" <stuhood@webmail.us> wrote:

> Hello,
> How well does Hadoop scale for multiple client inputs? For instance, could a
> reasonably powerful namenode handle 100 client machines copying in 10 MB every
> 10 minutes? Assume all of the clients would be running a wrapper around the
> "copyFromLocalFile" method.
> Thanks,
> Stu Hood
> Webmail.us
> "You manage your business. We'll manage your email."®

   Marco Nicosia - Kryptonite Grid
   Systems, Tools, and Services Group

View raw message