Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of capri.himanshu@gmail.com
 designates 209.85.220.177 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAEvH-XOGXNOWDOmn4nAF6CW1zh_Ae-HKxEuzXUBtXgfxoLALPg@mail.gmail.com>
References: 
 <CAEvH-XOGXNOWDOmn4nAF6CW1zh_Ae-HKxEuzXUBtXgfxoLALPg@mail.gmail.com>
Date: Thu, 2 Jul 2015 14:48:18 +0530
Message-ID: 
 <CAEvH-XM9Gkgtq0816fQ8kGKHZ6Gytdfv4aJedSHgtAEi2tkkdQ@mail.gmail.com>
Subject: Scalability of Name Node for external clients
From: Himanshu Jindal <capri.himanshu@gmail.com>
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=001a1149013e877e1e0519e0ebd2

--001a1149013e877e1e0519e0ebd2
Content-Type: text/plain; charset=UTF-8

I have a question regarding scalability of name node. Typically the name
node handles 2 type of clients:
1. Internal clients (data nodes - part of the hadoop cluster)
2. External clients (client nodes requesting for block locations in order
to perform read/writes on data nodes)

I am not much concerned about the throughput of Internal clients, However I
am more worried about throughput of the external clients. So what is
expected throughput of operations on name-node for external clients and how
scalable it is? To be more precise, Please look at following example:

There is a typical Name Node server running a cluster of 100 data nodes.
Now assuming the Internal clients use default block reports and heartbeat
requests, I have following questions regarding scalability of the NameNode:
1. What is number of simultaneous external clients connections the Name
Node can support? (a hundred thousands?)
2. What is the number of operations (get block locations) served per second?
3. What are different ways to increase the throughput for these external
clients?

Thanks
Himanshu

--001a1149013e877e1e0519e0ebd2
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<p dir=3D"ltr">I have a question regarding scalability of name node. Typica=
lly the name node handles 2 type of clients:<br>
1. Internal clients (data nodes - part of the hadoop cluster)<br>
2. External clients (client nodes requesting for block locations in order t=
o perform read/writes on data nodes)<br></p>
<p dir=3D"ltr">I am not much concerned about the throughput of Internal cli=
ents, However I am more worried about throughput of the external clients. S=
o what is expected throughput of operations on name-node for external clien=
ts and how scalable it is? To be more precise, Please look at following exa=
mple:</p>
<p dir=3D"ltr">There is a typical Name Node server running a cluster of 100=
 data nodes. Now assuming the Internal clients use default block reports an=
d heartbeat requests, I have following questions regarding scalability of t=
he NameNode:<br>
1. What is number of simultaneous external clients connections the Name Nod=
e can support? (a hundred thousands?)<br>
2. What is the number of operations (get block locations) served per second=
?<br>
3. What are different ways to increase the throughput for these external cl=
ients?</p>
<p dir=3D"ltr">Thanks<br>
Himanshu<br>
</p>

--001a1149013e877e1e0519e0ebd2--