hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Colin McCabe <cmcc...@alumni.cmu.edu>
Subject Re: max concurrent connection to HDFS name node
Date Thu, 12 Feb 2015 22:05:30 GMT
The NN can do somewhere around 30,000 - 50,000 RPCs per second
currently, depending on configuration.  In general you do not want to
have extremely high NN RPC traffic, because it will slow things down.
You might consider re-architecting your application to do more DN
traffic and less NN traffic, if possible.  Hope that helps.

best,
Colin

On Tue, Feb 10, 2015 at 4:29 PM, Demai Ni <nidmgg@gmail.com> wrote:
> hi, folks,
>
> Is there a max limit of concurrent connection to a name node? or whether
> there is a best practice?
>
> My scenario is simple. Client(java/c++) program will open a connection
> through hdfs api call, and then open a few hdfs files, maybe read a bit
> data, then close the connection. In some case, the number of clients may
> be  50,000~100,000 concurrently. Is the number of connection acceptable?
>
> Thanks.
>
> Demai

Mime
View raw message