hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jagaran das <jagaran_...@yahoo.co.in>
Subject Namenode Scalability
Date Wed, 10 Aug 2011 07:58:16 GMT
In my current project we  are planning to streams of data to Namenode (20 Node Cluster).
Data Volume would be around 1 PB per day.
But there are application which can publish data at 1GBPS.

Few queries:

1. Can a single Namenode handle such high speed writes? Or it becomes unresponsive when GC
cycle kicks in.
2. Can we have multiple federated Name nodes  sharing the same slaves and then we can
distribute the writes accordingly.
3. Can multiple region servers of HBase help us ??

Please suggest how we can design the streaming part to handle such scale of data. 

Jagaran Das 
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message