hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anfernee Xu <anfernee...@gmail.com>
Subject HDFS Federation address performance issue
Date Tue, 28 Jan 2014 18:04:11 GMT
Hi,

Based on
http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/Federation.html#Key_Benefits,
the overall performance can be improved by federation, but I'm not sure
federation address my usercase, could someone elaborate it?

My usercase is I have one single NM and several DN, and I have bunch of
concurrent MR jobs which will create new files(plan files and
sub-directory) under the same parent directory, the questions are:

1) Will these concurrent writes(new file, plan files and sub-directory
under the same parent directory) run in sequential because WRITE-once
control govened by single NM?

I need this answer to estimate the necessity of moving to HDFS federation.

Thanks

-- 
--Anfernee

Mime
View raw message