hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Yarn and HDFS federation
Date Wed, 18 Sep 2013 09:38:23 GMT
Using Federated Namespaces is optional.

YARN's services use HDFS minimally on their own. It is your MR job or
your YARN application that really uses the filesystem. So you can have
YARN host itself on any namespace, and it wouldn't matter to your
actual jobs/apps, which will submit with their own FS configured.

(There may be some caveats that apply with use of JHS and where the MR
staging dir goes, but it can be fixed via consistent configs)

On Tue, Sep 17, 2013 at 8:33 PM, Manickam P <manickam.p@outlook.com> wrote:
> Hello Experts,
> I know that hdfs federation helps adding multiple name nodes. In this case
> how this Yarn works?
> Where will be the Resource Manger if i have multiple name nodes? (Normally
> in Master Node I guess)
> Can we execute yarn applications in hadoop cluster without having this hdfs
> federation?
> Please clarify my doubt.
> Thanks,
> Manickam p

Harsh J

View raw message