hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arpit Agarwal <aagar...@hortonworks.com>
Subject Re: Hadoop 2.2.0 YARN isolation and Windows
Date Thu, 09 Jan 2014 22:15:01 GMT
Hi Kevin,

*> 2. According to
http://www.i-programmer.info/news/197-data-mining/6518-hadoop-2-introduces-yarn.html
<http://www.i-programmer.info/news/197-data-mining/6518-hadoop-2-introduces-yarn.html>,
Hadoop 2.2.0 supports Microsoft Windows. How do/Can you configure YARN for
secure container isolation in Windows? It seems that the ContainerExecutor
and DefaultContainerExecutor can detect and run on Windows, but the secure
LinuxContainerExecutor are for *nix systems, so is there anything in place
for maximum security like LCE is?*

Cluster security is not supported on Windows at the moment.


*> 1. Does YARN need to run on the same machines that are hosting the HDFS
services or can HDFS be remote of a YARN cluster? If this done by placing
the remote HDFS cluster's configuration files (core-site.xml and
hdfs-site.xml) on the YARN cluster's machines?*

*> 3. If 1 is yes, then is it possible to have a cluster mixed with both
Linux and Windows machines running YARN and working together?*
It should work in theory if you get the configuration right - I have not
tried it out so I am not sure. YARN containers and HDFS datanodes should be
collocated for good performance. The MapReduce compute model especially
depends on access to fast local storage.



On Thu, Jan 9, 2014 at 1:19 PM, Kevin <kevin.macksamie@gmail.com> wrote:

> Hi,
>
> Three questions about the new Hadoop release regarding YARN:
>
> 1. Does YARN need to run on the same machines that are hosting the HDFS
> services or can HDFS be remote of a YARN cluster? If this done by placing
> the remote HDFS cluster's configuration files (core-site.xml and
> hdfs-site.xml) on the YARN cluster's machines?
>
> 2. According to
> http://www.i-programmer.info/news/197-data-mining/6518-hadoop-2-introduces-yarn.html,
> Hadoop 2.2.0 supports Microsoft Windows. How do/Can you configure YARN for
> secure container isolation in Windows? It seems that the ContainerExecutor
> and DefaultContainerExecutor can detect and run on Windows, but the secure
> LinuxContainerExecutor are for *nix systems, so is there anything in place
> for maximum security like LCE is?
>
> 3. If 1 is yes, then is it possible to have a cluster mixed with both
> Linux and Windows machines running YARN and working together?
>
> Thanks,
> Kevin
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
View raw message