Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of aagarwal@hortonworks.com
 designates 209.85.214.179 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAJo9sUa1rnt=6FRxObF9hdS7f4u+MqTfqVguHmpQ=0ANHyQx3w@mail.gmail.com>
References: 
 <CAJo9sUa1rnt=6FRxObF9hdS7f4u+MqTfqVguHmpQ=0ANHyQx3w@mail.gmail.com>
Date: Thu, 9 Jan 2014 14:15:01 -0800
Message-ID: 
 <CAFiYk=qrrN=7yOn4jnGGngUQC9NyO4eYa1e+Z1_cRGZtda_ooQ@mail.gmail.com>
Subject: Re: Hadoop 2.2.0 YARN isolation and Windows
From: Arpit Agarwal <aagarwal@hortonworks.com>
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=047d7b5d340cd86fd404ef90f041

--047d7b5d340cd86fd404ef90f041
Content-Type: text/plain; charset=US-ASCII

Hi Kevin,

*> 2. According to
http://www.i-programmer.info/news/197-data-mining/6518-hadoop-2-introduces-yarn.html
<http://www.i-programmer.info/news/197-data-mining/6518-hadoop-2-introduces-yarn.html>,
Hadoop 2.2.0 supports Microsoft Windows. How do/Can you configure YARN for
secure container isolation in Windows? It seems that the ContainerExecutor
and DefaultContainerExecutor can detect and run on Windows, but the secure
LinuxContainerExecutor are for *nix systems, so is there anything in place
for maximum security like LCE is?*

Cluster security is not supported on Windows at the moment.


*> 1. Does YARN need to run on the same machines that are hosting the HDFS
services or can HDFS be remote of a YARN cluster? If this done by placing
the remote HDFS cluster's configuration files (core-site.xml and
hdfs-site.xml) on the YARN cluster's machines?*

*> 3. If 1 is yes, then is it possible to have a cluster mixed with both
Linux and Windows machines running YARN and working together?*
It should work in theory if you get the configuration right - I have not
tried it out so I am not sure. YARN containers and HDFS datanodes should be
collocated for good performance. The MapReduce compute model especially
depends on access to fast local storage.


On Thu, Jan 9, 2014 at 1:19 PM, Kevin <kevin.macksamie@gmail.com> wrote:

> Hi,
>
> Three questions about the new Hadoop release regarding YARN:
>
> 1. Does YARN need to run on the same machines that are hosting the HDFS
> services or can HDFS be remote of a YARN cluster? If this done by placing
> the remote HDFS cluster's configuration files (core-site.xml and
> hdfs-site.xml) on the YARN cluster's machines?
>
> 2. According to
> http://www.i-programmer.info/news/197-data-mining/6518-hadoop-2-introduces-yarn.html,
> Hadoop 2.2.0 supports Microsoft Windows. How do/Can you configure YARN for
> secure container isolation in Windows? It seems that the ContainerExecutor
> and DefaultContainerExecutor can detect and run on Windows, but the secure
> LinuxContainerExecutor are for *nix systems, so is there anything in place
> for maximum security like LCE is?
>
> 3. If 1 is yes, then is it possible to have a cluster mixed with both
> Linux and Windows machines running YARN and working together?
>
> Thanks,
> Kevin
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

--047d7b5d340cd86fd404ef90f041
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><div><div>Hi Kevin,<br><br></div><i>&gt; 2. According=
 to <a href=3D"http://www.i-programmer.info/news/197-data-mining/6518-hadoo=
p-2-introduces-yarn.html" target=3D"_blank">http://www.i-programmer.info/ne=
ws/197-data-mining/6518-hadoop-2-introduces-yarn.html</a>,
 Hadoop 2.2.0 supports Microsoft Windows. How do/Can you configure YARN=20
for secure container isolation in Windows? It seems that the=20
ContainerExecutor and DefaultContainerExecutor can detect and run on=20
Windows, but the secure LinuxContainerExecutor are for *nix systems, so=20
is there anything in place for maximum security like LCE is?</i><br></div><=
br></div>Cluster security is not supported on Windows at the moment.<br><br=
><i>&gt; 1. Does YARN need to run on the same machines that are hosting the=
 HDFS=20
services or can HDFS be remote of a YARN cluster? If this done by=20
placing the remote HDFS cluster&#39;s configuration files (core-site.xml an=
d
 hdfs-site.xml) on the YARN cluster&#39;s machines?<br></i><div><i>&gt; 3. =
If 1 is yes, then is it possible to have a cluster mixed with both=20
Linux and Windows machines running YARN and working together?<br></i><br></=
div><div>It should work in theory if you get the configuration right - I ha=
ve not tried it out so I am not sure. YARN containers and HDFS datanodes sh=
ould be collocated for good performance. The MapReduce compute model especi=
ally depends on access to fast local storage.<br>
<br></div></div><div class=3D"gmail_extra"><br><br><div class=3D"gmail_quot=
e">On Thu, Jan 9, 2014 at 1:19 PM, Kevin <span dir=3D"ltr">&lt;<a href=3D"m=
ailto:kevin.macksamie@gmail.com" target=3D"_blank">kevin.macksamie@gmail.co=
m</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div>Hi,<br></div><div><br>=
</div><div>Three questions about the new Hadoop release regarding YARN:<br>=
<br>
</div><div>1. Does YARN need to run on the same machines that are hosting t=
he HDFS services or can HDFS be remote of a YARN cluster? If this done by p=
lacing the remote HDFS cluster&#39;s configuration files (core-site.xml and=
 hdfs-site.xml) on the YARN cluster&#39;s machines?<br>

<br>2. According to <a href=3D"http://www.i-programmer.info/news/197-data-m=
ining/6518-hadoop-2-introduces-yarn.html" target=3D"_blank">http://www.i-pr=
ogrammer.info/news/197-data-mining/6518-hadoop-2-introduces-yarn.html</a>, =
Hadoop 2.2.0 supports Microsoft Windows. How do/Can you configure YARN for =
secure container isolation in Windows? It seems that the ContainerExecutor =
and DefaultContainerExecutor can detect and run on Windows, but the secure =
LinuxContainerExecutor are for *nix systems, so is there anything in place =
for maximum security like LCE is?<br>

<br></div><div>3. If 1 is yes, then is it possible to have a cluster mixed =
with both Linux and Windows machines running YARN and working together?<br>=
<br></div><div>Thanks,<br></div><div>Kevin<br></div></div>
</blockquote></div><br></div>

<br>
<span style=3D"color:rgb(128,128,128);font-family:Arial,sans-serif;font-siz=
e:10px">CONFIDENTIALITY NOTICE</span><br style=3D"color:rgb(128,128,128);fo=
nt-family:Arial,sans-serif;font-size:10px"><span style=3D"color:rgb(128,128=
,128);font-family:Arial,sans-serif;font-size:10px">NOTICE: This message is =
intended for the use of the individual or entity to which it is addressed a=
nd may contain information that is confidential, privileged and exempt from=
 disclosure under applicable law. If the reader of this message is not the =
intended recipient, you are hereby notified that any printing, copying, dis=
semination, distribution, disclosure or forwarding of this communication is=
 strictly prohibited. If you have received this communication in error, ple=
ase contact the sender immediately and delete it from your system. Thank Yo=
u.</span>
--047d7b5d340cd86fd404ef90f041--