hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nitin Pawar <nitinpawar...@gmail.com>
Subject Re: Hadoop Multi-node cluster on Windows
Date Thu, 05 Jun 2014 06:08:59 GMT
Currently as per my knowledge only Hortonwork's distribution comes for
windows.

As such hadoop does not need ssh to work but ssh in linux was needed for
the automated deployments.

If you want to avoid that then, I would say setup a single machine with all
installation where all packages and correct configurations are present.
then make an image out of it and build other nodes.

But then you will need to login to all nodes and start all services
manually.

Hortonwork's HDP 2.1 on windows step by step manual install guide is here
http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.1-Win-latest/bk_releasenotes_HDP-Win/content/index.html


On Thu, Jun 5, 2014 at 11:29 AM, <Wadood.Chaudhary@instinet.com> wrote:

> Thanks but that is almost a 5 year old tutorial, written when Hadoop was
> not natively supported on Windows. As of Hadoop 2.x, Windows is supported
> and setup is much simpler. No Cygwin required.
>
> [image: Inactive hide details for Ted Yu ---06/04/2014 10:05:15 PM---See
> also http://ebiquity.umbc.edu/Tutorials/Hadoop/05%20-%20Setup%]Ted Yu
> ---06/04/2014 10:05:15 PM---See also
> http://ebiquity.umbc.edu/Tutorials/Hadoop/05%20-%20Setup%20SSHD.html
>
> From: Ted Yu <yuzhihong@gmail.com>
> To: "common-user@hadoop.apache.org" <user@hadoop.apache.org>,
> Date: 06/04/2014 10:05 PM
>
> Subject: Re: Hadoop Multi-node cluster on Windows
> ------------------------------
>
>
>
> See also
> *http://ebiquity.umbc.edu/Tutorials/Hadoop/05%20-%20Setup%20SSHD.html*
> <http://ebiquity.umbc.edu/Tutorials/Hadoop/05%20-%20Setup%20SSHD.html>
>
>
> On Wed, Jun 4, 2014 at 6:43 PM, <*Wadood.Chaudhary@instinet.com*
> <Wadood.Chaudhary@instinet.com>> wrote:
>
>    Thanks. Linux cluster requires SSH etc. for every thing including
>    single node. In Windows until now I have been able to setup to pseudo
>    distributed level without SSH or CygWin. Someone was saying that some
>    Windows service needs to be installed for communication across machines.
>    Is that true?
>
>
>
>
>
>
>
>
>    ----- Message from "Olivier Renault" on 06/04/2014 09:36:17 PM -----
>
>
>
>    *To:*
>
>
>    *user@hadoop.apache.org* <user@hadoop.apache.org>
>
>
>    *Subject:*
>
>
>    RE: Hadoop Multi-node cluster on Windows
>
>
>    HDP run natively on Linux and Windows. It's the same code base. You
>    can install a windows cluster in the same way as you do for Linux.
>
>    Olivier
>
>    On 4 Jun 2014 18:23, <*Wadood.Chaudhary@instinet.com*
>    <Wadood.Chaudhary@instinet.com>> wrote:
>    Yes, but Horton runs Hadoop under Linux Virtual Machine. I am talking
>       of pure and simple Hadoop.
>
>
>
>
>
>
>
>
>       ----- Message from "Ted Yu" on 06/04/2014 09:18:51 PM -----
>
>
>
>    *To:*
>
>
>    "*common-user@hadoop.apache.org* <common-user@hadoop.apache.org>"
>
>
>    *Subject:*
>
>
>    Re: Hadoop Multi-node cluster on Windows
>
>
>
>    Have you seen this ?
>
>
>    *http://hortonworks.com/blog/install-hadoop-windows-hortonworks-data-platform-2-0/*
>    <http://hortonworks.com/blog/install-hadoop-windows-hortonworks-data-platform-2-0/>
>
>    Cheers
>
>
>    On Wed, Jun 4, 2014 at 5:47 PM, <*Wadood.Chaudhary@instinet.com*
>    <Wadood.Chaudhary@instinet.com>> wrote:
>       Has anyone successfully implemented Multi-node cluster under
>       Windows using the latest version of Hadoop. We were able to run Pseudo
>       distributed. Does it still needs SSH under Windows, or something installed
>       as service.
>
>       [image: Inactive hide details for ch huang ---06/04/2014 08:40:38
>       PM---hi,mailist: my company signed a new IDC , i need m]ch huang
>       ---06/04/2014 08:40:38 PM---hi,mailist:               my company signed a
>       new IDC , i need move the total 50T data
>
>       From: ch huang <*justlooks@gmail.com* <justlooks@gmail.com>>
>       To: *user@hadoop.apache.org* <user@hadoop.apache.org>,
>       Date: 06/04/2014 08:40 PM
>       Subject: issue about move data between two hdoop cluster
>       ------------------------------
>
>
>
>       hi,mailist:
>                     my company signed a new IDC , i need move the total
>       50T data from old hadoop cluster to the new cluster in new location ,how to
>       do it?
>
>
> *
>       =========================================================================================================
>       *
>
>       *<<<< Disclaimer >>>>*
>
>       *This message is intended solely for use by the named addressee(s).
>       If you receive this transmission in error, please immediately notify the
>       sender and destroy this message in its entirety, whether in electronic or
>       hard copy format. Any unauthorized use (and reliance thereon), copying,
>       disclosure, retention, or distribution of this transmission or the material
>       in this transmission is forbidden. We reserve the right to monitor and
>       archive electronic communications. This material does not constitute an
>       offer or solicitation with respect to the purchase or sale of any security.
>       It should not be construed to contain any recommendation regarding any
>       security or strategy. Any views expressed are those of the individual
>       sender, except where the message states otherwise and the sender is
>       authorized to state them to be the views of any such entity. This
>       communication is provided on an “as is” basis. It contains material that is
>       owned by Instinet Incorporated, its subsidiaries or its or their licensors,
>       and may not, in whole or in part, be (i) copied, photocopied or duplicated
>       in any form, by any means, or (ii) redistributed, posted, published,
>       excerpted, or quoted without Instinet Incorporated's prior written consent.
>       Please access the following link for important information and
>       instructions: *
>       *http://instinet.com/includes/index.jsp?thePage=/html/le_index.txt*
>       <http://instinet.com/includes/index.jsp?thePage=/html/le_index.txt>
>
>       *Securities products and services are provided by locally
>       registered brokerage subsidiaries of Instinet Incorporated: Instinet
>       Australia Pty Limited (ACN: 131 253 686 AFSL No: 327834), regulated by the
>       Australian Securities & Investments Commission; Instinet Canada Limited,
>       member IIROC/CIPF; Instinet Pacific Limited, authorized and regulated by
>       the Securities and Futures Commission of Hong Kong; Instinet Singapore
>       Services Private Limited, regulated by the Monetary Authority of Singapore,
>       trading member of The Singapore Exchange Securities Trading Private Limited
>       and clearing member of The Central Depository (Pte) Limited; and Instinet,
>       LLC, member SIPC. *
>
>
>
> *
>       =========================================================================================================
>       *
>
>
>
>
>
> *
>    =========================================================================================================
>    *
>
>    *<<<< Disclaimer >>>>*
>
>    *This message is intended solely for use by the named addressee(s). If
>    you receive this transmission in error, please immediately notify the
>    sender and destroy this message in its entirety, whether in electronic or
>    hard copy format. Any unauthorized use (and reliance thereon), copying,
>    disclosure, retention, or distribution of this transmission or the material
>    in this transmission is forbidden. We reserve the right to monitor and
>    archive electronic communications. This material does not constitute an
>    offer or solicitation with respect to the purchase or sale of any security.
>    It should not be construed to contain any recommendation regarding any
>    security or strategy. Any views expressed are those of the individual
>    sender, except where the message states otherwise and the sender is
>    authorized to state them to be the views of any such entity. This
>    communication is provided on an “as is” basis. It contains material that is
>    owned by Instinet Incorporated, its subsidiaries or its or their licensors,
>    and may not, in whole or in part, be (i) copied, photocopied or duplicated
>    in any form, by any means, or (ii) redistributed, posted, published,
>    excerpted, or quoted without Instinet Incorporated's prior written consent.
>    Please access the following link for important information and
>    instructions: *
>    *http://instinet.com/includes/index.jsp?thePage=/html/le_index.txt*
>    <http://instinet.com/includes/index.jsp?thePage=/html/le_index.txt>
>
>    *Securities products and services are provided by locally registered
>    brokerage subsidiaries of Instinet Incorporated: Instinet Australia Pty
>    Limited (ACN: 131 253 686 AFSL No: 327834), regulated by the Australian
>    Securities & Investments Commission; Instinet Canada Limited, member
>    IIROC/CIPF; Instinet Pacific Limited, authorized and regulated by the
>    Securities and Futures Commission of Hong Kong; Instinet Singapore Services
>    Private Limited, regulated by the Monetary Authority of Singapore, trading
>    member of The Singapore Exchange Securities Trading Private Limited and
>    clearing member of The Central Depository (Pte) Limited; and Instinet, LLC,
>    member SIPC. *
>
>
>
> *
>    =========================================================================================================
>    *
>
>
>
>
>    CONFIDENTIALITY NOTICE
>    NOTICE: This message is intended for the use of the individual or
>    entity to which it is addressed and may contain information that is
>    confidential, privileged and exempt from disclosure under applicable law.
>    If the reader of this message is not the intended recipient, you are hereby
>    notified that any printing, copying, dissemination, distribution,
>    disclosure or forwarding of this communication is strictly prohibited. If
>    you have received this communication in error, please contact the sender
>    immediately and delete it from your system. Thank You.
>
>
> *
>    =========================================================================================================
>    *
>
>    *<<<< Disclaimer >>>>*
>
>    *This message is intended solely for use by the named addressee(s). If
>    you receive this transmission in error, please immediately notify the
>    sender and destroy this message in its entirety, whether in electronic or
>    hard copy format. Any unauthorized use (and reliance thereon), copying,
>    disclosure, retention, or distribution of this transmission or the material
>    in this transmission is forbidden. We reserve the right to monitor and
>    archive electronic communications. This material does not constitute an
>    offer or solicitation with respect to the purchase or sale of any security.
>    It should not be construed to contain any recommendation regarding any
>    security or strategy. Any views expressed are those of the individual
>    sender, except where the message states otherwise and the sender is
>    authorized to state them to be the views of any such entity. This
>    communication is provided on an “as is” basis. It contains material that is
>    owned by Instinet Incorporated, its subsidiaries or its or their licensors,
>    and may not, in whole or in part, be (i) copied, photocopied or duplicated
>    in any form, by any means, or (ii) redistributed, posted, published,
>    excerpted, or quoted without Instinet Incorporated's prior written consent.
>    Please access the following link for important information and
>    instructions: *
>    *http://instinet.com/includes/index.jsp?thePage=/html/le_index.txt*
>    <http://instinet.com/includes/index.jsp?thePage=/html/le_index.txt>
>
>    *Securities products and services are provided by locally registered
>    brokerage subsidiaries of Instinet Incorporated: Instinet Australia Pty
>    Limited (ACN: 131 253 686 AFSL No: 327834), regulated by the Australian
>    Securities & Investments Commission; Instinet Canada Limited, member
>    IIROC/CIPF; Instinet Pacific Limited, authorized and regulated by the
>    Securities and Futures Commission of Hong Kong; Instinet Singapore Services
>    Private Limited, regulated by the Monetary Authority of Singapore, trading
>    member of The Singapore Exchange Securities Trading Private Limited and
>    clearing member of The Central Depository (Pte) Limited; and Instinet, LLC,
>    member SIPC. *
>
>
>
> *
>    =========================================================================================================
>    *
>
>
>
>
>
> =========================================================================================================
>
>
> *<<<< Disclaimer >>>>*
>
> This message is intended solely for use by the named addressee(s). If you
> receive this transmission in error, please immediately notify the sender
> and destroy this message in its entirety, whether in electronic or hard
> copy format. Any unauthorized use (and reliance thereon), copying,
> disclosure, retention, or distribution of this transmission or the material
> in this transmission is forbidden. We reserve the right to monitor and
> archive electronic communications. This material does not constitute an
> offer or solicitation with respect to the purchase or sale of any security.
> It should not be construed to contain any recommendation regarding any
> security or strategy. Any views expressed are those of the individual
> sender, except where the message states otherwise and the sender is
> authorized to state them to be the views of any such entity. This
> communication is provided on an “as is” basis. It contains material that is
> owned by Instinet Incorporated, its subsidiaries or its or their licensors,
> and may not, in whole or in part, be (i) copied, photocopied or duplicated
> in any form, by any means, or (ii) redistributed, posted, published,
> excerpted, or quoted without Instinet Incorporated's prior written consent.
> Please access the following link for important information and
> instructions:
> http://instinet.com/includes/index.jsp?thePage=/html/le_index.txt
>
> Securities products and services are provided by locally registered
> brokerage subsidiaries of Instinet Incorporated: Instinet Australia Pty
> Limited (ACN: 131 253 686 AFSL No: 327834), regulated by the Australian
> Securities & Investments Commission; Instinet Canada Limited, member
> IIROC/CIPF; Instinet Pacific Limited, authorized and regulated by the
> Securities and Futures Commission of Hong Kong; Instinet Singapore Services
> Private Limited, regulated by the Monetary Authority of Singapore, trading
> member of The Singapore Exchange Securities Trading Private Limited and
> clearing member of The Central Depository (Pte) Limited; and Instinet, LLC,
> member SIPC.
>
>
>
> =========================================================================================================
>
>
>


-- 
Nitin Pawar

Mime
View raw message