hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wadood.Chaudh...@Instinet.com
Subject Re: Hadoop Multi-node cluster on Windows
Date Thu, 05 Jun 2014 06:29:09 GMT
Thanks for looking into it. I have used Horton but that is a third party
tool and should not be required when Apache has started supporting Windows.
Here is the Wiki article on Windows from Apache's.
https://wiki.apache.org/hadoop/Hadoop2OnWindows
I am able to setup in Pseudo distributed mode using the instructions above.
Unfortunately, the article ends there saying that instructions for
Multi-cluster will be added later. No mention that you cannot do it under
Windows or you must use Horton.

Wadood




From:	Nitin Pawar <nitinpawar432@gmail.com>
To:	user@hadoop.apache.org,
Date:	06/05/2014 02:09 AM
Subject:	Re: Hadoop Multi-node cluster on Windows



Currently as per my knowledge only Hortonwork's distribution comes for
windows.

As such hadoop does not need ssh to work but ssh in linux was needed for
the automated deployments.

If you want to avoid that then, I would say setup a single machine with all
installation where all packages and correct configurations are present.
then make an image out of it and build other nodes.

But then you will need to login to all nodes and start all services
manually.

Hortonwork's HDP 2.1 on windows step by step manual install guide is here
http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.1-Win-latest/bk_releasenotes_HDP-Win/content/index.html


On Thu, Jun 5, 2014 at 11:29 AM, <Wadood.Chaudhary@instinet.com> wrote:
  Thanks but that is almost a 5 year old tutorial, written when Hadoop was
  not natively supported on Windows. As of Hadoop 2.x, Windows is supported
  and setup is much simpler. No Cygwin required.

  Inactive hide details for Ted Yu ---06/04/2014 10:05:15 PM---See also
  http://ebiquity.umbc.edu/Tutorials/Hadoop/05%20-%20Setup%Ted Yu
  ---06/04/2014 10:05:15 PM---See also
  http://ebiquity.umbc.edu/Tutorials/Hadoop/05%20-%20Setup%20SSHD.html

  From: Ted Yu <yuzhihong@gmail.com>
  To: "common-user@hadoop.apache.org" <user@hadoop.apache.org>,
  Date: 06/04/2014 10:05 PM



  Subject: Re: Hadoop Multi-node cluster on Windows




  See also
  http://ebiquity.umbc.edu/Tutorials/Hadoop/05%20-%20Setup%20SSHD.html


  On Wed, Jun 4, 2014 at 6:43 PM, <Wadood.Chaudhary@instinet.com> wrote:
        Thanks. Linux cluster requires SSH etc. for every thing including
        single node. In Windows until now I have been able to setup to
        pseudo distributed level without SSH or CygWin. Someone was saying
        that some Windows service needs to be installed for communication
        across machines.
        Is that true?








        ----- Message from "Olivier Renault" on 06/04/2014 09:36:17 PM
        -----
                                         
       To:         user@hadoop.apache.or 
                   g                     
                                         
       Subje       RE: Hadoop Multi-node 
       ct:         cluster on Windows    
                                         




        HDP run natively on Linux and Windows. It's the same code base. You
        can install a windows cluster in the same way as you do for Linux.


        Olivier


        On 4 Jun 2014 18:23, <Wadood.Chaudhary@instinet.com> wrote:


              Yes, but Horton runs Hadoop under Linux Virtual Machine. I am
              talking of pure and simple Hadoop.








              ----- Message from "Ted Yu" on 06/04/2014 09:18:51 PM -----


                                           
       To:         "                       
                   common-user@hadoop.apac 
                   he.org"                 
                                           
       Subje       Re: Hadoop Multi-node   
       ct:         cluster on Windows      
                                           



        Have you seen this ?

        http://hortonworks.com/blog/install-hadoop-windows-hortonworks-data-platform-2-0/


        Cheers


        On Wed, Jun 4, 2014 at 5:47 PM, <Wadood.Chaudhary@instinet.com>
        wrote:
              Has anyone successfully implemented Multi-node cluster under
              Windows using the latest version of Hadoop. We were able to
              run Pseudo distributed. Does it still needs SSH under
              Windows, or something installed as service.

              Inactive hide details for ch huang ---06/04/2014 08:40:38
              PM---hi,mailist:               my company signed a new IDC ,
              i need mch huang ---06/04/2014 08:40:38 PM---hi,mailist:
              my company signed a new IDC , i need move the total 50T data

              From: ch huang <justlooks@gmail.com>
              To: user@hadoop.apache.org,
              Date: 06/04/2014 08:40 PM
              Subject: issue about move data between two hdoop cluster



              hi,mailist:
                            my company signed a new IDC , i need move the
              total 50T data from old hadoop cluster to the new cluster in
              new location ,how to do it?


              =========================================================================================================



              <<<< Disclaimer >>>>


              This message is intended solely for use by the named
              addressee(s). If you receive this transmission in error,
              please immediately notify the sender and destroy this message
              in its entirety, whether in electronic or hard copy format.
              Any unauthorized use (and reliance thereon), copying,
              disclosure, retention, or distribution of this transmission
              or the material in this transmission is forbidden. We reserve
              the right to monitor and archive electronic communications.
              This material does not constitute an offer or solicitation
              with respect to the purchase or sale of any security. It
              should not be construed to contain any recommendation
              regarding any security or strategy. Any views expressed are
              those of the individual sender, except where the message
              states otherwise and the sender is authorized to state them
              to be the views of any such entity. This communication is
              provided on an “as is” basis. It contains material that is
              owned by Instinet Incorporated, its subsidiaries or its or
              their licensors, and may not, in whole or in part, be (i)
              copied, photocopied or duplicated in any form, by any means,
              or (ii) redistributed, posted, published, excerpted, or
              quoted without Instinet Incorporated's prior written consent.
              Please access the following link for important information
              and instructions:
              http://instinet.com/includes/index.jsp?thePage=/html/le_index.txt


              Securities products and services are provided by locally
              registered brokerage subsidiaries of Instinet Incorporated:
              Instinet Australia Pty Limited (ACN: 131 253 686 AFSL No:
              327834), regulated by the Australian Securities & Investments
              Commission; Instinet Canada Limited, member IIROC/CIPF;
              Instinet Pacific Limited, authorized and regulated by the
              Securities and Futures Commission of Hong Kong; Instinet
              Singapore Services Private Limited, regulated by the Monetary
              Authority of Singapore, trading member of The Singapore
              Exchange Securities Trading Private Limited and clearing
              member of The Central Depository (Pte) Limited; and Instinet,
              LLC, member SIPC.




              =========================================================================================================







        =========================================================================================================



        <<<< Disclaimer >>>>


        This message is intended solely for use by the named addressee(s).
        If you receive this transmission in error, please immediately
        notify the sender and destroy this message in its entirety, whether
        in electronic or hard copy format. Any unauthorized use (and
        reliance thereon), copying, disclosure, retention, or distribution
        of this transmission or the material in this transmission is
        forbidden. We reserve the right to monitor and archive electronic
        communications. This material does not constitute an offer or
        solicitation with respect to the purchase or sale of any security.
        It should not be construed to contain any recommendation regarding
        any security or strategy. Any views expressed are those of the
        individual sender, except where the message states otherwise and
        the sender is authorized to state them to be the views of any such
        entity. This communication is provided on an “as is” basis. It
        contains material that is owned by Instinet Incorporated, its
        subsidiaries or its or their licensors, and may not, in whole or in
        part, be (i) copied, photocopied or duplicated in any form, by any
        means, or (ii) redistributed, posted, published, excerpted, or
        quoted without Instinet Incorporated's prior written consent.
        Please access the following link for important information and
        instructions:
        http://instinet.com/includes/index.jsp?thePage=/html/le_index.txt


        Securities products and services are provided by locally registered
        brokerage subsidiaries of Instinet Incorporated: Instinet Australia
        Pty Limited (ACN: 131 253 686 AFSL No: 327834), regulated by the
        Australian Securities & Investments Commission; Instinet Canada
        Limited, member IIROC/CIPF; Instinet Pacific Limited, authorized
        and regulated by the Securities and Futures Commission of Hong
        Kong; Instinet Singapore Services Private Limited, regulated by the
        Monetary Authority of Singapore, trading member of The Singapore
        Exchange Securities Trading Private Limited and clearing member of
        The Central Depository (Pte) Limited; and Instinet, LLC, member
        SIPC.




        =========================================================================================================




        CONFIDENTIALITY NOTICE
        NOTICE: This message is intended for the use of the individual or
        entity to which it is addressed and may contain information that is
        confidential, privileged and exempt from disclosure under
        applicable law. If the reader of this message is not the intended
        recipient, you are hereby notified that any printing, copying,
        dissemination, distribution, disclosure or forwarding of this
        communication is strictly prohibited. If you have received this
        communication in error, please contact the sender immediately and
        delete it from your system. Thank You.


        =========================================================================================================



        <<<< Disclaimer >>>>


        This message is intended solely for use by the named addressee(s).
        If you receive this transmission in error, please immediately
        notify the sender and destroy this message in its entirety, whether
        in electronic or hard copy format. Any unauthorized use (and
        reliance thereon), copying, disclosure, retention, or distribution
        of this transmission or the material in this transmission is
        forbidden. We reserve the right to monitor and archive electronic
        communications. This material does not constitute an offer or
        solicitation with respect to the purchase or sale of any security.
        It should not be construed to contain any recommendation regarding
        any security or strategy. Any views expressed are those of the
        individual sender, except where the message states otherwise and
        the sender is authorized to state them to be the views of any such
        entity. This communication is provided on an “as is” basis. It
        contains material that is owned by Instinet Incorporated, its
        subsidiaries or its or their licensors, and may not, in whole or in
        part, be (i) copied, photocopied or duplicated in any form, by any
        means, or (ii) redistributed, posted, published, excerpted, or
        quoted without Instinet Incorporated's prior written consent.
        Please access the following link for important information and
        instructions:
        http://instinet.com/includes/index.jsp?thePage=/html/le_index.txt


        Securities products and services are provided by locally registered
        brokerage subsidiaries of Instinet Incorporated: Instinet Australia
        Pty Limited (ACN: 131 253 686 AFSL No: 327834), regulated by the
        Australian Securities & Investments Commission; Instinet Canada
        Limited, member IIROC/CIPF; Instinet Pacific Limited, authorized
        and regulated by the Securities and Futures Commission of Hong
        Kong; Instinet Singapore Services Private Limited, regulated by the
        Monetary Authority of Singapore, trading member of The Singapore
        Exchange Securities Trading Private Limited and clearing member of
        The Central Depository (Pte) Limited; and Instinet, LLC, member
        SIPC.




        =========================================================================================================








  =========================================================================================================



  <<<< Disclaimer >>>>


  This message is intended solely for use by the named addressee(s). If you
  receive this transmission in error, please immediately notify the sender
  and destroy this message in its entirety, whether in electronic or hard
  copy format. Any unauthorized use (and reliance thereon), copying,
  disclosure, retention, or distribution of this transmission or the
  material in this transmission is forbidden. We reserve the right to
  monitor and archive electronic communications. This material does not
  constitute an offer or solicitation with respect to the purchase or sale
  of any security. It should not be construed to contain any recommendation
  regarding any security or strategy. Any views expressed are those of the
  individual sender, except where the message states otherwise and the
  sender is authorized to state them to be the views of any such entity.
  This communication is provided on an “as is” basis. It contains material
  that is owned by Instinet Incorporated, its subsidiaries or its or their
  licensors, and may not, in whole or in part, be (i) copied, photocopied
  or duplicated in any form, by any means, or (ii) redistributed, posted,
  published, excerpted, or quoted without Instinet Incorporated's prior
  written consent. Please access the following link for important
  information and instructions:
  http://instinet.com/includes/index.jsp?thePage=/html/le_index.txt


  Securities products and services are provided by locally registered
  brokerage subsidiaries of Instinet Incorporated: Instinet Australia Pty
  Limited (ACN: 131 253 686 AFSL No: 327834), regulated by the Australian
  Securities & Investments Commission; Instinet Canada Limited, member
  IIROC/CIPF; Instinet Pacific Limited, authorized and regulated by the
  Securities and Futures Commission of Hong Kong; Instinet Singapore
  Services Private Limited, regulated by the Monetary Authority of
  Singapore, trading member of The Singapore Exchange Securities Trading
  Private Limited and clearing member of The Central Depository (Pte)
  Limited; and Instinet, LLC, member SIPC.




  =========================================================================================================






--
Nitin Pawar

=========================================================================================================
 <<<< Disclaimer >>>>   This message is intended solely for use by
the named addressee(s). If you receive this transmission in error, please immediately notify
the sender and destroy this message in its entirety, whether in electronic or hard copy format.
Any unauthorized use (and reliance thereon), copying, disclosure, retention, or distribution
of this transmission or the material in this transmission is forbidden. We reserve the right
to monitor and archive electronic communications. This material does not constitute an offer
or solicitation with respect to the purchase or sale of any security. It should not be construed
to contain any recommendation regarding any security or strategy. Any views expressed are
those of the individual sender, except where the message states otherwise and the sender is
authorized to state them to be the views of any such entity. This communication is provided
on an “as is” basis. It contains material that is owned by Instinet Incorporated, its
subsidiaries or its or their licensors, and may not, in whole or in part, be (i) copied, photocopied
or duplicated in any form, by any means, or (ii) redistributed, posted, published, excerpted,
or quoted without Instinet Incorporated's prior written consent. Please access the following
link for important information and instructions:  http://instinet.com/includes/index.jsp?thePage=/html/le_index.txt
  Securities products and services are provided by locally registered brokerage subsidiaries
of Instinet Incorporated: Instinet Australia Pty Limited (ACN: 131 253 686 AFSL No: 327834),
regulated by the Australian Securities & Investments Commission; Instinet Canada Limited,
member IIROC/CIPF; Instinet Pacific Limited, authorized and regulated by the Securities and
Futures Commission of Hong Kong; Instinet Singapore Services Private Limited, regulated by
the Monetary Authority of Singapore, trading member of The Singapore Exchange Securities Trading
Private Limited and clearing member of The Central Depository (Pte) Limited; and Instinet,
LLC, member SIPC.  

=========================================================================================================
 
Mime
View raw message