hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Vimont <dan...@commonvox.org>
Subject Re: Fully distributed mode on Windows
Date Tue, 16 Aug 2016 02:50:25 GMT
Hello Xiaochun,

Below are the notes that I took when I was trying to install HBase in a
Windows environment using Cygwin. These notes are from approximately May of
2015, so I have no idea how well they apply to any current attempt to
install HBase in this kind of environment. Also, I'm afraid that I can't
provide any clarification on anything in the notes below, since I have only
the vaguest memories of having performed the steps outlined. Importantly,
also note that I attempted this install BEFORE completing the
rather-straightforward installation of HBase in Ubuntu, so some of my steps
below may have been blatantly wrong in the context of ANY attempted
installation of HBase.

I simply offer these unedited notes to you "as is" in case there's anything
here that might help you.

Again, good luck!!

Dan

===================
*HBase with cygwin installation notes*

Decided to try to install HBase using the instructions on the project site
itself (although even this resource may not be up-to-date):
https://hbase.apache.org/cygwin.html

In parallel, I also consulted the webpage on the O’Reilly site with
instructions for installing Hadoop:
http://archive.oreilly.com/pub/a/other-programming/excerpts/hadoop-tdg/installing-apache-hadoop.html

Downloaded cygwin and installed it. The elaborate instructions that state
that setting of Windows Environment Variable (CYGWIN_HOME) and PATH
variable appear to be extraneous, since Cygwin “just worked” for me after I
installed it.

The project site install instructions advise you to install the SSH modules
of Cygwin; however, the O’Reilly site advises that SSH is not needed in
stand-alone mode (since no daemons will be started). So, for now, I skipped
SSH installation (although I will need SSH when I get my new full
development environment set up in a few days).

Rather curiously, the project site install instructions then say that HBase
must be “installed in Cygwin”!! (The O’Reilly instructions merely say that
it must be somewhere on the machine, but the O’Reilly instructions assume
you’re in Unix, so I will go with the project-site instructions and install
HBase in a Cygwin subdirectory, as they advise.

Ran the “unzip” (tar) facility to unzip the HBase files after I downloaded
them from Apache server.

Then the “readme” directed me to the “index.html” file in the “docs” folder
of the installed files; I went into this and went directly to “Quick start
(standalone HBase)”.

The quick-start guide said to set JAVA_HOME, which I’ve already done.
Then it says to alter hbase-site.xml in the ./conf subdirectory, in order
to set the directories to which HBase and ZooKeeper write data. However, it
says that if you do NOT do this, then data will be written to a /tmp
directory. This is okay with me for now, so will NOT modify hbase-site.xml
for now.

Ugh -- quite confusing contradictory instructions in the various guides, so
now I go back to the “HBase” chapter in the book, which has very simple
(perhaps too simple?) instructions.

Followed the chapter’s instructions to create HBASE_HOME environment
variable, and then issue the command in Cygwin to add HBASE_HOME/bin to
path. However, the “export” command does not apparently work in the Cygwin
context, so manually added HBASE_HOME/bin in Windows “path” env variable.

Subsequently running the start-hbase.sh script (as simple book instructions
say to do) results in errors (various basic HBase classes not found) --
apparent classpath issues.

SOOO -- better go back to more complex instructions
https://hbase.apache.org/cygwin.html
 -- SPECIFICALLY, need to go back to Cygwin installation and set
appropriate env and path variables: CYGWIN_HOME and add CYGWIN_HOME/bin to
path.

Had a hard time interpreting the following (but finally figured it out):

>
>
>
>
> *JavaOne important thing to remember in shell scripting in general (i.e.
> *nix and Windows) is that managing, manipulating and assembling path names
> that contains spaces can be very hard, due to the need to escape and quote
> those characters and strings. So we try to stay away from spaces in path
> names. *nix environments can help us out here very easily by using symbolic
> links.Create a link in /usr/local to the Java home directory by using the
> following command and substituting the name of your chosen Java
> environment:LN -s /cygdrive/c/Program\ Files/Java/<jre name>
> /usr/local/<jre name>Test your java installation by changing directories to
> your Java folder CD /usr/local/<jre name> and issueing the command
> ./bin/java -version. This should output your version of the chosen JRE.*


The left slash after “Program” above is an escape character for the space
in “Program Files”!!

Major FUBAR: The text says to modify hbase-defaults.xml in the conf/
directory, but the file does not exist. Another source says that
hbase-defaults should NOT be modified, but instead hbase-site.xml is the
one to be modified. This makes sense; in fact, it makes sense that in the
current release of HBase that they TOOK AWAY the defaults.xml, since the
user is NOT supposed to edit it anyway.

Correct instructions (which have not been applied to the “Windows”
instructions) appear to be here: https://hbase.apache.org/book.html

=================
Picking up on next day (2015-05-26) -- going back to top of Windows-install
instructions and doing all steps.

Roadblock:
Got to the step for configuration of SSH (even though it seems to not be
germane to stand-alone usage):
On the bullet point where it says to modify rights-settings on certain
files, “passwd” and “group” files DO NOT EXIST!!
NOTE that these files DID come into existence after I ran the
ssh-host-config script below -- so these steps appear to be out of order!!

Shortly after that problem, this ambiguity came up:
Instruction sez:
Edit the /etc/hosts.allow file using your favorite editor (why not VI in
the shell!) and make sure the following two lines are in there before the
PARANOID line:
ALL : localhost 127.0.0.1/32 : allow
ALL : [::1]/128 : allow

And my hosts.allow file has the single line:
ALL : localhost 127.0.0.1/32 [::1]/128 [::fff:127.0.0.1]/128 : allow
I’m gonna presume that this one line covers the need for the two lines in
the instructions.

Ran the ssh-host-config script, following the vague instructions given.
NOTE that cyg_server password is ********
At end of script, message said:
To start the [new] service now, call ‘net start sshd’ or ‘cygrunsrv --S
sshd’.
I suppose that, in this context, “call” simply means -- enter on the
command line??

OKAY: ABANDONED NETWORKED MODE -- GOING LOCAL ONLY
Will avoid using anything other than local mode on Windows, as it does not
appear to be fully supported. (Wait for virtual Ubuntu, after I get new
Windows box.)

After much searching, found help with missing winutils.exe file. Somebody
made them available on GitHub project:
http://zutai.blogspot.com/2014/06/build-install-and-run-hadoop-24-240-on.html
The “fix” also required making modes to hbase settings script outlined here:
http://stackoverflow.com/questions/23136092/hadoophbase-cluster-on-windows-winutils-not-found

Now I can start and stop HBase in a Cygwin shell with only this type of
error/warning:
“cygpath: can’t convert empty path” -- hopefully this is just informational
and not deadly; however, it certainly suggests Windows environment is not
anywhere near to “robust” for HBase.

Now I try to create a table in HBase, and get the following error:
ERROR: The node /hbase is not in ZooKeeper. It should have been written by
the master. Check the value configured in 'zookeeper.znode.parent'. There
could be a mismatch with the one configured in the master.

Looking this up leads me to stackoverflow entry which suggests mismatch
between HBase and Hadoop versions!!

CANNOT GET PAST ZooKeeper error (same prob in both v0.98 and v1.1).
Abandoning this until I can install/run something in Ubuntu.


On Tue, Aug 16, 2016 at 10:53 AM, Xiaochun Liang <xiaochun.liang@skype.net>
wrote:

> Thanks Dan!
>
> Looks like it is a little bit harder to setup HBase on Windows with
> Cygwin. Thanks a lot for your suggestions. Looks like I still need to setup
> HBase with Cygwin. Could you please share the problems you met?
>
> Regards,
> xiaochun
>
> -----Original Message-----
> From: Daniel Vimont [mailto:daniel@commonvox.org]
> Sent: Monday, August 15, 2016 11:16 AM
> To: user@hbase.apache.org
> Subject: Re: Fully distributed mode on Windows
>
> FWIW, I tried quite some time ago to set up HBase on Windows with CygWin
> (just to see if I could do it), and I ran into so many problems that I
> simply quit trying after a few hours.
>
> However, I DO use a Windows box to do all of my HBase-contributor work on
> (and I do it in the way that Mike has already recommended to you): I do
> that by running VirtualBox with an instance of Ubuntu running inside of it,
> and then just do straightforward installation of HBase in that virtual
> Ubuntu environment. If you must use a Windows box (or multiple Windows
> workstations), you might want to consider a setup similar to this to avoid
> what I would anticipate to be a mountain of frustrations.
>
> Good luck!
>
> Dan
>
> On Sun, Aug 14, 2016 at 9:21 AM, Xiaochun Liang <xiaochun.liang@skype.net>
> wrote:
>
> > Thanks Mike! Is it possible to setup fully distributed mode of HBase
> > on Windows with Cygwin?
> >
> > Regards,
> > xiaochun
> >
> > -----Original Message-----
> > From: Mike Thomsen [mailto:mikerthomsen@gmail.com]
> > Sent: Saturday, August 13, 2016 8:03 PM
> > To: user@hbase.apache.org
> > Subject: Re: Fully distributed mode on Windows
> >
> > You probably shouldn't be using Windows servers for this. For example,
> > Cloudera doesn't list any version of Windows at all as a supported OS:
> >
> > https://na01.safelinks.protection.outlook.com/?url=
> > http%3a%2f%2fwww.cloudera.com%2fdocumentation%2fcdh%2f5-0-x%
> > 2fCDH5-Requirements-and-Supported-Versions%2fcdhrsv_os.html&data=01%7c
> > 01%
> > 7cxiliang%40064d.mgd.microsoft.com%7c2935de2adccc4f60232508d3c371ce3f%
> > 7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=glqVHjC8sWJTYA%
> > 2f2P9nlD63U5sdyQDLqrU%2boMzT%2bN5s%3d
> >
> > You'd almost certainly be better off setting up a few Linux VMs on
> > those servers and running it that way.
> >
> > On Fri, Aug 12, 2016 at 10:36 PM, Xiaochun Liang
> > <xiaochun.liang@skype.net
> > >
> > wrote:
> >
> > > Hi,
> > >
> > > Does Hbase support fully distributed mode on Windows?
> > >
> > > I am now trying to setup Hbase on Windows servers. I followed the
> > > steps in
> > > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fhba
> > > se .apache.org%2fcygwin.html&data=01%7c01%7cxiliang%40064d.
> > mgd.microsoft.com%7c2935de2adccc4f60232508d3c371ce3f%
> > 7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=wcNiVI9Tvi6v%2b%
> > 2fMm3l5d9p9hgS3UXdEmXPBrCxFHI5Q%3d, and can launch a standalone Hbase
> > successfully on Windows. Next step I want to setup Hbase on Windows
> > servers with distributed mode. However I saw followings in
> 'start-hbase.cmd':
> > >
> > > "if "%distMode%"=="true" (
> > >      @echo This is not implemented yet. Stay tuned."
> > >
> > > I am wondering if distributed mode of Hbase is supported on Windows
> > > or
> > not.
> > >
> > > Anyone has experience to setup fully-distributed Hbase on Windows?
> > >
> > > Regards,
> > > xiaochun
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message