hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From yonghu <yongyong...@gmail.com>
Subject Re: zookeeper connection problem in pseudo mode
Date Fri, 06 Jan 2012 16:59:59 GMT
Hello Tom,

Thanks for your suggestion. I also felt that Zookeeper had problem. I
will try to install Zookeeper separate from HBase.

Yong

On Fri, Jan 6, 2012 at 5:01 PM, Tom Wilcox <Tom.Wilcox@cse.org.uk> wrote:
> Yong,
>
> Following Leonardo's response, when I run ifconfig on our regionserver (also our namenode)
I get the following:
>
>
> [hadoop1@namenode hadoop-1.0.0]$ ifconfig
> eth0      Link encap:Ethernet  HWaddr 00:15:5D:00:E5:09
>          inet addr:10.0.0.235  Bcast:10.255.255.255  Mask:255.0.0.0
>          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>          RX packets:19919141 errors:0 dropped:0 overruns:0 frame:4
>          TX packets:12620289 errors:0 dropped:0 overruns:0 carrier:0
>          collisions:0 txqueuelen:1000
>          RX bytes:15386337069 (14.3 GiB)  TX bytes:5571908867 (5.1 GiB)
>          Interrupt:9 Base address:0x6000
>
> lo        Link encap:Local Loopback
>          inet addr:127.0.0.1  Mask:255.0.0.0
>          UP LOOPBACK RUNNING  MTU:16436  Metric:1
>          RX packets:3121776 errors:0 dropped:0 overruns:0 frame:0
>          TX packets:3121776 errors:0 dropped:0 overruns:0 carrier:0
>          collisions:0 txqueuelen:0
>          RX bytes:1272206567 (1.1 GiB)  TX bytes:1272206567 (1.1 GiB)
>
> Here's the contents of our hbase-site.xml, however, please note that we are running our
zookeepers outside of hbase (i.e. in $HBASE_HOME/conf/hbase-env.sh we have set the following:
 "export HBASE_MANAGES_ZK=false" ) and we are running in fully distributed mode, so I am
not sure how useful this will be to you:
>
> [hadoop1@namenode hadoop-1.0.0]$ cat /opt/hbase/hbase-0.92/conf/hbase-site.xml
> <?xml version="1.0"?>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> <!--
> /**
>  * Copyright 2010 The Apache Software Foundation
>  *
>  * Licensed to the Apache Software Foundation (ASF) under one
>  * or more contributor license agreements.  See the NOTICE file
>  * distributed with this work for additional information
>  * regarding copyright ownership.  The ASF licenses this file
>  * to you under the Apache License, Version 2.0 (the
>  * "License"); you may not use this file except in compliance
>  * with the License.  You may obtain a copy of the License at
>  *
>  *     http://www.apache.org/licenses/LICENSE-2.0
>  *
>  * Unless required by applicable law or agreed to in writing, software
>  * distributed under the License is distributed on an "AS IS" BASIS,
>  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>  * See the License for the specific language governing permissions and
>  * limitations under the License.
>  */
> -->
> <configuration>
> <!-- Fully Distributed -->
>  <property>
>    <name>hbase.rootdir</name>
>    <value>hdfs://namenode:8020/hbase</value>
>    <description>The directory shared by RegionServers.</description>
>  </property>
>  <property>
>    <name>hbase.cluster.distributed</name>
>    <value>true</value>
>    <description>
>                The mode the cluster will be in. Possible values are
>                false: standalone and pseudo-distributed setups with managed Zookeeper
>                true: fully-distributed with unmanaged Zookeeper Quorum (see hbase-env.sh)
>    </description>
>  </property>
>
> <!-- ZooKeeper config -->
>    <property>
>      <name>hbase.zookeeper.property.clientPort</name>
>      <value>2181</value>
>      <description>
>                Property from ZooKeeper's config zoo.cfg.
>                The port at which the clients will connect.
>      </description>
>    </property>
>    <property>
>      <name>hbase.zookeeper.quorum</name>
>      <value>namenode, jobtracker, slave0</value>
>      <description>
>        Comma separated list of servers in the ZooKeeper Quorum.
>        For example, "host1.mydomain.com,host2.mydomain.com,host3.mydomain.com".
>        By default this is set to localhost for local and pseudo-distributed modes
>        of operation. For a fully-distributed setup, this should be set to a full
>        list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in hbase-env.sh
>        this is the list of servers which we will start/stop ZooKeeper on.
>      </description>
>    </property>
>    <property>
>      <name>hbase.zookeeper.property.dataDir</name>
>      <value>/zookeeper_data</value>
>      <description>
>                Property from ZooKeeper's config zoo.cfg.
>                The directory where the snapshot is stored.
>      </description>
>    </property>
>
> <!-- Pseudo-distributed -->
> <!--
>        <property>
>                <name>hbase.rootdir</name>
>                <value>hdfs://namenode:8020/hbase</value>
>                <description>
>                        The directory shared by RegionServers.
>                </description>
>        </property>
>        <property>
>                <name>dfs.replication</name>
>                <value>1</value>
>                <description>
>                        The replication count for HLog and HFile storage.
Should not be greater than                                          
                                                           
     HDFS datanode count.
>                </description>
>        </property>
> -->
> <!-- Standalone -->
> <!--
>        <property>
>            <name>hbase.rootdir</name>
>            <value>file:///hbase_data</value>
>        </property>
> -->
> </configuration>
>
>
>
> It does sound like it could be a ZooKeeper issue. Have you tried installing zookeeper
separately as we have done? Note that the pseudo-distributed settings we used are still commented
out at the bottom.
>
> HTH
>
> Tom
>
> ________________________________________
> From: Leonardo Gamas [leogamas@jusbrasil.com.br]
> Sent: 06 January 2012 15:05
> To: user@hbase.apache.org
> Subject: Re: zookeeper connection problem in pseudo mode
>
> Seems you does not have a loopback device configured. How many network
> interfaces appear when you run the command:
>
> $ ifconfig
>
>
> 2012/1/6 yonghu <yongyong313@gmail.com>
>
>> Hallo,
>>
>> I input the first command
>> $ telnet localhost 2181
>> the output is
>>
>> Trying 127.0.0.1...
>> Trying ::1...
>> telnet: Unable to connect to remote host: Network is unreachable
>>
>> As I run the pseudo mode, server and client are the same. So I input
>>
>>  netstat -na | grep LISTEN | grep 2181
>>
>> But I got no output information.
>>
>> Yong
>> On Fri, Jan 6, 2012 at 2:25 PM, Leonardo Gamas
>> <leogamas@jusbrasil.com.br> wrote:
>> > Yong,
>> >
>> > Just to have more info, if you run locally to your server:
>> >
>> > $ telnet localhost 2181
>> >
>> > It connects?
>> >
>> > And if you run this command from your host to the server:
>> >
>> > $ telnet <server_ip or hostname> 2181
>> >
>> > What happens?
>> >
>> > In the server:
>> >
>> > $ netstat -na | grep LISTEN | grep 2181
>> >
>> > What is printed?
>> >
>> > 2012/1/6 yonghu <yongyong313@gmail.com>
>> >
>> >> Thanks for your response. I use Ubuntu 10.04 and the following is the
>> >> command information I got
>> >>
>> >> huyong@huyong-laptop:~$ sudo /etc/init.d/iptables stop
>> >> [sudo] password for huyong:
>> >> sudo: /etc/init.d/iptables: command not found
>> >>
>> >> The way you did can't work for me. The way I used is from
>> >> https://help.ubuntu.com/community/IptablesHowTo
>> >>
>> >> Can you show me your hbase configuration. I mean the content of
>> >> hbase-site.xml. Or you can give me link that I can check if I did the
>> >> right configuration steps. Thanks very much.
>> >>
>> >> Yong
>> >>
>> >> On Fri, Jan 6, 2012 at 12:23 PM, Tom Wilcox <Tom.Wilcox@cse.org.uk>
>> wrote:
>> >> > Hi Yong,
>> >> >
>> >> > When I run:
>> >> >
>> >> > sudo /etc/init.d/iptables stop
>> >> >
>> >> > I see the following output:
>> >> >
>> >> > iptables: Flushing firewall rules:                      
  [  OK  ]
>> >> > iptables: Setting chains to policy ACCEPT: filter          [  OK
 ]
>> >> > iptables: Unloading modules:                          
    [  OK  ]
>> >> >
>> >> > And my connection stopped being refused. Since I wasn't 100% sure what
>> >> ports needed opening for HBase and co to work and my system was on a
>> closed
>> >> development network, I was happy just to stop the firewall services
>> >> completely as above, rather than to modify the rules.
>> >> >
>> >> > We have successfully combined (Hadoop 0.20.2 with HBase 0.90.4) and
>> >> (Hadoop 1.0.0 with HBase 0.92).
>> >> >
>> >> > HTH
>> >> >
>> >> > Tom
>> >> > ________________________________________
>> >> > From: yonghu [yongyong313@gmail.com]
>> >> > Sent: 05 January 2012 21:22
>> >> > To: user@hbase.apache.org
>> >> > Subject: Re: zookeeper connection problem in pseudo mode
>> >> >
>> >> > this is how I stop the firewall.
>> >> https://help.ubuntu.com/community/IptablesHowTo
>> >> >
>> >> > sudo nano -w /root/fw.stop
>> >> >
>> >> > echo "Stopping firewall and allowing everyone..."
>> >> > iptables -F
>> >> > iptables -X
>> >> > iptables -t nat -F
>> >> > iptables -t nat -X
>> >> > iptables -t mangle -F
>> >> > iptables -t mangle -X
>> >> > iptables -P INPUT ACCEPT
>> >> > iptables -P FORWARD ACCEPT
>> >> > iptables -P OUTPUT ACCEPT
>> >> >
>> >> > sudo /root/fw.stop
>> >> >
>> >> > at last I saw the output:
>> >> > huyong@huyong-laptop:~/hbase-0.90.3/bin$ sudo /root/fw.stop
>> >> > stopping firewall
>> >> >
>> >> > I am not sure weather it is what you mean to close the firewall. I
>> >> > don't know if my Hadoop version 0.20.2 and Hbase 0.90.3 version are
>> >> > compatible.
>> >> > Yong
>> >> >
>> >> >
>> >> > On Thu, Jan 5, 2012 at 6:32 PM, Royston Sellman
>> >> > <royston.sellman@googlemail.com> wrote:
>> >> >> Just to check - did you disable it the way Tom suggested? By stopping
>> >> iptables?
>> >> >> It's not sufficient just to turn off firewall from the control
>> >> panel/app.
>> >> >>
>> >> >> Best,
>> >> >> Royston
>> >> >>
>> >> >>
>> >> >> On 5 Jan 2012, at 17:00, yonghu wrote:
>> >> >>
>> >> >>> I have already disabled ipv6 and close the firewall, but I
still get
>> >> >>> the same problem. :(
>> >> >>>
>> >> >>> Yong
>> >> >>>
>> >> >>> On Thu, Jan 5, 2012 at 5:16 PM, yonghu <yongyong313@gmail.com>
>> wrote:
>> >> >>>> I set up a pseudo mode, it also needs to close the firewall?
>> >> >>>>
>> >> >>>> Yong
>> >> >>>>
>> >> >>>> On Thu, Jan 5, 2012 at 4:59 PM, Tom Wilcox <Tom.Wilcox@cse.org.uk>
>> >> wrote:
>> >> >>>>> I've seen that error before. Have you tried turning
off your
>> >> firewalls on each node?
>> >> >>>>>
>> >> >>>>> I do it like this:
>> >> >>>>>
>> >> >>>>> cat $HADOOP_INSTALL/conf/slaves | xargs -I{} ssh root@{}
>> >> '/etc/init.d/iptables stop'
>> >> >>>>> ssh root@jobtracker '/etc/init.d/iptables stop'
>> >> >>>>> ssh root@namenode '/etc/init.d/iptables stop'
>> >> >>>>>
>> >> >>>>> Worth a crack if you haven't tried it yet.
>> >> >>>>> ________________________________________
>> >> >>>>> From: Leonardo Gamas [leogamas@jusbrasil.com.br]
>> >> >>>>> Sent: 05 January 2012 15:45
>> >> >>>>> To: user@hbase.apache.org
>> >> >>>>> Subject: Re: zookeeper connection problem in pseudo
mode
>> >> >>>>>
>> >> >>>>> I had a similar problem some time ago. The problem
seems to be in
>> >> the ipv6
>> >> >>>>> configuration. I solved it disabling ipv6 on my host
editing
>> >> /etc/hosts.
>> >> >>>>>
>> >> >>>>> 2012/1/5 yonghu <yongyong313@gmail.com>
>> >> >>>>>
>> >> >>>>>> Hallo,
>> >> >>>>>>
>> >> >>>>>> I tried the pseudo distribution mode of HBase.
The Hadoop
>> version is
>> >> >>>>>> 0.20.2. I configured the hbase-site.xml as
>> >> >>>>>>
>> >> >>>>>> <configuration>
>> >> >>>>>>  <property>
>> >> >>>>>>    <name>hbase.rootdir</name>
>> >> >>>>>>    <value>hdfs://localhost:9000/hbase</value>
>> >> >>>>>>  </property>
>> >> >>>>>>  <property>
>> >> >>>>>>    <name>dfs.replication</name>
>> >> >>>>>>    <value>1</value>
>> >> >>>>>>  </property>
>> >> >>>>>> </configuration>
>> >> >>>>>>
>> >> >>>>>> as same as in core-site.xml and hdfs-site.xml files
of Hadoop,
>> >> separately.
>> >> >>>>>>
>> >> >>>>>> I can successfully start HBase. But when I tried
to create a
>> table,
>> >> I
>> >> >>>>>> got following error information:
>> >> >>>>>> ERROR: org.apache.hadoop.hbase.ZooKeeperConnectionException:
>> HBase
>> >> is
>> >> >>>>>> able to connect to ZooKeeper but the connection
closes
>> immediately.
>> >> >>>>>> This could be a sign that the server has too many
connections
>> (30 is
>> >> >>>>>> the default). Consider inspecting your ZK server
logs for that
>> error
>> >> >>>>>> and then make sure you are reusing HBaseConfiguration
as often as
>> >> you
>> >> >>>>>> can. See HTable's javadoc for more information.
>> >> >>>>>>
>> >> >>>>>> And I checked my ZK log and found:
>> >> >>>>>>
>> >> >>>>>> 2012-01-05 15:59:02,909 INFO org.apache.zookeeper.ClientCnxn:
>> >> Opening
>> >> >>>>>> socket connection to server localhost/0:0:0:0:0:0:0:1:2181
>> >> >>>>>> 2012-01-05 15:59:02,909 WARN org.apache.zookeeper.ClientCnxn:
>> >> Session
>> >> >>>>>> 0x134ae4d4a6d0001 for server null, unexpected error,
closing
>> socket
>> >> >>>>>> connection and attempting reconnect
>> >> >>>>>> java.net.ConnectException: Connection refused
>> >> >>>>>>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native
>> Method)
>> >> >>>>>>        at
>> >> >>>>>>
>> >> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
>> >> >>>>>>        at
>> >> >>>>>>
>> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119)
>> >> >>>>>>
>> >> >>>>>> This error information appeared repeatly. Can anyone
tell how to
>> >> solve
>> >> >>>>>> this problem?
>> >> >>>>>>
>> >> >>>>>> Thanks!
>> >> >>>>>>
>> >> >>>>>> Yong
>> >> >>>>>>
>> >> >>>>>
>> >> >>>>>
>> >> >>>>>
>> >> >>>>> --
>> >> >>>>>
>> >> >>>>> *Leonardo Gamas*
>> >> >>>>> Software Engineer
>> >> >>>>> +557134943514
>> >> >>>>> +557581347440
>> >> >>>>> leogamas@jusbrasil.com.br
>> >> >>>>> www.jusbrasil.com.br
>> >> >>
>> >>
>> >
>> >
>> >
>> > --
>> >
>> > *Leonardo Gamas*
>> > Software Engineer
>> > T +55 (71) 3494-3514
>> > C +55 (75) 8134-7440
>> > leogamas@jusbrasil.com.br
>> > www.jusbrasil.com.br
>>
>
>
>
> --
>
> *Leonardo Gamas*
> Software Engineer
> T +55 (71) 3494-3514
> C +55 (75) 8134-7440
> leogamas@jusbrasil.com.br
> www.jusbrasil.com.br

Mime
View raw message