hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcos Ortiz <mlor...@uci.cu>
Subject Re: Failed to start master with HBase 0.94.2 on Hadoop 1.0.4
Date Tue, 06 Nov 2012 13:58:03 GMT

You don´t have to disable firewall, just enable the required ports for 
Hadoop and HBase:
All ports for Hadoop 1.0.4:
  NameNode RPC Server 8020
  JobTracker RPC Server 8021
  RPC Service 50020
  TCP/IP Server for block tranfers 50010
  NameNode HTTP Service 50070
  DataNode HTTP Service 50075
  JobTracker HTTP Service 50030
  TaskTracker HTTP Service 50060
  Secondary NameNode HTTP Service 50090

All ports for HBase:
HBase Master RPC Service 60000
HBase Master HTTP Service 60010
Region Server RPC Service 60020
Region Server HTTP Service 60030


On 11/05/2012 11:38 PM, yiyu jia wrote:
> After disable iptables service on my CentOS 6.3 machine, HBase master
> starts up. This is strange as I open all ports number i can find in cfg
> file and log files already.
>
> anybody has a list of port number, which should be opened through firewall?
>
>
>
> On Mon, Nov 5, 2012 at 11:16 PM, yiyu jia <jia.yiyu@gmail.com> wrote:
>
>> Hi,
>>
>> I try to start HBase in a distribute model. But, I am failure with the
>> following error mssae. Anybody here has hint to solve this issue? thanks in
>> advance!
>>
>> Yiyu
>>
>>
>>
>>
>> 2012-11-05 23:10:21,217 ERROR
>> org.apache.hadoop.hbase.master.HMasterCommandLine: Failed to start master
>> java.lang.RuntimeException: HMaster Aborted
>>   at
>> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:154)
>> at
>> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:103)
>>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> at
>> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
>>   at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1806)
>> 2012-11-05 23:10:21,221 ERROR org.apache.hadoop.hdfs.DFSClient: Exception
>> closing file /hbase-hadoopuser/hbase/hbase.version :
>> org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>> /hbase-hadoopuser/hbase/hbase.version could only be replicated to 0 nodes,
>> instead of 1
>>   at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
>>   at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>   at java.lang.reflect.Method.invoke(Method.java:616)
>> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
>>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
>>   at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:416)
>>   at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>>
>>
>>
>> My hbase-site.xml :
>>
>> <?xml version="1.0"?>
>> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>> <!--
>> /**
>>   * Licensed to the Apache Software Foundation (ASF) under one
>>   * or more contributor license agreements.  See the NOTICE file
>>   * distributed with this work for additional information
>>   * regarding copyright ownership.  The ASF licenses this file
>>   * to you under the Apache License, Version 2.0 (the
>>   * "License"); you may not use this file except in compliancef
>>   * with the License.  You may obtain a copy of the License at
>>   *
>>   *     http://www.apache.org/licenses/LICENSE-2.0
>>   *
>>   * Unless required by applicable law or agreed to in writing, software
>>   * distributed under the License is distributed on an "AS IS" BASIS,
>>   * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>   * See the License for the specific language governing permissions and
>>   * limitations under the License.
>>   */
>> -->
>> <configuration>
>>    <property>
>>      <name>hbase.rootdir</name>
>>      <value>hdfs://cent63V1.corp.ybusa.net:9000/hbase-${user.name}/hbase<http://cent63V1.corp.ybusa.net:9000/hbase-$%7Buser.name%7D/hbase>
>> </value>
>>      <description>The directory shared by region servers and into
>>      which HBase persists.  The URL should be 'fully-qualified'
>>      to include the filesystem scheme.  For example, to specify the
>>      HDFS directory '/hbase' where the HDFS instance's namenode is
>>      running at namenode.example.org on port 9000, set this value to:
>>      hdfs://namenode.example.org:9000/hbase.  By default HBase writes
>>      into /tmp.  Change this configuration else all data will be lost
>>      on machine restart.
>>      </description>
>>    </property>
>>    <property>
>>      <name>hbase.master.port</name>
>>      <value>60000</value>
>>      <description>The port the HBase Master should bind to.</description>
>>    </property>
>>    <property>
>>      <name>hbase.cluster.distributed</name>
>>      <value>true</value>
>>      <description>The mode the cluster will be in. Possible values are
>>        false for standalone mode and true for distributed mode.  If
>>        false, startup will run all HBase and ZooKeeper daemons together
>>        in the one JVM.
>>      </description>
>>    </property>
>>     <property>
>>      <name>hbase.master.info.port</name>
>>      <value>60010</value>
>>      <description>The port for the HBase Master web UI.
>>      Set to -1 if you do not want a UI instance run.
>>      </description>
>>    </property>
>>    <property>
>>      <name>hbase.master.info.bindAddress</name>
>>      <value>0.0.0.0</value>
>>      <description>The bind address for the HBase Master web UI
>>      </description>
>>    </property>
>>    <property>
>>      <name>hbase.client.write.buffer</name>
>>      <value>2097152</value>
>>      <description>Default size of the HTable clien write buffer in bytes.
>>      A bigger buffer takes more memory -- on both the client and server
>>      side since server instantiates the passed write buffer to process
>>      it -- but a larger buffer size reduces the number of RPCs made.
>>      For an estimate of server-side memory-used, evaluate
>>      hbase.client.write.buffer * hbase.regionserver.handler.count
>>      </description>
>>    </property>
>>    <property>
>>      <name>hbase.regionserver.port</name>
>>      <value>60020</value>
>>      <description>The port the HBase RegionServer binds to.
>>      </description>
>>    </property>
>>    <property>
>>      <name>hbase.regionserver.info.port</name>
>>      <value>60030</value>
>>      <description>The port for the HBase RegionServer web UI
>>      Set to -1 if you do not want the RegionServer UI to run.
>>      </description>
>>    </property>
>>      <property>
>>      <name>hbase.regionserver.info.bindAddress</name>
>>      <value>0.0.0.0</value>
>>      <description>The address for the HBase RegionServer web UI
>>      </description>
>>    </property>
>>
>>    <property>
>>      <name>hbase.zookeeper.dns.nameserver</name>
>>      <value>10.0.40.1</value>
>>      <description>The host name or IP address of the name server (DNS)
>>        which a ZooKeeper server should use to determine the host name used
>> by the
>>        master for communication and display purposes.
>>      </description>
>>    </property>
>>    <property>
>>      <name>hbase.regionserver.dns.nameserver</name>
>>      <value>10.0.40.1</value>
>>      <description>The host name or IP address of the name server (DNS)
>>        which a region server should use to determine the host name used by
>> the
>>        master for communication and display purposes.
>>      </description>
>>    </property>
>>    <property>
>>      <name>hbase.master.dns.nameserver</name>
>>      <value>10.0.40.1</value>
>>      <description>The host name or IP address of the name server (DNS)
>>        which a master should use to determine the host name used
>>        for communication and display purposes.
>>      </description>
>>    </property>
>>
>>    <property>
>>      <name>zookeeper.session.timeout</name>
>>      <value>180000</value>
>>      <description>ZooKeeper session timeout.
>>        HBase passes this to the zk quorum as suggested maximum time for a
>>        session (This setting becomes zookeeper's 'maxSessionTimeout').  See
>>
>> http://hadoop.apache.org/zookeeper/docs/current/zookeeperProgrammers.html#ch_zkSessions
>>        "The client sends a requested timeout, the server responds with the
>>        timeout that it can give the client. " In milliseconds.
>>      </description>
>>    </property>
>>
>>
>>    <!--
>>    The following three properties are used together to create the list of
>>    host:peer_port:leader_port quorum servers for ZooKeeper.
>> 172.23.34.12,172.23.34.80,
>>    -->
>>    <property>
>>      <name>hbase.zookeeper.quorum</name>
>>      <value>172.23.34.12,172.23.34.80,172.23.34.85</value>
>>      <description>Comma separated list of servers in the ZooKeeper Quorum.
>>      For example, "host1.mydomain.com,host2.mydomain.com,host3.mydomain.com
>> ".
>>      By default this is set to localhost for local and pseudo-distributed
>> modes
>>      of operation. For a fully-distributed setup, this should be set to a
>> full
>>      list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in
>> hbase-env.sh
>>      this is the list of servers which we will start/stop ZooKeeper on.
>>      </description>
>>    </property>
>>    <property>
>>      <name>hbase.zookeeper.peerport</name>
>>      <value>2888</value>
>>      <description>Port used by ZooKeeper peers to talk to each other.
>>      See
>> http://hadoop.apache.org/zookeeper/docs/r3.1.1/zookeeperStarted.html#sc_RunningReplicatedZooKeeper
>>      for more information.
>>      </description>
>>    </property>
>>    <property>
>>      <name>hbase.zookeeper.leaderport</name>
>>      <value>3888</value>
>>      <description>Port used by ZooKeeper for leader election.
>>      See
>> http://hadoop.apache.org/zookeeper/docs/r3.1.1/zookeeperStarted.html#sc_RunningReplicatedZooKeeper
>>      for more information.
>>      </description>
>>    </property>
>>    <!-- End of properties used to generate ZooKeeper host:port quorum list.
>> -->
>>
>>    <!--
>>    Beginning of properties that are directly mapped from ZooKeeper's
>> zoo.cfg.
>>    All properties with an "hbase.zookeeper.property." prefix are converted
>> for
>>    ZooKeeper's configuration. Hence, if you want to add an option from
>> zoo.cfg,
>>    e.g.  "initLimit=10" you would append the following to your
>> configuration:
>>      <property>
>>        <name>hbase.zookeeper.property.initLimit</name>
>>        <value>10</value>
>>      </property>
>>    -->
>>    <property>
>>      <name>hbase.zookeeper.property.initLimit</name>
>>      <value>10</value>
>>      <description>Property from ZooKeeper's config zoo.cfg.
>>      The number of ticks that the initial synchronization phase can take.
>>      </description>
>>    </property>
>>    <property>
>>      <name>hbase.zookeeper.property.syncLimit</name>
>>      <value>5</value>
>>      <description>Property from ZooKeeper's config zoo.cfg.
>>      The number of ticks that can pass between sending a request and
>> getting an
>>      acknowledgment.
>>      </description>
>>    </property>
>>    <property>
>>      <name>hbase.zookeeper.property.dataDir</name>
>>      <value>${hbase.tmp.dir}/zookeeper</value>
>>      <description>Property from ZooKeeper's config zoo.cfg.
>>      The directory where the snapshot is stored.
>> This should be set to other directory that will not be erased when the
>> system is restarted.
>>      </description>
>>    </property>
>>    <property>
>>      <name>hbase.zookeeper.property.clientPort</name>
>>      <value>2181</value>
>>      <description>Property from ZooKeeper's config zoo.cfg.
>>      The port at which the clients will connect.
>>      </description>
>>    </property>
>>    <property>
>>      <name>hbase.zookeeper.property.maxClientCnxns</name>
>>      <value>300</value>
>>      <description>Property from ZooKeeper's config zoo.cfg.
>>      Limit on number of concurrent connections (at the socket level) that a
>>      single client, identified by IP address, may make to a single member of
>>      the ZooKeeper ensemble. Set high to avoid zk connection issues running
>>      standalone and pseudo-distributed.
>>      </description>
>>    </property>
>>    <!-- End of properties that are directly mapped from ZooKeeper's zoo.cfg
>> -->
>>    <property>
>>      <name>hbase.rest.port</name>
>>      <value>8080</value>
>>      <description>The port for the HBase REST server.</description>
>>    </property>
>>    <property>
>>      <name>hbase.rest.readonly</name>
>>      <value>false</value>
>>      <description>
>>      Defines the mode the REST server will be started in. Possible values
>> are:
>>      false: All HTTP methods are permitted - GET/PUT/POST/DELETE.
>>      true: Only the GET method is permitted.
>>      </description>
>>    </property>
>>
>>    <property skipInDoc="false">
>>      <name>hbase.defaults.for.version</name>
>>      <value>0.94.2</value>
>>      <description>
>>      This defaults file was compiled for version @@@VERSION@@@. This
>> variable is used
>>      to make sure that a user doesn't have an old version of
>> hbase-default.xml on the
>>      classpath.
>>      </description>
>>    </property>
>>    <property>
>>      <name>hbase.defaults.for.version.skip</name>
>>      <value>false</value>
>>      <description>
>>      Set to true to skip the 'hbase.defaults.for.version' check.
>>      Setting this to true can be useful in contexts other than
>>      the other side of a maven generation; i.e. running in an
>>      ide.  You'll want to set this boolean to true to avoid
>>      seeing the RuntimException complaint: "hbase-default.xml file
>>      seems to be for and old version of HBase (@@@VERSION@@@), this
>>      version is X.X.X-SNAPSHOT"
>>      </description>
>>    </property>
>>
>>    <property>
>>      <name>dfs.support.append</name>
>>      <value>true</value>
>>      <description>Does HDFS allow appends to files?
>>      This is an hdfs config. set in here so the hdfs client will do append
>> support.
>>      You must ensure that this config. is true serverside too when running
>> hbase
>>      (You will have to restart your cluster after setting it).
>>      </description>
>>    </property>
>>
>>    <property>
>>      <name>hbase.metrics.showTableName</name>
>>      <value>true</value>
>>      <description>Whether to include the prefix "tbl.tablename" in
>> per-column family metrics.
>> If true, for each metric M, per-cf metrics will be reported for
>> tbl.T.cf.CF.M, if false,
>>   per-cf metrics will be aggregated by column-family across tables, and
>> reported for cf.CF.M.
>> In both cases, the aggregated metric M across tables and cfs will be
>> reported.
>>      </description>
>>    </property>
>>
>>
>> </configuration>
>>
>>
>>
>>
>

-- 

Marcos Luis Ortíz Valmaseda
about.me/marcosortiz <http://about.me/marcosortiz>
@marcosluis2186 <http://twitter.com/marcosluis2186>



10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION

http://www.uci.cu
http://www.facebook.com/universidad.uci
http://www.flickr.com/photos/universidad_uci
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message