hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashish Singhi <ashish.singhi.apa...@gmail.com>
Subject Re: org.apache.hadoop.hbase.exceptions.DeserializationException: Missing pb magic PBUF prefix
Date Fri, 23 Oct 2015 16:55:58 GMT
Hi Pankil.

A similar issue was reported few days back (
http://search-hadoop.com/m/YGbbknQt52rKBDS1&subj=HRegionServer+failed+due+to+replication
).

May be this is due to hbase-indexer code ?
One more Q, did you upgrade hbase from 0.94 and you see this issue ?

Regards,
Ashish Singhi

On Fri, Oct 23, 2015 at 2:47 AM, Pankil Doshi <forpankil@gmail.com> wrote:

> Hi,
>
> I am using hbase-0.98.15-hadoop2 and hbase-indexer from lily (
> http://ngdata.github.io/hbase-indexer/).
>
> I am seeing below error when I add my indexer:
>
>
> 2015-10-22 14:08:27,468 INFO  [regionserver60020-EventThread]
> replication.ReplicationTrackerZKImpl: /hbase/replication/peers znode
> expired, triggering peerListChanged event
>
> 2015-10-22 14:08:27,473 ERROR [regionserver60020-EventThread]
> regionserver.ReplicationSourceManager: Error while adding a new peer
>
> org.apache.hadoop.hbase.replication.ReplicationException: Error adding peer
> with id=Indexer_newtest2
>
> at
>
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:386)
>
> at
>
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.peerAdded(ReplicationPeersZKImpl.java:358)
>
> at
>
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.peerListChanged(ReplicationSourceManager.java:514)
>
> at
>
> org.apache.hadoop.hbase.replication.ReplicationTrackerZKImpl$PeersWatcher.nodeChildrenChanged(ReplicationTrackerZKImpl.java:189)
>
> at
>
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:468)
>
> at
>
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
>
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
>
> Caused by: org.apache.hadoop.hbase.replication.ReplicationException: Error
> starting the peer state tracker for peerId=Indexer_newtest2
>
> at
>
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:454)
>
> at
>
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:384)
>
> ... 6 more
>
> Caused by: org.apache.zookeeper.KeeperException$DataInconsistencyException:
> KeeperErrorCode = DataInconsistency
>
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.convert(ZKUtil.java:2063)
>
> at
>
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.startStateTracker(ReplicationPeerZKImpl.java:85)
>
> at
>
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:452)
>
> ... 7 more
>
> Caused by: org.apache.hadoop.hbase.exceptions.DeserializationException:
> Missing pb magic PBUF prefix
>
> at
>
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.expectPBMagicPrefix(ProtobufUtil.java:270)
>
> at
>
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.parseStateFrom(ReplicationPeerZKImpl.java:243)
>
> at
>
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.isStateEnabled(ReplicationPeerZKImpl.java:232)
>
> at
>
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.readPeerStateZnode(ReplicationPeerZKImpl.java:90)
>
> at
>
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.startStateTracker(ReplicationPeerZKImpl.java:83)
>
> ... 8 more
>
>
>
> My Hbase-site.xml:
>
>
> <?xml version="1.0"?>
>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>
> <!--
>
> /**
>
>  * Copyright 2010 The Apache Software Foundation
>
>  *
>
>  * Licensed to the Apache Software Foundation (ASF) under one
>
>  * or more contributor license agreements.  See the NOTICE file
>
>  * distributed with this work for additional information
>
>  * regarding copyright ownership.  The ASF licenses this file
>
>  * to you under the Apache License, Version 2.0 (the
>
>  * "License"); you may not use this file except in compliance
>
>  * with the License.  You may obtain a copy of the License at
>
>  *
>
>  *     http://www.apache.org/licenses/LICENSE-2.0
>
>  *
>
>  * Unless required by applicable law or agreed to in writing, software
>
>  * distributed under the License is distributed on an "AS IS" BASIS,
>
>  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>
>  * See the License for the specific language governing permissions and
>
>  * limitations under the License.
>
>  */
>
> -->
>
> <configuration>
>
> <!--<property>
>
>                 <name>zookeeper.znode.parent</name>
>
>                 <value>/hbase-unsecure</value>
>
>         </property>-->
>
>
>         <property>
>
>                 <name>hbase.cluster.distributed</name>
>
>                 <value>true</value>
>
>         </property>
>
> //Here you have to set the path where you want HBase to store its files.
>
>    <property>
>
>       <name>hbase.rootdir</name>
>
>       <value>file:/tmp/HBase/HFiles</value>
>
>    </property>
>
> <property>
>
>       <name>hbase.zookeeper.property.clientPort</name>
>
>       <value>2181</value>
>
>       <description>Property from ZooKeeper's config zoo.cfg.
>
>       The port at which the clients will connect.
>
>       </description>
>
>     </property>
>
>     <property>
>
>       <name>hbase.zookeeper.quorum</name>
>
>       <value>localhost</value>
>
>       <description>Comma separated list of servers in the ZooKeeper Quorum.
>
>       For example, "host1.mydomain.com,host2.mydomain.com,
> host3.mydomain.com
> ".
>
>       By default this is set to localhost for local and pseudo-distributed
> modes
>
>       of operation. For a fully-distributed setup, this should be set to a
> full
>
>       list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in
> hbase-env.sh
>
>       this is the list of servers which we will start/stop ZooKeeper on.
>
>       </description>
>
>     </property>
>
> <property>
>
>    <name>hbase.zookeeper.property.dataDir</name>
>
>    <value>/tmp/zookeeper</value>
>
>    <description>Property from ZooKeeper config zoo.cfg.
>
>    The directory where the snapshot is stored.
>
>    </description>
>
> </property>
>
> // FOR HBASE INDEXER
>
>
> <!-- SEP is basically replication, so enable it -->
>
>   <property>
>
>     <name>hbase.replication</name>
>
>     <value>true</value>
>
>   </property>
>
>   <!-- Source ratio of 100% makes sure that each SEP consumer is actually
>
>        used (otherwise, some can sit idle, especially with small clusters)
> -->
>
>   <property>
>
>     <name>replication.source.ratio</name>
>
>     <value>1.0</value>
>
>   </property>
>
>   <!-- Maximum number of hlog entries to replicate in one go. If this is
>
>        large, and a consumer takes a while to process the events, the
>
>        HBase rpc call will time out. -->
>
>   <property>
>
>     <name>replication.source.nb.capacity</name>
>
>     <value>1000</value>
>
>   </property>
>
>   <!-- A custom replication source that fixes a few things and adds
>
>        some functionality (doesn't interfere with normal replication
>
>        usage). -->
>
>   <property>
>
>     <name>replication.replicationsource.implementation</name>
>
>     <value>com.ngdata.sep.impl.SepReplicationSource</value>
>
>   </property>
>
> </configuration>
>
>
> All research so far shows me one can face problem while upgrading hbase.
> But in my case, I am not upgrading.
>
> Any ideas/suggestions will be very helpful.
>
> Thanks,
> Pankil
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message