hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pankil Doshi <forpan...@gmail.com>
Subject Re: org.apache.hadoop.hbase.exceptions.DeserializationException: Missing pb magic PBUF prefix
Date Fri, 23 Oct 2015 18:19:29 GMT
Hi Ashish,

Yes that is due to hbase-indexer code. Still trying to find out soln.
No I didnt upgrade hbase from 0.94.

I directly installed hbase-0.98.15-hadoop2.

Thanks,
Pankil

On Fri, Oct 23, 2015 at 9:55 AM, Ashish Singhi <
ashish.singhi.apache@gmail.com> wrote:

> Hi Pankil.
>
> A similar issue was reported few days back (
>
> http://search-hadoop.com/m/YGbbknQt52rKBDS1&subj=HRegionServer+failed+due+to+replication
> ).
>
> May be this is due to hbase-indexer code ?
> One more Q, did you upgrade hbase from 0.94 and you see this issue ?
>
> Regards,
> Ashish Singhi
>
> On Fri, Oct 23, 2015 at 2:47 AM, Pankil Doshi <forpankil@gmail.com> wrote:
>
> > Hi,
> >
> > I am using hbase-0.98.15-hadoop2 and hbase-indexer from lily (
> > http://ngdata.github.io/hbase-indexer/).
> >
> > I am seeing below error when I add my indexer:
> >
> >
> > 2015-10-22 14:08:27,468 INFO  [regionserver60020-EventThread]
> > replication.ReplicationTrackerZKImpl: /hbase/replication/peers znode
> > expired, triggering peerListChanged event
> >
> > 2015-10-22 14:08:27,473 ERROR [regionserver60020-EventThread]
> > regionserver.ReplicationSourceManager: Error while adding a new peer
> >
> > org.apache.hadoop.hbase.replication.ReplicationException: Error adding
> peer
> > with id=Indexer_newtest2
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:386)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.peerAdded(ReplicationPeersZKImpl.java:358)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.peerListChanged(ReplicationSourceManager.java:514)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationTrackerZKImpl$PeersWatcher.nodeChildrenChanged(ReplicationTrackerZKImpl.java:189)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:468)
> >
> > at
> >
> >
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
> >
> > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
> >
> > Caused by: org.apache.hadoop.hbase.replication.ReplicationException:
> Error
> > starting the peer state tracker for peerId=Indexer_newtest2
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:454)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:384)
> >
> > ... 6 more
> >
> > Caused by:
> org.apache.zookeeper.KeeperException$DataInconsistencyException:
> > KeeperErrorCode = DataInconsistency
> >
> > at org.apache.hadoop.hbase.zookeeper.ZKUtil.convert(ZKUtil.java:2063)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.startStateTracker(ReplicationPeerZKImpl.java:85)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:452)
> >
> > ... 7 more
> >
> > Caused by: org.apache.hadoop.hbase.exceptions.DeserializationException:
> > Missing pb magic PBUF prefix
> >
> > at
> >
> >
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.expectPBMagicPrefix(ProtobufUtil.java:270)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.parseStateFrom(ReplicationPeerZKImpl.java:243)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.isStateEnabled(ReplicationPeerZKImpl.java:232)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.readPeerStateZnode(ReplicationPeerZKImpl.java:90)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.startStateTracker(ReplicationPeerZKImpl.java:83)
> >
> > ... 8 more
> >
> >
> >
> > My Hbase-site.xml:
> >
> >
> > <?xml version="1.0"?>
> >
> > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> >
> > <!--
> >
> > /**
> >
> >  * Copyright 2010 The Apache Software Foundation
> >
> >  *
> >
> >  * Licensed to the Apache Software Foundation (ASF) under one
> >
> >  * or more contributor license agreements.  See the NOTICE file
> >
> >  * distributed with this work for additional information
> >
> >  * regarding copyright ownership.  The ASF licenses this file
> >
> >  * to you under the Apache License, Version 2.0 (the
> >
> >  * "License"); you may not use this file except in compliance
> >
> >  * with the License.  You may obtain a copy of the License at
> >
> >  *
> >
> >  *     http://www.apache.org/licenses/LICENSE-2.0
> >
> >  *
> >
> >  * Unless required by applicable law or agreed to in writing, software
> >
> >  * distributed under the License is distributed on an "AS IS" BASIS,
> >
> >  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> >
> >  * See the License for the specific language governing permissions and
> >
> >  * limitations under the License.
> >
> >  */
> >
> > -->
> >
> > <configuration>
> >
> > <!--<property>
> >
> >                 <name>zookeeper.znode.parent</name>
> >
> >                 <value>/hbase-unsecure</value>
> >
> >         </property>-->
> >
> >
> >         <property>
> >
> >                 <name>hbase.cluster.distributed</name>
> >
> >                 <value>true</value>
> >
> >         </property>
> >
> > //Here you have to set the path where you want HBase to store its files.
> >
> >    <property>
> >
> >       <name>hbase.rootdir</name>
> >
> >       <value>file:/tmp/HBase/HFiles</value>
> >
> >    </property>
> >
> > <property>
> >
> >       <name>hbase.zookeeper.property.clientPort</name>
> >
> >       <value>2181</value>
> >
> >       <description>Property from ZooKeeper's config zoo.cfg.
> >
> >       The port at which the clients will connect.
> >
> >       </description>
> >
> >     </property>
> >
> >     <property>
> >
> >       <name>hbase.zookeeper.quorum</name>
> >
> >       <value>localhost</value>
> >
> >       <description>Comma separated list of servers in the ZooKeeper
> Quorum.
> >
> >       For example, "host1.mydomain.com,host2.mydomain.com,
> > host3.mydomain.com
> > ".
> >
> >       By default this is set to localhost for local and
> pseudo-distributed
> > modes
> >
> >       of operation. For a fully-distributed setup, this should be set to
> a
> > full
> >
> >       list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in
> > hbase-env.sh
> >
> >       this is the list of servers which we will start/stop ZooKeeper on.
> >
> >       </description>
> >
> >     </property>
> >
> > <property>
> >
> >    <name>hbase.zookeeper.property.dataDir</name>
> >
> >    <value>/tmp/zookeeper</value>
> >
> >    <description>Property from ZooKeeper config zoo.cfg.
> >
> >    The directory where the snapshot is stored.
> >
> >    </description>
> >
> > </property>
> >
> > // FOR HBASE INDEXER
> >
> >
> > <!-- SEP is basically replication, so enable it -->
> >
> >   <property>
> >
> >     <name>hbase.replication</name>
> >
> >     <value>true</value>
> >
> >   </property>
> >
> >   <!-- Source ratio of 100% makes sure that each SEP consumer is actually
> >
> >        used (otherwise, some can sit idle, especially with small
> clusters)
> > -->
> >
> >   <property>
> >
> >     <name>replication.source.ratio</name>
> >
> >     <value>1.0</value>
> >
> >   </property>
> >
> >   <!-- Maximum number of hlog entries to replicate in one go. If this is
> >
> >        large, and a consumer takes a while to process the events, the
> >
> >        HBase rpc call will time out. -->
> >
> >   <property>
> >
> >     <name>replication.source.nb.capacity</name>
> >
> >     <value>1000</value>
> >
> >   </property>
> >
> >   <!-- A custom replication source that fixes a few things and adds
> >
> >        some functionality (doesn't interfere with normal replication
> >
> >        usage). -->
> >
> >   <property>
> >
> >     <name>replication.replicationsource.implementation</name>
> >
> >     <value>com.ngdata.sep.impl.SepReplicationSource</value>
> >
> >   </property>
> >
> > </configuration>
> >
> >
> > All research so far shows me one can face problem while upgrading hbase.
> > But in my case, I am not upgrading.
> >
> > Any ideas/suggestions will be very helpful.
> >
> > Thanks,
> > Pankil
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message