Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of ahmic.samir@gmail.com
 designates 209.85.223.169 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <5264DD74.3000701@post.km.ru>
References: <5264DD74.3000701@post.km.ru>
Date: Mon, 21 Oct 2013 10:17:17 +0200
Message-ID: 
 <CADKW6+D1Em=BLnYVB8fnSOWtiiNbPjCBR=c80t7vxxj-3SCqZw@mail.gmail.com>
Subject: Re: CDH4.4 and HBASE-8912 issue
From: Samir Ahmic <ahmic.samir@gmail.com>
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=047d7bdca43092bc7804e93be917

--047d7bdca43092bc7804e93be917
Content-Type: text/plain; charset=ISO-8859-1

Hi, Boris

Did you check RS logs ? There should be exception regarding why assignment
failed. Can you past that exception ?

Cheers :)


On Mon, Oct 21, 2013 at 9:53 AM, Boris Emelyanov <emelyanov@post.km.ru>wrote:

>  >Boris, what does hbck say?
> >
> >We have had this issue a couple times before. To fix it I had to stop the cluster, run offline meta repair tool,
> >delete zk-store on each zk quorum node
> >Offline Meta repair tool will not work if there are  inconsistencies  in HBase - you better try hbase hbck
> >-fixAll first.
> >
> >Best regards,
> >Vladimir Rodionov
> >Principal Platform Engineer
> >Carrier IQ, www.carrieriq.com
>
> >e-mail: vrodionov@... <http://gmane.org/get-address.php?address=vrodionov%2dSvj7bELwklqcm8Fc2pXOzQ%40public.gmane.org>
>
> Hbck says "0 inconsistencies detected".
> I stopped hbase cluster, deleted zk-database on all quorum nodes, ran "hbase org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair",
> and got "INFO util.HBaseFsck: Success! .META. table rebuilt.".
> After that, cluster continued crashing during auto-loadbalancing.
>
>
>
> --
> Best regards,
>
> Boris Emelyanov.
>
>

--047d7bdca43092bc7804e93be917
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi, Boris<div><br></div><div>Did you check RS logs ? There=
 should be exception regarding why assignment failed. Can you past that exc=
eption ?</div><div><br></div><div>Cheers :)=A0</div></div><div class=3D"gma=
il_extra">
<br><br><div class=3D"gmail_quote">On Mon, Oct 21, 2013 at 9:53 AM, Boris E=
melyanov <span dir=3D"ltr">&lt;<a href=3D"mailto:emelyanov@post.km.ru" targ=
et=3D"_blank">emelyanov@post.km.ru</a>&gt;</span> wrote:<br><blockquote cla=
ss=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;pa=
dding-left:1ex">

 =20
   =20
 =20
  <div bgcolor=3D"#FFFFFF" text=3D"#000000">
   =20
    <pre style=3D"color:rgb(34,34,34);font-style:normal;font-variant:normal=
;font-weight:normal;letter-spacing:normal;line-height:24px;text-align:justi=
fy;text-indent:0px;text-transform:none;word-spacing:0px"><div class=3D"im">
&gt;Boris, what does hbck say?
&gt;
&gt;We have had this issue a couple times before. To fix it I had to stop t=
he cluster, run offline meta repair tool,
&gt;delete zk-store on each zk quorum node
&gt;Offline Meta repair tool will not work if there are  inconsistencies  i=
n HBase - you better try hbase hbck
&gt;-fixAll first.
&gt;
&gt;Best regards,
&gt;Vladimir Rodionov
&gt;Principal Platform Engineer
&gt;Carrier IQ, <a rel=3D"nofollow" href=3D"http://www.carrieriq.com/" styl=
e=3D"color:rgb(0,51,102);font-weight:bold;text-decoration:none" target=3D"_=
blank">www.carrieriq.com</a></div>
&gt;e-mail: <a href=3D"http://gmane.org/get-address.php?address=3Dvrodionov=
%2dSvj7bELwklqcm8Fc2pXOzQ%40public.gmane.org" rel=3D"nofollow" style=3D"col=
or:rgb(0,51,102);font-weight:bold;text-decoration:none" target=3D"_blank">v=
rodionov@...</a>

Hbck says &quot;0 inconsistencies detected&quot;.=20
I stopped hbase cluster, deleted zk-database on all quorum nodes, ran &quot=
;hbase org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair&quot;,
and got &quot;INFO util.HBaseFsck: Success! .META. table rebuilt.&quot;.
After that, cluster continued crashing during auto-loadbalancing.


</pre><div class=3D"im">
    <pre cols=3D"72">--=20
Best regards,

Boris Emelyanov.</pre>
  </div></div>

</blockquote></div><br></div>

--047d7bdca43092bc7804e93be917--