Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of asaf.mesika@gmail.com
 designates 74.125.83.41 as permitted sender)
Subject: Re: Region is not online: .META.,,1
Mime-Version: 1.0 (Apple Message framework v1283)
Content-Type: multipart/alternative;
 boundary="Apple-Mail=_868E4746-A2EF-424F-B679-B715363C81A4"
From: "Mesika, Asaf" <asaf.mesika@gmail.com>
In-Reply-To: <A93B6056-ECC2-46BF-82C2-B132122ABFFE@hortonworks.com>
Date: Thu, 8 Nov 2012 14:23:36 +0200
Cc: user@hbase.apache.org
Message-Id: <AC05623A-B5C8-469E-A197-AC77B468BC12@gmail.com>
References: <B9AE38F0-EDB8-42A0-A3BF-3BAA247DB89F@gmail.com>
 <4324BC71-7508-4EC4-858A-0E4F5BF34530@gmail.com>
 <A93B6056-ECC2-46BF-82C2-B132122ABFFE@hortonworks.com>
To: Leonid Fedotov <lfedotov@hortonworks.com>

--Apple-Mail=_868E4746-A2EF-424F-B679-B715363C81A4
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

See my 2nd mail regarding this: Didn't help.
We're still heavily investigating this, so far without any progress.

On Nov 7, 2012, at 5:43 PM, Leonid Fedotov wrote:

> The first to try is=20
>=20
> hbase hbck -repair
>=20
> Check here:
> http://hbase.apache.org/book/apbs04.html#d2067e11173
>=20
> Thank you!
>=20
> Sincerely,
> Leonid Fedotov
>=20
>=20
> On Nov 6, 2012, at 10:08 PM, Mesika, Asaf wrote:
>=20
>> After restart, running hbck results in:
>> 12/11/07 06:07:28 DEBUG client.MetaScanner: Scanning .META. starting =
at row=3D for max=3D2147483647 rows using =
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementatio=
n@51b4a74b
>>=20
>> 12/11/07 06:07:28 INFO util.HBaseFsck: getHTableDescriptors =3D=3D =
tableNames =3D> []
>> Exception in thread "main" java.lang.NullPointerException
>> 	at =
org.apache.hadoop.hbase.util.HBaseFsck.reportTablesInFlux(HBaseFsck.java:5=
60)
>> 	at =
org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.j=
ava:346)
>> 	at =
org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:382)
>> 	at =
org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3120)
>>=20
>> HBase is version 0.94.0
>>=20
>>=20
>> On Nov 7, 2012, at 7:28 AM, Mesika, Asaf wrote:
>>=20
>>> Hi,
>>>=20
>>> I have a cluster with 10 RS and 1 Master.
>>> =46rom some reason, any call to the HBase fails. I ran hbck and got =
the same failure:
>>>=20
>>> Tue Nov 06 13:27:07 UTC 2012, =
org.apache.hadoop.hbase.client.ScannerCallable@7d8a8ce2, =
org.apache.hadoop.hbase.NotServingRegionException: =
org.apache.hadoop.hbase.NotServingRegionException: Region is not online: =
.META.,,1
>>>=20
>>> 	at =
org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.j=
ava:183)
>>> 	at =
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:196)
>>> 	at =
org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:48)=

>>> 	at =
org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:126)=

>>> 	at =
org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:123)=

>>> 	at =
org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManag=
er.java:359)
>>> 	at =
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:123)
>>> 	at =
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:99)
>>> 	at =
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:77)
>>> 	at =
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:61)
>>> 	at =
org.apache.hadoop.hbase.util.HBaseFsck.loadMetaEntries(HBaseFsck.java:2234=
)
>>> 	at =
org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.j=
ava:334)
>>> 	at =
org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:382)
>>> 	at =
org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3120)
>>>=20
>>>=20
>>> Anyone has any idea how to fix it without the obvious restart?
>>>=20
>>=20
>=20


--Apple-Mail=_868E4746-A2EF-424F-B679-B715363C81A4--