Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of saint.ack@gmail.com designates
 209.85.213.41 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=references:in-reply-to:mime-version:content-transfer-encoding
         :content-type:message-id:cc:x-mailer:from:subject:date:to;
        b=XYm7CNQ/91Dd68sRfI1jzNqYt1vAmSupBgd/X27DIy/WwAtcjZRckW51J2RcdzEXg2
         5nZdmg53zbxa4pcOekZznPuIPpLHNyq83MIe7Bf11IdnuvaPF4KL1vamsC2Ag25XkoN3
         Gs+WlVSmuxybTuxiMbxJixO4DptEqFhaB2H1M=
References: <94F6FBD2-D8F0-4644-8807-E40B84998440@troove.net>
In-Reply-To: <94F6FBD2-D8F0-4644-8807-E40B84998440@troove.net>
Mime-Version: 1.0 (iPhone Mail 8B117)
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii
Message-Id: <0F9D2315-33C2-44F5-994A-D43D03B380D6@gmail.com>
Cc: "dev@hbase.apache.org" <dev@hbase.apache.org>
From: Stack <saint.ack@gmail.com>
Subject: Re: How to handle data migration?
Date: Fri, 14 Jan 2011 09:54:20 -0800
To: "dev@hbase.apache.org" <dev@hbase.apache.org>

It does seem like a regression.   If u kill the zk data dir and restart the c=
luster does it work? (root location is up in zk)


Stack


On Jan 13, 2011, at 11:37, James Kennedy <james.kennedy@troove.net> wrote:

> I'm currently validating the new 0.90.0 RC3 with the hbase-trx layer and o=
ur own application.
>=20
> All seems well so far except for the fact that I now find that HBase doesn=
't adapt if I try to run the same data on different machines.
>=20
> e.g.
> 1) I work from home and generated our seeded test data.
> 2) Run the test suite and all tests pass
> 3) I go to the office and re-run the tests.
>=20
> Result: HMaster fails because the .ROOT data has the wrong ip address for l=
ocating the .META. At least that is my understanding from the stacktrace bel=
ow.  Note that the 192.168.1.102 IP address in that trace is the IP from my h=
ome network and is incorrect.
>=20
> This wasn't an issue with previous versions of HBase as far as I've notice=
d.  And this seems to be a big data portability fail.
> Surely the HMaster should be able to absorb stale metadata and wait for ne=
w region-servers to check in.
> Instead it just keels over and dies.
> But before logging a case I wanted to know if there was something I'm obvi=
ously missing or doing wrong.
>=20
> The seeded test data is on HDFS.
>=20
> Thoughts?
>=20
>=20
> [13/01/11 10:58:42] 5939   [           main] INFO  ion.service.HBaseRegion=
Service  - troove> Starting region server thread.
> [13/01/11 11:00:15] 98699  [        HMaster] FATAL he.hadoop.hbase.master.=
HMaster  - Unhandled exception. Starting shutdown.
> java.net.SocketTimeoutException: 20000 millis timeout while waiting for ch=
annel to be ready for connect. ch : java.nio.channels.SocketChannel[connecti=
on-pending remote=3D192.168.1.102/192.168.1.102:60020]
>    at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeou=
t.java:213)
>    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:404)
>    at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HB=
aseClient.java:311)
>    at org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.ja=
va:865)
>    at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:732)
>    at org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:25=
8)
>    at $Proxy15.getProtocolVersion(Unknown Source)
>    at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419)
>    at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393)
>    at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444)
>    at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:349)=

>    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImpleme=
ntation.getHRegionConnection(HConnectionManager.java:954)
>    at org.apache.hadoop.hbase.catalog.CatalogTracker.getCachedConnection(C=
atalogTracker.java:384)
>    at org.apache.hadoop.hbase.catalog.CatalogTracker.getMetaServerConnecti=
on(CatalogTracker.java:283)
>    at org.apache.hadoop.hbase.catalog.CatalogTracker.verifyMetaRegionLocat=
ion(CatalogTracker.java:478)
>    at org.apache.hadoop.hbase.master.HMaster.assignRootAndMeta(HMaster.jav=
a:435)
>    at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.=
java:382)
>    at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:277)
>    at java.lang.Thread.run(Thread.java:680)
>=20
>=20
> James Kennedy
> Troove Inc.
>=20
>=20