hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Marc Spaggiari <jean-m...@spaggiari.org>
Subject Re: copyTable from 0.94 to 0.96?
Date Fri, 15 Nov 2013 01:47:19 GMT
I have added some logs on the TableOutputFormat class and it seems to
receive the right parameters. However, it's still not able to connect to
the other cluster.

13/11/14 20:38:42  mapreduce.TableOutputFormat:
address=hbasetest1.distparser.com:2181:/hbase
13/11/14 20:38:42  mapreduce.TableOutputFormat: zkClientPort=0
13/11/14 20:38:42  mapreduce.TableOutputFormat: serverClass=null
13/11/14 20:38:42  mapreduce.TableOutputFormat: serverImpl=null

Basically, it's trying to connect from a 0.94.12 client to a 0.96.0
cluster. 0.96.0 talk PB. But I guess 0.94.12 not... Can this be why it's
not working? Has anyone been able to copytable from 0.94 to 0.96?


2013/11/14 Jean-Marc Spaggiari <jean-marc@spaggiari.org>

> Thanks for looking a it St.Ack.
>
> Here are my regions on the source table:
> Table RegionsNameRegion ServerStart Key End Key Requests
> dns,,1379202070789.bb65f685cdefc4f2491d246f376fc1f0. node3:60030
> gestion-v.com 0  dns,gestion-v.com
> ,1379202070789.d02ce8e3fa1a200c7f034b349acf8cc8. buldo:60030 gestion-v.com
> rfkcenter.org 0  dns,rfkcenter.org
> ,1379202071281.c6569a72cc3c2750d14976ab85f02315. node5:60030 rfkcenter.org
> theafronews.ca 0  dns,theafronews.ca
> ,1379202071281.d5b74aaa2853b00b0ad0f20f60c74398. node7:60030
> theafronews.ca
> 0
> Sound like "normal".
>
> I tried to scan the source table from the shell and it's working fine.
>
>
> Now, since the stack show TableOutputFormat does it mean it's on the
> destination cluster? If I put a wrong server name or port, I get a
> connection exception, which mean server name is right.
>
> So on the target side, I can also scan the table:
>
> hbase(main):001:0> scan 'dns'
> 2013-11-14 07:58:20,597 WARN  [main] util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> ROW
> COLUMN+CELL
>
> 0 row(s) in 0.0620 seconds
>
> hbase(main):002:0>
>
>
> Also put is working fine:
> hbase(main):003:0> put 'dns', 'test', '@:@', 'TestValue'
> 0 row(s) in 0.2090 seconds
>
> hbase(main):004:0> scan 'dns'
> ROW
> COLUMN+CELL
>
>  test                                         column=@:@,
> timestamp=1384433996320,
> value=TestValue
>
> 1 row(s) in 0.0570 seconds
>
> hbase(main):005:0>
>
>
> I have 3 ZK servers on the source and one on the target.
>
> ZK shell is working fine on target cluster. However, there is this:
> [zk: localhost:2181(CONNECTED) 6] ls /hbase/master
> []
>
> It's the same on 0.94. The master ZNode is empty. Is that normal? Can this
> be the issue?
>
> JM
>
>
> 2013/11/14 Stack <stack@duboce.net>
>
>> On Thu, Nov 14, 2013 at 9:23 PM, Jean-Marc Spaggiari <
>> jean-marc@spaggiari.org> wrote:
>>
>> > Hum. I let is run over night and got that:
>> > 13/11/13 22:24:17 INFO zookeeper.ClientCnxn: Session establishment
>> complete
>> > on server hbasetest1/192.168.23.51:2181, sessionid = 0x1423ef50f7d0241,
>> > negotiated timeout = 40000
>> > 13/11/13 23:24:41 ERROR mapreduce.TableOutputFormat:
>> > org.apache.hadoop.hbase.client.NoServerForRegionException: Unable to
>> find
>> > region for dns,,99999999999999 after 14 tries.
>> >
>>
>> dns,,99999999999999 is an artificial key made in the client doing get
>> closest or before; it is how we find start key for a region (the first
>> region in the table in this case).  It should not be coming up and out as
>> an exception.  Was there no region whose key was '',,' in the dns table?
>>
>> St.Ack
>>
>>
>> > 13/11/13 23:24:41 INFO mapred.JobClient: Cleaning up the staging area
>> >
>> >
>> file:/tmp/hadoop-hbase/mapred/staging/hbase-368698754/.staging/job_local_0001
>> > Exception in thread "main" java.lang.RuntimeException:
>> > org.apache.hadoop.hbase.client.NoServerForRegionException: Unable to
>> find
>> > region for dns,,99999999999999 after 14 tries.
>> >     at
>> >
>> >
>> org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:206)
>> >     at
>> > org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
>> >     at
>> >
>> >
>> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
>> >     at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:884)
>> >     at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
>> >     at java.security.AccessController.doPrivileged(Native Method)
>> >     at javax.security.auth.Subject.doAs(Subject.java:415)
>> >     at
>> >
>> >
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>> >     at
>> > org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
>> >     at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
>> >     at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
>> >     at
>> org.apache.hadoop.hbase.mapreduce.CopyTable.run(CopyTable.java:274)
>> >     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> >     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>> >     at
>> org.apache.hadoop.hbase.mapreduce.CopyTable.main(CopyTable.java:265)
>> > Caused by: org.apache.hadoop.hbase.client.NoServerForRegionException:
>> > Unable to find region for dns,,99999999999999 after 14 tries.
>> >     at
>> >
>> >
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1095)
>> >     at
>> >
>> >
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1000)
>> >     at
>> >
>> >
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1102)
>> >     at
>> >
>> >
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1004)
>> >     at
>> >
>> >
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:961)
>> >     at
>> org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:227)
>> >     at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:170)
>> >     at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:129)
>> >     at
>> >
>> >
>> org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:201)
>> >     ... 14 more
>> >
>> >
>> > Looking at the destination server I can see that my table and regions
>> are
>> > online nut not with the same name:
>> > dns,,1384397901249.cfcf94a7ec8186baf462b2627d138481
>> >
>> > I don't have anywhere (nor source, neither destination) a region called
>> > dns,,99999999999999.
>> >
>> > Strange.
>> >
>> > JM
>> >
>> >
>> > 2013/11/13 Jean-Marc Spaggiari <jean-marc@spaggiari.org>
>> >
>> > > Hi,
>> > >
>> > > Is there anything I missed for my copyTable command?
>> > >
>> > > hbase@node3:~/hbase-0.94.12$ bin/hbase
>> > > org.apache.hadoop.hbase.mapreduce.CopyTable --endtime=1384395265000
>> > > --peer.adr=hbasetest1:2181:/hbase dns
>> > > 13/11/13 21:28:37 INFO zookeeper.RecoverableZooKeeper: The identifier
>> of
>> > > this process is 31738@node3
>> > >
>> > > I killed it after 30 minutes. There was nothing done. I have my 2
>> > clusters
>> > > running. DistCP between the 2 is working fine, and my 2 HBase are also
>> > > working fine. If I stop HBase on the target, distcp, upgrade and
>> start, I
>> > > have all my data. I dropped all the tables and tried CopyTable and I
>> see
>> > > nothing going.
>> > >
>> > > I tried with the short name (hbasetest1) or with FQDN, same result. I
>> > > tried to run rowcount on the target cluster on an existing table, it's
>> > > working fine.
>> > >
>> > > Any idea what I might be missing? I see nothing on the masters logs
>> (nor
>> > > in the source, neither in the target).
>> > >
>> > > Thanks,
>> > >
>> > > JM
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message