hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Ryaboy <dvrya...@gmail.com>
Subject Re: What is the recommended way to get pig 0.8 to talk with CDH3u0 HBase
Date Sun, 24 Apr 2011 20:04:29 GMT
I suspect the problem here is that you don't have your hbase config
directory on Pig's classpath. You can add it by modifying the PIG_CLASSPATH
environment variable.

D

On Sun, Apr 24, 2011 at 12:41 PM, sulabh choudhury <sulabhc@gmail.com>wrote:

> Have you tried loading like :-
>
> *my_data = LOAD 'hbase://test' using
> org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf:a', '-caching 100') as
> *(a:bytearray)*;dump my_data;*
> or
> *my_data = LOAD 'hbase://test' using
> org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf:a', '-caching 100') as
> (a);dump my_data;*
>
> I am just trying to see if it works if you supply a different datatype ,
> bytearray instead of chararray, or without any datatype at all.
>
> On Sun, Apr 24, 2011 at 10:40 AM, byambajargal <byambaa.0111@gmail.com
> >wrote:
>
> > Hello guys
> >
> > I am running cloudere distribution cdh3u0 on my cluster and i am trying
> to
> > connect pig with Hbase. I have 11 nodes on my cluster so i have
> configured
> > one machine as HBaseMaster and rest are Hbase Region Servers.
> > On my Master manshine:
> >
> >    1. Output of the command $*sudo jps is :*
> >
> >     6907 HMaster
> >   14108 Jps
> >   31470 QuorumPeerMain
> >   22240 JobTracker
> >   21602 SecondaryNameNode
> >   21155 NameNode
> >
> >    2. Output of the command*$status 'detailed'*
> >
> >  -bash-3.2$ hbase shell
> > HBase Shell; enter 'help<RETURN>' for list of supported commands.
> > Type "exit<RETURN>" to leave the HBase Shell
> > Version 0.90.1-cdh3u0, r, Fri Mar 25 16:10:51 PDT 2011
> >
> > hbase(main):001:0> *status 'detailed'*
> > version 0.90.1-cdh3u0
> > 0 regionsInTransition
> > 10 live servers
> >    haisen2.ux.uis.no:60020 1303287066114
> >        requests=0, regions=1, usedHeap=34, maxHeap=987
> >        -ROOT-,,0
> >            stores=1, storefiles=1, storefileSizeMB=0, memstoreSizeMB=0,
> > storefi
> >         leIndexSizeMB=0
> >    haisen10.ux.uis.no:60020 1303287733515
> >        requests=0, regions=1, usedHeap=35, maxHeap=987
> >        task1,,1303326469811.418e9f137f5b1d33604419ef6a8acb6e.
> >            stores=1, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
> > storefi
> >         leIndexSizeMB=0
> >    haisen4.ux.uis.no:60020 1303287334273
> >        requests=0, regions=0, usedHeap=33, maxHeap=987
> >    haisen1.ux.uis.no:60020 1303643944066
> >        requests=0, regions=0, usedHeap=35, maxHeap=987
> >    haisen7.ux.uis.no:60020 1303287560071
> >        requests=0, regions=1, usedHeap=32, maxHeap=987
> >        test,,1303644907708.63ec7397e09d3335c8471aa90bcc5549.
> >            stores=1, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
> > storefi
> >         leIndexSizeMB=0
> >    haisen9.ux.uis.no:60020 1303287672340
> >        requests=0, regions=1, usedHeap=24, maxHeap=987
> >        .META.,,1
> >            stores=1, storefiles=1, storefileSizeMB=0, memstoreSizeMB=0,
> > storefi
> >         leIndexSizeMB=0
> >    haisen5.ux.uis.no:60020 1303287407405
> >        requests=0, regions=0, usedHeap=28, maxHeap=987
> >    haisen6.ux.uis.no:60020 1303287480842
> >        requests=0, regions=0, usedHeap=23, maxHeap=987
> >    haisen3.ux.uis.no:60020 1303287189985
> >        requests=0, regions=0, usedHeap=27, maxHeap=987
> >    haisen8.ux.uis.no:60020 1303287626373
> >        requests=0, regions=0, usedHeap=30, maxHeap=987
> > 0 dead servers
> >
> >
> > I can put data into HBase and pull it out just fine by hbase shell
> command
> > and i have a problem when i load data from hbase by using the following
> Pig
> > query:
> >
> > *my_data = LOAD 'hbase://test' using
> > org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf:a', '-caching 100')
> as
> > (a:chararray);dump my_data;*
> >
> > i got a error message and my job was failure. If someone could help me to
> > connect hbase with pig that would be great.
> >
> >
> > the problem I am having now is the following exception on the pig log
> file:
> >
> > Backend error message
> > ---------------------
> > org.apache.hadoop.hbase.ZooKeeperConnectionException:
> > org.apache.hadoop.hbase.ZooKeeperConnectionException:
> > org.apache.zookeeper.KeeperException$ConnectionLossException:
> > KeeperErrorCode = ConnectionLoss for /hbase
> >        at
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getZooKeeperWatcher(HConnectionManager.java:991)
> >        at
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.setupZookeeperTrackers(HConnectionManager.java:302)
> >        at
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:293)
> >        at
> >
> org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:156)
> >        at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:167)
> >        at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:145)
> >        at
> >
> org.apache.pig.backend.hadoop.hbase.HBaseStorage.setLocation(HBaseStorage.java:302)
> >        at
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.mergeSplitSpecificConf(PigInputFormat.java:133)
> >        at
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.createRecordReader(PigInputFormat.java:111)
> >        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> >        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322)
> >        at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
> >        at java.security.AccessController.doPrivileged(Native Method)
> >        at javax.security.auth.Subject.doAs(Subject.java:396)
> >        at
> >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
> >        at org.apache.hadoop.mapred.Child.main(Child.java:262)
> > Caused by: org.apache.hadoop.hbase.ZooKeeperConnectionException:
> > org.apache.zookeeper.KeeperException$ConnectionLossException:
> > KeeperErrorCode = ConnectionLoss for /hbase
> >        at
> >
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:147)
> >        at
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getZooKeeperWatcher(HConnectionManager.java:989)
> >        ... 15 more
> > Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException:
> > KeeperErrorCode = ConnectionLoss for /hbase
> >        at
> > org.apache.zookeeper.KeeperException.create(KeeperException.java:90)
> >        at
> > org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> >        at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637)
> >        at
> >
> org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndFailSilent(ZKUtil.java:902)
> >        at
> >
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:133)
> >        ... 16 more
> >
> > Pig Stack Trace
> > ---------------
> > ERROR 2997: Unable to recreate exception from backed error:
> > org.apache.hadoop.hbase.ZooKeeperConnectionException:
> > org.apache.hadoop.hbase.ZooKeeperConnectionException:
> > org.apache.zookeeper.KeeperException$ConnectionLossException:
> > KeeperErrorCode = ConnectionLoss for /hbase
> >
> > org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
> > open iterator for alias my_data. Backend error : Unable to recreate
> > exception from backed error:
> > org.apache.hadoop.hbase.ZooKeeperConnectionException:
> > org.apache.hadoop.hbase.ZooKeeperConnectionException:
> > org.apache.zookeeper.KeeperException$ConnectionLossException:
> > KeeperErrorCode = ConnectionLoss for /hbase
> >        at org.apache.pig.PigServer.openIterator(PigServer.java:742)
> >        at
> > org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
> >        at
> >
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
> >        at
> >
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
> >
> >
> >
> >
> > Thanks for helping
> >
> >
> > Byambajargal
> >
> >
>
>
> --
>
> --
> Thanks and Regards,
> Sulabh Choudhury
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message