hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gökhan Çapan <gkhn...@gmail.com>
Subject Re: Importing to HBase from Java problem
Date Wed, 08 Dec 2010 22:14:31 GMT
Dave,

I have switched it to 3.2.2. Still the same despite ZK says:"Client
connection successful"


On Wed, Dec 8, 2010 at 11:53 PM, Buttler, David <buttler1@llnl.gov> wrote:

> Why are you using zookeeper 3.3.0 with hbase 0.20.6?  hbase uses zk 3.2.2
> internally -- mixing and matching zk versions is not a great plan
>
> Dave
>
> -----Original Message-----
> From: Gökhan Çapan [mailto:gkhncpn@gmail.com]
> Sent: Wednesday, December 08, 2010 1:19 PM
> To: user@hbase.apache.org
> Subject: Re: Importing to HBase from Java problem
>
> Sure.
>
> This is a prototype project for me to test some algorithms before deploying
> them to cloud for production.
>
> I wanted to persist results of a Mapreduced MinHash algorithm to a NoSQL
> storage.
> After that, I wanted to execute some scans, gets,... as a simple test for
> me.
>
> It is a maven project which has dependencies to hadoop 0.20.2, hbase 0.20.6
> (i have added the downloaded jar to our internal archiva), zookeeper 3.3.0.
> Plus, an hbase-site.xml:
> <configuration>
>    <property>
>        <name>hbase.zookeeper.quorum</name>
>        <value>localhost</value>
>        <description>
>        </description>
>    </property>
> </configuration>
>
> is in its classpath.
>
>
>
> One module of the project is a series of MR jobs that finally persists
> results to an HBase table. When I tried to run it, I got some errors.
> Following your instructions, they've been solved.
>
>
> There is another module with exactly same dependencies. This is a "has
> nothing but a main" class that performs some scans on the HBase table
> produced in the previous step.
>
> after a mvn package with dependencies and adding the hbase-site.xml above
> as
> a resource, with SimpleClient as main class, I couldn't java -jar
> SimpleClient.jar, it gave me the exception I have posted.
>
> Thanks in advance.
>
>
> On Wed, Dec 8, 2010 at 8:11 PM, Stack <stack@duboce.net> wrote:
>
> > Tell us more about your client project.  What is it?  Is it a
> > container of some type?  A tomcat or something?  You could try logging
> > the CLASSPATH your client sees.  The ZooKeeper that we have in TRUNK
> > and 0.90 branch does it.  It doesn't look like the one in 0.20.6 does
> > going by the log below.
> > St.Ack
> >
> > On Wed, Dec 8, 2010 at 6:44 AM, Gökhan Çapan <gkhncpn@gmail.com> wrote:
> > > Hi,
> > >
> > > Now I am getting the same Exception while trying to run a simple HBase
> > > client.
> > >
> > > HBase 0.20.6 is both in classpath of Hadoop and the Client project.
> > > HBase and hadoop configuration is in the classpath of the Client
> Project
> > > HBase configuration is in the Hadoop classpath.
> > > HBase shell runs well.
> > > Hadoop runs well.
> > > My previous MapReduce app with same dependencies works well.
> > >
> > > Below is the log and Exception:
> > >
> > > 43:44 INFO zookeeper.ZooKeeper: Client
> > >
> >
> environment:java.library.path=.:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java
> > > 10/12/08 13:43:44 INFO zookeeper.ZooKeeper: Client
> > >
> >
> environment:java.io.tmpdir=/var/folders/9x/9x-OkSEyFDWn+13ntAMH6++++TI/-Tmp-/
> > > 10/12/08 13:43:44 INFO zookeeper.ZooKeeper: Client
> > > environment:java.compiler=<NA>
> > > 10/12/08 13:43:44 INFO zookeeper.ZooKeeper: Client environment:os.name
> > =Mac
> > > OS X
> > > 10/12/08 13:43:44 INFO zookeeper.ZooKeeper: Client
> > > environment:os.arch=x86_64
> > > 10/12/08 13:43:44 INFO zookeeper.ZooKeeper: Client
> > > environment:os.version=10.6.5
> > > 10/12/08 13:43:44 INFO zookeeper.ZooKeeper: Client environment:
> user.name
> > > =gcapan
> > > 10/12/08 13:43:44 INFO zookeeper.ZooKeeper: Client
> > > environment:user.home=/Users/gcapan
> > > 10/12/08 13:43:44 INFO zookeeper.ZooKeeper: Client
> > > environment:user.dir=/Users/gcapan/workspace/***
> > > 10/12/08 13:43:44 INFO zookeeper.ZooKeeper: Initiating client
> connection,
> > > connectString=localhost:2181 sessionTimeout=60000
> > >
> >
> watcher=org.apache.hadoop.hbase.client.HConnectionManager$TableServers@2b2d96f2
> > > 10/12/08 13:43:44 INFO zookeeper.ClientCnxn: Opening socket connection
> to
> > > server localhost/127.0.0.1:2181
> > > 10/12/08 13:43:44 INFO zookeeper.ClientCnxn: Socket connection
> > established
> > > to localhost/127.0.0.1:2181, initiating session
> > > 10/12/08 13:43:44 INFO zookeeper.ClientCnxn: Session establishment
> > complete
> > > on server localhost/127.0.0.1:2181, sessionid = 0x12cc531bd4c0006,
> > > negotiated timeout = 60000
> > > Exception in thread "main"
> java.lang.reflect.UndeclaredThrowableException
> > >    at $Proxy0.getRegionInfo(Unknown Source)
> > >    at
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRootRegion(HConnectionManager.java:874)
> > >    at
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:515)
> > >    at
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:491)
> > >    at
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:565)
> > >    at
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:524)
> > >    at
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:491)
> > >    at
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:565)
> > >    at
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:528)
> > >    at
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:491)
> > >    at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:123)
> > >    at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:101)
> > >    at com.***.***.***.***HBaseClientDemo.main(HBaseClientDemo.java:26)
> > > Caused by: org.apache.hadoop.ipc.RemoteException: java.io.IOException:
> > > java.io.IOException: Could not find requested method, the usual cause
> is
> > a
> > > version mismatch between client and server.
> > >    at
> org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:648)
> > >    at
> > >
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
> > >
> > >    at
> org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:723)
> > >    at
> > > org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:328)
> > >    ... 13 more
> > >
> > >
> > >
> > > On Tue, Dec 7, 2010 at 11:30 AM, Gökhan Çapan <gkhncpn@gmail.com>
> wrote:
> > >
> > >> I had forgotten add HBase-conf to MapReduce ClassPath.
> > >>
> > >> That solved the problem,
> > >>
> > >> Thanks.
> > >>
> > >>
> > >>
> > >> On Mon, Dec 6, 2010 at 8:40 PM, Gökhan Çapan <gkhncpn@gmail.com>
> wrote:
> > >>
> > >>>
> > >>>
> > >>> On Mon, Dec 6, 2010 at 7:54 PM, Stack <stack@duboce.net> wrote:
> > >>>
> > >>>> That looks like a mismatch between client and server hbase versions.
> > >>>> Ensure you have same running all over your cluster.  Make sure
you
> > >>>> don't have a mix of 0.20.x and 0.89... or 0.90 release candidates.
> > >>>>
> > >>>> You seem to be feeling your way.  Have you seen
> > >>>>
> > >>>>
> >
> https://hudson.apache.org/hudson/view/G-L/view/HBase/job/hbase-0.90/ws/trunk/target/site/apidocs/org/apache/hadoop/hbase/mapreduce/package-summary.html#package_description
> > >>>> ?
> > >>>>
> > >>>> You added zookeeper.  Be sure to add the hbase conf directory to
> your
> > >>>> MapReduce CLASSPATH.
> > >>>>
> > >>>> St.Ack
> > >>>>
> > >>>
> > >>> I hadn't added the hbase conf directory to my MapReduce CLASSPATH.
> > >>> Unfortunately, now I am not able to try. I will try tomorrow, and let
> > you
> > >>> know.
> > >>>
> > >>> Thanks.
> > >>>
> > >>>>
> > >>>>
> > >>>> On Mon, Dec 6, 2010 at 8:12 AM, Gökhan Çapan <gkhncpn@gmail.com>
> > wrote:
> > >>>> > Hi,
> > >>>> >
> > >>>> > I am facing with some issues when I try to write the results
of my
> > >>>> Reducer
> > >>>> > to an HBase table, in pseudo-distributed mode.
> > >>>> >
> > >>>> > HBase version: 0.20.6
> > >>>> > Hadoop version: 0.20.2
> > >>>> >
> > >>>> > I let HBase manage Zookeeper.
> > >>>> >
> > >>>> > After starting HBase, I can successfully start HBase shell,
and
> > create
> > >>>> a
> > >>>> > table.
> > >>>> >
> > >>>> > In my MapReduce app, I have a dependency to HBase 0.20.2.
> > >>>>
> > >>>
> > >>> Sorry, that was also HBase 0.20.6
> > >>>
> > >>>
> > >>>>  >
> > >>>> > However, it gave me the following error:
> > >>>> > Error: java.lang.ClassNotFoundException:
> > org.apache.zookeeper.Watcher
> > >>>> >    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> > >>>> >    at java.security.AccessController.doPrivileged(Native Method)
> > >>>> >    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> > >>>> >    at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> > >>>> >    at
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> > >>>> >    at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> > >>>> >    at java.lang.ClassLoader.defineClass1(Native Method)
> > >>>> >    at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632)
> > >>>> >    at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
> > >>>> >    at
> > >>>> >
> > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
> > >>>> >    at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
> > >>>> >    at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
> > >>>> >    at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
> > >>>> >    at java.security.AccessController.doPrivileged(Native Method)
> > >>>> >    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> > >>>> >    at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> > >>>> >    at
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> > >>>> >    at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> > >>>> >    at
> > >>>> >
> > >>>>
> >
> org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:93)
> > >>>> >    at
> org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:119)
> > >>>> >    at
> org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:101)
> > >>>> >    at
> > >>>> >
> > >>>>
> >
> org.apache.hadoop.hbase.mapreduce.TableOutputFormat.getRecordWriter(TableOutputFormat.java:116)
> > >>>> >    at
> > >>>> >
> > org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553)
> > >>>> >    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> > >>>> >    at org.apache.hadoop.mapred.Child.main(Child.java:170)
> > >>>> > .
> > >>>> > .
> > >>>> > .
> > >>>> >
> > >>>> >
> > >>>> >
> > >>>> > So I have added ZooKeeper as a dependency. Now it throws the
> > following
> > >>>> > Exception for ZooKeeper 3.2.2, 3.3.0, and 3.3.1:
> > >>>> >
> > >>>> > 10/12/06 17:51:53 INFO mapred.JobClient:  map 100% reduce
27%
> > >>>> > 10/12/06 17:51:58 INFO mapred.JobClient: Task Id :
> > >>>> > attempt_201012061721_0002_r_000000_2, Status : FAILED
> > >>>> > java.lang.reflect.UndeclaredThrowableException
> > >>>> >    at $Proxy2.getRegionInfo(Unknown Source)
> > >>>> >    at
> > >>>> >
> > >>>>
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRootRegion(HConnectionManager.java:874)
> > >>>> >    at
> > >>>> >
> > >>>>
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:515)
> > >>>> >    at
> > >>>> >
> > >>>>
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:491)
> > >>>> >    at
> > >>>> >
> > >>>>
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:565)
> > >>>> >    at
> > >>>> >
> > >>>>
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:524)
> > >>>> >    at
> > >>>> >
> > >>>>
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:491)
> > >>>> >    at
> > >>>> >
> > >>>>
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:565)
> > >>>> >    at
> > >>>> >
> > >>>>
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:528)
> > >>>> >    at
> > >>>> >
> > >>>>
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:491)
> > >>>> >    at
> org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:123)
> > >>>> >    at
> org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:101)
> > >>>> >    at
> > >>>> >
> > >>>>
> >
> org.apache.hadoop.hbase.mapreduce.TableOutputFormat.getRecordWriter(TableOutputFormat.java:116)
> > >>>> >    at
> > >>>> >
> > org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553)
> > >>>> >    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> > >>>> >    at org.apache.hadoop.mapred.Child.main(Child.java:170)
> > >>>> > Caused by: org.apache.hadoop.ipc.RemoteException:
> > java.io.IOException:
> > >>>> > java.io.IOException: Could not find requested method, the
usual
> > cause
> > >>>> is a
> > >>>> > version mismatch between client and server.
> > >>>> >    at
> > >>>> org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:648)
> > >>>> >    at
> > >>>> >
> > >>>>
> > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
> > >>>> >
> > >>>> >    at
> > >>>> org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:723)
> > >>>> >    at
> > >>>> >
> > org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:328)
> > >>>> >    ... 16 more
> > >>>> >
> > >>>> >
> > >>>> > I guess this is a compatibility issue between HBase and ZooKeeper,
> > but
> > >>>> I
> > >>>> > couldn't find the right version of ZooKeeper to add as a
> dependency.
> > >>>> >
> > >>>> >
> > >>>> > Thanks in advance,
> > >>>> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message