hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: hbase export in cluster mode
Date Fri, 09 Apr 2010 04:46:46 GMT
Feel free to comment in this JIRA:
https://issues.apache.org/jira/browse/HADOOP-6695

On Thu, Apr 8, 2010 at 4:39 PM, Jean-Daniel Cryans <jdcryans@apache.org>wrote:

> Ted, please make sure you keep the conversation on the mailing list,
> private answers have no value for an open source community.
>
> About JT, it would be awesome not to have to restart it but I don't
> know enough about its internals to comment on how hard it would be to
> modify. Maybe there's even an open jira.
>
> Or maybe someone else who listens could comment? Cloudera people?
>
> J-D
>
> On Thu, Apr 8, 2010 at 4:33 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> > J-D:
> > Do you think it makes for job tracker to dynamically refresh classpath ?
> > I can see this need because HBase, HIVE, etc may initiate request in
> cluster
> > mode and we don't want to restart job tracker in production often.
> >
> > Cheers
> >
> > On Thu, Apr 8, 2010 at 3:52 PM, Jean-Daniel Cryans <jdcryans@apache.org>
> > wrote:
> >>
> >> Actually the preferred method is putting the hbase and zookeeper jar
> >> on the HADOOP_CLASSPATH of conf/hadoop-env.sh (putting the hbase conf
> >> folder there is also good).
> >>
> >> Yeah have to restart JT, but AFAIK it's not HBase's fault right? ;)
> >>
> >> J-D
> >>
> >> On Thu, Apr 8, 2010 at 3:47 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> >> > I copied zookeeper-3.2.1.jar after seeing the exception below.
> >> > Turns out that I have to restart job tracker to refresh classpath :-(
> >> >
> >> > On Thu, Apr 8, 2010 at 3:03 PM, Jean-Daniel Cryans
> >> > <jdcryans@apache.org>wrote:
> >> >
> >> >> Zookeeper jar isn't in the path of hadoop?
> >> >>
> >> >> btw https://issues.apache.org/jira/browse/HBASE-2335
> >> >>
> >> >> J-D
> >> >>
> >> >> On Thu, Apr 8, 2010 at 2:58 PM, Ted Yu <yuzhihong@gmail.com>
wrote:
> >> >> > I ran hbase export in cluster mode.
> >> >> >
> >> >> > From Saturn:
> >> >> > 10/04/07 14:18:22 INFO zookeeper.ClientCnxn: zookeeper.
> >> >> > disableAutoWatchReset is false
> >> >> > 10/04/07 14:18:22 INFO zookeeper.ClientCnxn: Attempting connection
> to
> >> >> server
> >> >> > uranus/10.10.31.18:2181
> >> >> > 10/04/07 14:18:22 INFO zookeeper.ClientCnxn: Priming connection
to
> >> >> > java.nio.channels.SocketChannel[connected local=/10.10.31.16:34170
> >> >> remote=uranus/
> >> >> > 10.10.31.18:2181]
> >> >> > 10/04/07 14:18:22 INFO zookeeper.ClientCnxn: Server connection
> >> >> > successful
> >> >> > 10/04/07 14:18:22 INFO mapreduce.TableInputFormatBase: split:
> >> >> 0*->Neptune*:,
> >> >> > 10/04/07 14:18:23 INFO mapred.JobClient: Running job:
> >> >> job_201004051529_0015
> >> >> > 10/04/07 14:18:24 INFO mapred.JobClient:  map 0% reduce 0%
> >> >> > 10/04/07 14:18:36 INFO mapred.JobClient: Task Id :
> >> >> > attempt_201004051529_0015_m_000000_0, Status : FAILED
> >> >> > Error: java.lang.ClassNotFoundException:
> org.apache.zookeeper.Watcher
> >> >> >        at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
> >> >> >        at java.security.AccessController.doPrivileged(Native
> Method)
> >> >> >        at
> java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> >> >> >        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> >> >> >        at
> >> >> > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> >> >> >        at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
> >> >> >        at
> >> >> > java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
> >> >> >        at java.lang.ClassLoader.defineClass1(Native Method)
> >> >> >        at java.lang.ClassLoader.defineClass(ClassLoader.java:621)
> >> >> >        at
> >> >> >
> >> >> >
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)
> >> >> >        at
> >> >> > java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
> >> >> >        at
> java.net.URLClassLoader.access$000(URLClassLoader.java:56)
> >> >> >        at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
> >> >> >        at java.security.AccessController.doPrivileged(Native
> Method)
> >> >> >        at
> java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> >> >> >        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> >> >> >        at
> >> >> > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> >> >> >        at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
> >> >> >        at
> >> >> > java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
> >> >> >        at
> >> >> >
> >> >>
> >> >>
> org.apache.hadoop.hbase.client.HConnectionManager.getClientZooKeeperWatcher(HConnectionManager.java:151)
> >> >> >        at
> >> >> >
> >> >>
> >> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getZooKeeperWrapper(HConnectionManager.java:885)
> >> >> >        at
> >> >> >
> >> >>
> >> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRootRegion(HConnectionManager.java:901)
> >> >> >        at
> >> >> >
> >> >>
> >> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:580)
> >> >> >        at
> >> >> >
> >> >>
> >> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:556)
> >> >> >        at
> >> >> >
> >> >>
> >> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:630)
> >> >> >        at
> >> >> >
> >> >>
> >> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:589)
> >> >> >        at
> >> >> >
> >> >>
> >> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:556)
> >> >> >        at
> >> >> >
> >> >>
> >> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:630)
> >> >> >        at
> >> >> >
> >> >>
> >> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:593)
> >> >> >        at
> >> >> >
> >> >>
> >> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:556)
> >> >> >        at
> >> >> > org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:127)
> >> >> >        at
> >> >> > org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:105)
> >> >> >        at
> >> >> >
> >> >>
> >> >>
> org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:73)
> >> >> >        at
> >> >> >
> >> >> >
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
> >> >> >        at
> >> >> >
> >> >>
> >> >>
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> >> >> >        at
> >> >> > org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:536)
> >> >> >        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> >> >> >        at org.apache.hadoop.mapred.Child.main(Child.java:170)
> >> >> >
> >> >> > I wonder why the above exception was thrown because we have same
> >> >> > hbase
> >> >> > deployment on all the servers:
> >> >> > [root@neptune software]# ls -l lib/zookeeper-3.2.1.jar
> >> >> > -rwxr-xr-x 1 hbaseadmin hbaseadmin 913093 Oct  7 11:55
> >> >> > lib/zookeeper-3.2.1.jar
> >> >> >
> >> >> > Can someone shed some light on how to get over the above exception
> ?
> >> >> >
> >> >> > Thanks
> >> >> >
> >> >>
> >> >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message