hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Royston Sellman <royston.sell...@googlemail.com>
Subject Not a host:port issue
Date Tue, 10 Apr 2012 09:58:40 GMT
We have been running M-R jobs successfully on Hadoop v1 and HBase 0.93 SNAPSHOT (built from
trunk) using the HBase Java API. We recently updated our Hadoop and HBase installations to
the latest versions of the code from the source repositories. 

 

We now have a working Hadoop 1.0.2 cluster with HBase (trunk) and Zookeeper (3.3.3) running.
Standard M-R jobs run fine. HBase shell works fine (you can read and write to tables).

 

However, when we try to run M-R jobs that use the HBase API we get the following error:

 

  [sshexec] Exception in thread "main" 

  [sshexec] java.lang.IllegalArgumentException: Not a host:port pair: �[][][]

  [sshexec] 5086@namenode[][]namenode,60000,1334048759631

  [sshexec] 

  [sshexec]          at org.apache.hadoop.hbase.HServerAddress.<init>(HServerAddress.java:60)

  [sshexec] 

  [sshexec]          at org.apache.hadoop.hbase.MasterAddressTracker.getMasterAddress(MasterAddressTracker.java:63)

  [sshexec] 

  [sshexec]          at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:352)

  [sshexec] 

  [sshexec]          at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:90)

  [sshexec]          at uk.org.cse.utility.HBaseUtility.CreateHBaseTable(HBaseUtility.java:41)

  [sshexec]          at uk.org.cse.ingestion.SampleUploader.main(SampleUploader.java:198)

  [sshexec]          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

  [sshexec]          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

  [sshexec]          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

  [sshexec]          at java.lang.reflect.Method.invoke(Method.java:597)

  [sshexec]          at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

 

NOTE: The "[]" strings are to represent the unknown character symbol presented in the program
output. 

 

After some searching we found that this problem can be caused by classpath (client/server
jar version mismatches) so we removed all references to the HBase jars from the HADOOP_CLASSPATH
in hadoop-env.sh and added them using the backtick `hbase classpath` method described in the
documentation (http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapreduce/package-summary.html#classpath).


 

So we now launch a M-R HBase API job as follows:

 

HADOOP_CLASSPATH=`$HBASE_HOME/bin/hbase classpath` $HADOOP_INSTALL/bin/hadoop jar ${targetjardir2}/${jarfilename}
${targetclass} ${testargs}

 

One of our programs prints the current system classpath before running any M-R code. We captured
the output from this program (in the pastebin below) and it shows that the classpath contains
lib jars for both Hadoop and HBase. 

 

Please find our hadoop-env.sh, hbase-env.sh and complete program output in the following pastebins:

 

Hadoop-env.sh:

 

http://pastebin.com/3CQcHjds

 

HBase-env.sh:

 

http://pastebin.com/49AdAzv7

 

Program output:

 

http://pastebin.com/wbwEL9Li

 

Any help will be most gratefully received,

 

Royston


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message