hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Williams <mobiusinvers...@gmail.com>
Subject Re: Help with HBaseTestingUtility / MiniDFSCluster
Date Sat, 13 Jul 2013 23:08:05 GMT
Hi Ted,

I updated the dependencies and ran the tests again, and on my Mac OSX they pass and on CentOS
I get the same error:


$ lein test
Retrieving org/apache/hbase/hbase/0.94.9/hbase-0.94.9.pom from central
Retrieving org/apache/hbase/hbase/0.94.9/hbase-0.94.9.jar from central
Retrieving org/apache/hbase/hbase/0.94.9/hbase-0.94.9-tests.jar from central

lein test hbase.config-test

lein test hbase.table-test
Starting DataNode 0 with dfs.data.dir: /home/dwilliams/Desktop/Repos/hbase/target/test-data/1140edc6-7242-40cd-8ed8-05847fb14949/dfscluster_1e40ce89-1986-450b-ba6d-983caa9aeb78/dfs/data/data1,/home/dwilliams/Desktop/Repos/hbase/target/test-data/1140edc6-7242-40cd-8ed8-05847fb14949/dfscluster_1e40ce89-1986-450b-ba6d-983caa9aeb78/dfs/data/data2

lein test :only hbase.table-test/create-table

ERROR in (create-table) (MiniDFSCluster.java:426)
Uncaught exception, not in assertion.
expected: nil
  actual: java.lang.NullPointerException: null
 at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes (MiniDFSCluster.java:426)
    org.apache.hadoop.hdfs.MiniDFSCluster.<init> (MiniDFSCluster.java:284)
    org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster (HBaseTestingUtility.java:451)
    org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster (HBaseTestingUtility.java:619)
    org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster (HBaseTestingUtility.java:575)
    org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster (HBaseTestingUtility.java:562)
    hbase.table_test$test_config.doInvoke (table_test.clj:10)
    clojure.lang.RestFn.invoke (RestFn.java:397)
    hbase.table_test/fn (table_test.clj:19)

On Jul 13, 2013, at 1:11 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> Do you mind trying the following change to see if the problem persists for latest 0.94
release ?
> 
> Thanks
> 
> diff --git a/project.clj b/project.clj
> index 2554784..0d8be0e 100644
> --- a/project.clj
> +++ b/project.clj
> @@ -5,8 +5,8 @@
>         :dependencies [
>                 [org.clojure/clojure "1.5.1"]
>                 [org.apache.hadoop/hadoop-core "1.2.0"]
> -               [org.apache.hbase/hbase "0.94.6.1"]
> +               [org.apache.hbase/hbase "0.94.9"]
>                 [org.apache.hadoop/hadoop-test "1.2.0"]
> -               [org.apache.hbase/hbase "0.94.6.1" :classifier "tests"]]
> +               [org.apache.hbase/hbase "0.94.9" :classifier "tests"]]
>         :plugins [[lein-marginalia "0.7.1"]])
> 
> 
> On Fri, Jul 12, 2013 at 10:22 PM, David Williams <mobiusinversion@gmail.com> wrote:
> Hi Ted,
> 
> In terms of versions, here are the jars I'm using, which come from Maven Central.
> 
> org.apache.hadoop/hadoop-core "1.2.0"
> 
> org.apache.hbase/hbase "0.94.6.1"
> org.apache.hadoop/hadoop-test "1.2.0"
> 
> org.apache.hbase/hbase "0.94.6.1" :classifier "tests"
> 
> 
> The flag ':classifier "tests"' above is a specific instruction to to the Leiningen 2.0
to use the pomegranate library to handle Sonatype Aether and dynamic runtime modification
of the classpath, which in this case is needed to import org.apache.hadoop.hbase HBaseTestingUtility.
> 
> https://github.com/cemerick/pomegranate
> 
> 
> I just checked on address resolution, on my Mac OSX where the TestingUtility passes:
> 
> user=> (import 'java.net.InetSocketAddress)
> java.net.InetSocketAddress
> user=> (def x (InetSocketAddress. 8000))
> #'user/x
> user=> (.getAddress x)
> #<Inet4Address 0.0.0.0/0.0.0.0>
> user=>
> 
> Then I check on a CentOS vm, the unit tests still fail but the address resolution also
worked in the repl and produced the same output as above.
> 
> 
> 
> 
> On Jul 12, 2013, at 9:30 PM, Ted Yu wrote:
> 
>> I installed lein on Mac and Linux.
>> 
>> I tried 'lein test' on both platforms and the test passed on both:
>> 
>> lein test hbase.table-test
>> Starting DataNode 0 with dfs.data.dir: /homes/hortonzy/mobius/target/test-data/246828b9-1be9-4949-9bbc-b215b378fb67/dfscluster_9ed0bd88-d309-4fed-9823-3bbf86973ae4/dfs/data/data1,/homes/hortonzy/mobius/target/test-data/246828b9-1be9-4949-9bbc-b215b378fb67/dfscluster_9ed0bd88-d309-4fed-9823-3bbf86973ae4/dfs/data/data2
>> Cluster is active
>> 
>> Ran 11 tests containing 14 assertions.
>> 0 failures, 0 errors.
>> 
>> Here is Linux OS:
>> 
>> Linux a.net 2.6.32-220.23.1.el6.YAHOO.20120713.x86_64 #1 SMP Fri Jul 13 11:40:51
CDT 2012 x86_64 x86_64 x86_64 GNU/Linux
>> 
>> Looking at MiniDFSCluster.java, line 426:
>> 
>>       String ipAddr = dn.getSelfAddr().getAddress().getHostAddress();
>> 
>> It seems dn.getSelfAddr().getAddress() returned null.
>> 
>> According to:
>> http://docs.oracle.com/javase/7/docs/api/java/net/InetSocketAddress.html#getAddress()
>> 
>> This would mean address resolution problem.
>> 
>> Can you check ?
>> 
>> Cheers
>> 
>> On Fri, Jul 12, 2013 at 7:37 PM, David Williams <mobiusinversion@gmail.com>
wrote:
>> Hi all,
>> 
>> I am having an issue starting the a mini cluster for the HBaseTestingUtility.  In
short I can on Mac OSX, but cannot on Linux.   But the error is cryptic and I don't know what
to do next.
>> 
>> I submitted a ticket with full details on StackOverflow,
>> 
>> http://stackoverflow.com/questions/17625938/hbase-minidfscluster-java-fails-in-certain-environments
>> 
>> But when I call .startMiniCluster  on an instance of HBaseTestingUtility , on Linux
(CentOS x86_64), I receive this error:
>> 
>> 
>> 
>> ERROR in (create-table) (MiniDFSCluster.java:426)
>> Uncaught exception, not in assertion.
>> expected: nil
>>   actual: java.lang.NullPointerException: null
>>  at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes (MiniDFSCluster.java:426)
>>     org.apache.hadoop.hdfs.MiniDFSCluster.<init> (MiniDFSCluster.java:284)
>>     org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster (HBaseTestingUtility.java:444)
>>     org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster (HBaseTestingUtility.java:612)
>>     org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster (HBaseTestingUtility.java:568)
>>     org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster (HBaseTestingUtility.java:555)
>> 
>> I would appreciate help in finding out whats going on and how to set up my ENV to
use the HBaseTestingUtility.
>> 
>> Thanks
>> David
>>  
>> 
> 
> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message