hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From llpind <sonny_h...@hotmail.com>
Subject Re: Loading large resultset into HBase
Date Fri, 15 May 2009 23:53:33 GMT

Hey All,

Finally finished.  But didn't complete (~ between 24,770,000 - 25,760,000
records processed).  Got the following exception before java upload app
died:

org.apache.hadoop.hbase.client.NoServerForRegionException: No server address
listed in .META. for region

I believe it has to do with "Problem: DFS instability and/or regionserver
lease timeouts" (http://wiki.apache.org/hadoop/Hbase/Troubleshooting).  So i
was going to try to put some JVM flags in for GC.  Also, when creating the
table, is there any column options I should use that will help in upload?

When i try to get results from within HBase shell it gives the following
exception:

org.apache.hadoop.hbase.NotServingRegionException:
org.apache.hadoop.hbase.NotServingRegionException:
<TABLE_NAME>,<ROWID>,1242429308026
        at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2071)
        at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRow(HRegionServer.java:1534)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at
org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:632)
        at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:912)


        from org/apache/hadoop/hbase/client/HConnectionManager.java:858:in
`getRegionServerWithRetries'
        from org/apache/hadoop/hbase/client/HTable.java:608:in `getRow'
        from org/apache/hadoop/hbase/client/HTable.java:583:in `getRow'
        from org/apache/hadoop/hbase/client/HTable.java:478:in `getRow'
        from org/apache/hadoop/hbase/client/HTable.java:425:in `getRow'
        from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
        from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
        from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
        from java/lang/reflect/Method.java:597:in `invoke'
        from org/jruby/javasupport/JavaMethod.java:298:in
`invokeWithExceptionHandling'
        from org/jruby/javasupport/JavaMethod.java:259:in `invoke'
        from org/jruby/java/invokers/InstanceMethodInvoker.java:44:in `call'
        from org/jruby/runtime/callsite/CachingCallSite.java:280:in
`cacheAndCall'
        from org/jruby/runtime/callsite/CachingCallSite.java:116:in `call'
        from org/jruby/ast/CallOneArgNode.java:57:in `interpret'
        from org/jruby/ast/LocalAsgnNode.java:120:in `interpret'
... 120 levels...
        from org/jruby/internal/runtime/methods/DynamicMethod.java:226:in
`call'
        from org/jruby/internal/runtime/methods/CompiledMethod.java:216:in
`call'
        from org/jruby/internal/runtime/methods/CompiledMethod.java:71:in
`call'
        from org/jruby/runtime/callsite/CachingCallSite.java:260:in
`cacheAndCall'
        from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
        from home/hadoop/hbase19/bin/$_dot_dot_/bin/hirb.rb:441:in
`__file__'
        from home/hadoop/hbase19/bin/$_dot_dot_/bin/hirb.rb:-1:in `__file__'
        from home/hadoop/hbase19/bin/$_dot_dot_/bin/hirb.rb:-1:in `load'
        from org/jruby/Ruby.java:564:in `runScript'
        from org/jruby/Ruby.java:467:in `runNormally'
        from org/jruby/Ruby.java:340:in `runFromMain'
        from org/jruby/Main.java:214:in `run'
        from org/jruby/Main.java:100:in `run'
        from org/jruby/Main.java:84:in `main'
        from /home/hadoop/hbase19/bin/../bin/hirb.rb:338:in `get'



I am open to any other ideas.  Should my batch update be smaller than
10,000? ...5,000?  I'd like to make any adjustments before i start this long
upload again.  Thanks.
-- 
View this message in context: http://www.nabble.com/Loading-large-resultset-into-HBase-tp23566568p23568944.html
Sent from the HBase User mailing list archive at Nabble.com.


Mime
View raw message