hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xine Jar <xineja...@googlemail.com>
Subject Re: ScannerTimeoutException
Date Thu, 10 Dec 2009 13:15:36 GMT
Great!
thank you this was really helpful I appreciate it.

regards,
CJ

On Wed, Dec 9, 2009 at 7:00 PM, Jean-Daniel Cryans <jdcryans@apache.org>wrote:

> In 0.19.3 (which is very old btw). the default
> hbase.regionserver.lease.period is 60 seconds so you have to make sure
> you report back to the region server by doing a next() on the scanner
> before it expires else you get a ScannerTimeoutException.
>
> Also if you use scanner pre-fetching (by default you don't) then you
> have to be even more careful.
>
> If your use case requires you to spend more than one minute on a row
> after a next() invocation, then you should change the default value of
> hbase.regionserver.lease.period to something much higher.
>
> J-D
>
> On Wed, Dec 9, 2009 at 9:39 AM, Xine Jar <xinejar22@googlemail.com> wrote:
> > *Hallo, can you please help me, I am getting the exception
> > ScannerTimeoutException.
> >
> > **I-My Network:*
> > I have 6 machines running hadoop 0.19.1 and hbase 0.19.3
> >
> > *II-My Program:*
> > I have an hbase table containing one column family and four
> > columns:xcoordinate, ycoordinate, nodeid, temperature.
> > The hbase table has 100 rows each row contains the x and y coordinates of
> a
> > sensor node, the node ID and a temperature value.
> >
> >  *Map function:*
> >  1-scans the htable and searches all pair of nodes that are d1 kilometers
> > away from each others.
> >  2-For each pair it calculates the difference of their temperatures.
> >  3-It adds the result of each 4 differences together, and sends to the
> > reducer the key and the value in a Text format:
> >   key="d1"
> >   value= "result counter"  ; The counter indicates the number of
> > differences that are summed to gether <=4
> >
> > *Reduce function:*
> > 1- Sums the results of the same key all together
> > 2- Sums the counters of the same key all together
> > 3-divides a division: results/counters
> >
> > *III-Problem*
> > For a small number of nodes in the hbase and with a small number of
> > distances, the program works fine. Everytime I increase the total number
> of
> > nodes in the hbase
> > and the number of distances, i get a ScannerTimeOutException:
> >
> > *Error message*:
> >
> > *9/12/09 19:41:48 INFO mapred.JobClient: Task Id :
> > attempt_200912091918_0004_m_000000_0, Status : FAILED*
> > *org.apache.hadoop.hbase.client.ScannerTimeoutException*
> > *        at
> >
> org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1669)*
> > *        at
> >
> org.apache.hadoop.hbase.mapred.TableInputFormatBase$TableRecordReader.next(TableInputFormatBase.java:212)
> > *
> > *        at
> >
> org.apache.hadoop.hbase.mapred.TableInputFormatBase$TableRecordReader.next(TableInputFormatBase.java:87)
> > *
> > *        at
> >
> org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:192)
> > *
> > *        at
> >
> org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:176)*
> > *        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)*
> > *        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)*
> > *        at org.apache.hadoop.mapred.Child.main(Child.java:158)*
> > *Caused by: org.apache.hadoop.hbase.UnknownScannerException:
> > org.apache.hadoop.hbase.UnknownScannerException: Name:
> 1917118727592717270*
> > *        at
> >
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1582)
> > *
> > *        at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)*
> > *        at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > *
> > *        at java.lang.reflect.Method.invoke(Method.java:597)*
> > *        at
> > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:632)*
> > *        at
> >
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:912)*
> >
> > *        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> > Method)*
> > *        at
> >
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> > *
> > *        at
> >
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> > *
> > *        at
> java.lang.reflect.Constructor.newInstance(Constructor.java:513)*
> > *        at
> >
> org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:94)
> > *
> > *        at
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:852)
> > *
> > *        at
> >
> org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1665)*
> >
> > *NameNode:* After statring up it generates the repetitive sentence
> >
> > *2009-12-09 20:15:49,009 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
> > NameSystem.processReport: block blk_-8866672104836793422_1226 on
> > 134.130.223.151:50010 size 134119 does not belong to any file.*
> > *2009-12-09 20:15:49,010 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
> > NameSystem.addToInvalidates: blk_-8866672104836793422 is added to
> invalidSet
> > of 134.130.223.151:50010*
> >
> >
> > *Tasktracker:* After starting up it generates the repetitive sentence
> > *org.apache.hadoop.util.DiskCheckerDiskErrorException: Could not find
> > taskTracker/jobcachejob_....../attempt.../output/file.out in any of the
> > configured directories*
> >
> >
> > *IV-Heeeeeeeeeelp*
> > *I have checked this problem on the web and I have seen that people tried
> to
> > set the "dfs.datanode.socket.timeout" to 0. I have done the same and
> > restarted hadoop and hbase, but the problem was still the same. Could you
> > please tell me where this error is coming from? and how is it possible to
> > fix it?*
> >
> > *Many Thanks,
> > CJ*
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message