hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xine Jar <xineja...@googlemail.com>
Subject ScannerTimeoutException
Date Wed, 09 Dec 2009 17:39:45 GMT
*Hallo, can you please help me, I am getting the exception
ScannerTimeoutException.

**I-My Network:*
I have 6 machines running hadoop 0.19.1 and hbase 0.19.3

*II-My Program:*
I have an hbase table containing one column family and four
columns:xcoordinate, ycoordinate, nodeid, temperature.
The hbase table has 100 rows each row contains the x and y coordinates of a
sensor node, the node ID and a temperature value.

 *Map function:*
 1-scans the htable and searches all pair of nodes that are d1 kilometers
away from each others.
 2-For each pair it calculates the difference of their temperatures.
 3-It adds the result of each 4 differences together, and sends to the
reducer the key and the value in a Text format:
   key="d1"
   value= "result counter"  ; The counter indicates the number of
differences that are summed to gether <=4

*Reduce function:*
1- Sums the results of the same key all together
2- Sums the counters of the same key all together
3-divides a division: results/counters

*III-Problem*
For a small number of nodes in the hbase and with a small number of
distances, the program works fine. Everytime I increase the total number of
nodes in the hbase
and the number of distances, i get a ScannerTimeOutException:

*Error message*:

*9/12/09 19:41:48 INFO mapred.JobClient: Task Id :
attempt_200912091918_0004_m_000000_0, Status : FAILED*
*org.apache.hadoop.hbase.client.ScannerTimeoutException*
*        at
org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1669)*
*        at
org.apache.hadoop.hbase.mapred.TableInputFormatBase$TableRecordReader.next(TableInputFormatBase.java:212)
*
*        at
org.apache.hadoop.hbase.mapred.TableInputFormatBase$TableRecordReader.next(TableInputFormatBase.java:87)
*
*        at
org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:192)
*
*        at
org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:176)*
*        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)*
*        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)*
*        at org.apache.hadoop.mapred.Child.main(Child.java:158)*
*Caused by: org.apache.hadoop.hbase.UnknownScannerException:
org.apache.hadoop.hbase.UnknownScannerException: Name: 1917118727592717270*
*        at
org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1582)
*
*        at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)*
*        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
*
*        at java.lang.reflect.Method.invoke(Method.java:597)*
*        at
org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:632)*
*        at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:912)*

*        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)*
*        at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
*
*        at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
*
*        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)*
*        at
org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:94)
*
*        at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:852)
*
*        at
org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1665)*

*NameNode:* After statring up it generates the repetitive sentence

*2009-12-09 20:15:49,009 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
NameSystem.processReport: block blk_-8866672104836793422_1226 on
134.130.223.151:50010 size 134119 does not belong to any file.*
*2009-12-09 20:15:49,010 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
NameSystem.addToInvalidates: blk_-8866672104836793422 is added to invalidSet
of 134.130.223.151:50010*


*Tasktracker:* After starting up it generates the repetitive sentence
*org.apache.hadoop.util.DiskCheckerDiskErrorException: Could not find
taskTracker/jobcachejob_....../attempt.../output/file.out in any of the
configured directories*


*IV-Heeeeeeeeeelp*
*I have checked this problem on the web and I have seen that people tried to
set the "dfs.datanode.socket.timeout" to 0. I have done the same and
restarted hadoop and hbase, but the problem was still the same. Could you
please tell me where this error is coming from? and how is it possible to
fix it?*

*Many Thanks,
CJ*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message