hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Esteban Gutierrez <este...@cloudera.com>
Subject Re: Hbase Error When running Map reduce
Date Thu, 22 Jan 2015 20:05:46 GMT
Hi Nishanth,

There are too many things that might be causing that problem and might
depend on your cluster deployment (cluster size, network, spindles, etc.)
or even related to the key design or how you are using filters in the
scanner that triggers the timeout. Have you looked into if the regions are
evenly distributed, if regions are balanced across the cluster and data
locality is acceptable? If you could share with us more details that would
be really useful.

thanks.
esteban.





--
Cloudera, Inc.


On Thu, Jan 22, 2015 at 11:52 AM, Nishanth S <nishanth.2884@gmail.com>
wrote:

> Hi All,
> I am running a map reduce job which scans the hbase table for  a particular
> time period and then  creates some files from that.The job runs fine for 10
> minutes or so and few around 10% of maps get completed succesfully.Here is
> the error that I am getting.Can some one help?
>
>
> 15/01/22 19:34:33 INFO mapreduce.TableRecordReaderImpl: recovered from
> org.apache.hadoop.hbase.client.ScannerTimeoutException: 559843ms
> passed since the last invocation, timeout is currently set to 60000
>         at
> org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:352)
>         at
> org.apache.hadoop.hbase.mapreduce.TableRecordReaderImpl.nextKeyValue(TableRecordReaderImpl.java:194)
>         at
> org.apache.hadoop.hbase.mapreduce.TableRecordReader.nextKeyValue(TableRecordReader.java:138)
>         at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:533)
>         at
> org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
>         at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
>         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
> Caused by: org.apache.hadoop.hbase.UnknownScannerException:
> org.apache.hadoop.hbase.UnknownScannerException: Name:
> 3432603283499371482, already closed?
>         at
> org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:2973)
>         at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:26929)
>         at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2175)
>         at
> org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1879)
>
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
>         at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>         at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
>         at
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
>         at
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
>         at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:277)
>         at
> org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:198)
>         at
> org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:57)
>         at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:120)
>         at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:96)
>         at
> org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:336)
>         ... 13 more
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message