hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ryan rawson (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2073) IllegalArgumentException causing regionserver failure
Date Mon, 04 Jan 2010 23:14:54 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796424#action_12796424
] 

ryan rawson commented on HBASE-2073:
------------------------------------

the log doesnt have much detail, typically in the past these kinds of errors can indicate
short read from HDFS. I'm not seeing any of those in this logfile, is there anything interesting
in the HDFS logs?

For this error to have happened, there should have been a short read in the block, which is
"impossible".

> IllegalArgumentException causing regionserver failure
> -----------------------------------------------------
>
>                 Key: HBASE-2073
>                 URL: https://issues.apache.org/jira/browse/HBASE-2073
>             Project: Hadoop HBase
>          Issue Type: Bug
>          Components: regionserver
>    Affects Versions: 0.20.2
>         Environment: Ubuntu 8.10, Java 1.6.0_10, HBase 0.20.2
>            Reporter: Greg Lu
>            Assignee: ryan rawson
>             Fix For: 0.20.3
>
>         Attachments: hbase-hadoop-regionserver-factory05.lab.mtl.log
>
>
> After a regionserver went down last night, I checked its logs and found the following
exception:
> 2009-12-29 00:17:27,663 INFO org.apache.hadoop.hbase.regionserver.HLog: Roll /hbase/amsterdam_factory/.logs/factory05.lab.mtl,60020,1262042255724/hlog.dat.1262060247637,
entries=1830, calcsize=22946017, filesize=22758899. New hlog /hbase/amsterdam_factory/.logs/factory05.lab.mtl,60020,1262042255724/hlog.dat.1262063847659
> 2009-12-29 00:34:36,210 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: 
> java.lang.IllegalArgumentException
> 	at java.nio.Buffer.position(Buffer.java:218)
> 	at org.apache.hadoop.hbase.io.hfile.HFile$Reader$Scanner.next(HFile.java:1114)
> 	at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:58)
> 	at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:79)
> 	at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:189)
> 	at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
> 	at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1776)
> 	at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1719)
> 	at org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1944)
> 	at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:648)
> 	at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
> 2009-12-29 00:34:36,214 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 0
on 60020, call next(4170645244799815171, 1) from 192.168.1.108:53401: error: java.io.IOException:
java.lang.IllegalArgumentException
> java.io.IOException: java.lang.IllegalArgumentException
> 	at org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:869)
> 	at org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:859)
> 	at org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1965)
> 	at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:648)
> 	at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
> Caused by: java.lang.IllegalArgumentException
> 	at java.nio.Buffer.position(Buffer.java:218)
> 	at org.apache.hadoop.hbase.io.hfile.HFile$Reader$Scanner.next(HFile.java:1114)
> 	at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:58)
> 	at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:79)
> 	at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:189)
> 	at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
> 	at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:1776)
> 	at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:1719)
> 	at org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1944)
> 	... 5 more
> Looks like this bug was encountered before at https://issues.apache.org/jira/browse/HBASE-1495
and spanned a few JIRAs. It's supposed to be resolved as of 0.20.0, but we're running 0.20.2
and it took down one of our regionservers.
> I'm also attaching more of the log.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message