hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rong-En Fan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1459) CME during memcache flush forces HRS shutdown
Date Tue, 02 Jun 2009 02:19:07 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715335#action_12715335
] 

Rong-En Fan commented on HBASE-1459:
------------------------------------

stack, here you go, and I'm ok to move out this from 0.19.4.

2009-05-30 03:47:52,079 INFO org.apache.hadoop.hbase.regionserver.HLog: Closed hdfs://nn:8020/hbase/log_x.x.x.x_1243602404362_60020/hlog.dat.1243625003085,
entries=100001. New log writer: /hbase/log_x.x.x.x_1243602404362_60020/hlog.dat.1243626472077
2009-05-30 03:57:19,559 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay
of hlog required. Forcing server shutdown
org.apache.hadoop.hbase.DroppedSnapshotException: region: table,row,1243547152622
    at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.j ava:897)
    at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:790)
    at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:228)
    at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:138)Caused
by: java.util.ConcurrentModificationException    
    at java.util.TreeMap$PrivateEntryIterator.nextEntry(TreeMap.java:1100)
    at java.util.TreeMap$EntryIterator.next(TreeMap.java:1136)    
    at java.util.TreeMap$EntryIterator.next(TreeMap.java:1131)
    at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:678)
    at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:636)
    at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:882)
    ... 3 more
2009-05-30 03:57:19,572 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 12 on 60020,
call batchUpdates([B@4051c386, [Lorg.apache.hadoop.hbase.io.BatchUpdate;@37fbb367) from x.x.x.y:37001:
error: java.io.IOException: Server not running, aborting
java.io.IOException: Server not running, aborting
    at org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:2109)
    at org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdates(HRegionServer.java:1618)
    at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:632)
    at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:912)


> CME during memcache flush forces HRS shutdown
> ---------------------------------------------
>
>                 Key: HBASE-1459
>                 URL: https://issues.apache.org/jira/browse/HBASE-1459
>             Project: Hadoop HBase
>          Issue Type: Bug
>          Components: regionserver
>    Affects Versions: 0.19.1
>            Reporter: Rong-En Fan
>             Fix For: 0.20.0, 0.19.4
>
>
> I just saw this on my 0.19.1 cluster:
> 2009-05-30 03:57:19,559 FATAL org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Replay
of hlog required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: table,rowkey,1243547152622
>     at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:897)
>     at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:790)
>     at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.flushRegion(MemcacheFlusher.java:228)
>     at org.apache.hadoop.hbase.regionserver.MemcacheFlusher.run(MemcacheFlusher.java:138)
> Caused by: java.util.ConcurrentModificationException
>     at java.util.TreeMap$PrivateEntryIterator.nextEntry(TreeMap.java:1100)
>     at java.util.TreeMap$EntryIterator.next(TreeMap.java:1136)
>     at java.util.TreeMap$EntryIterator.next(TreeMap.java:1131)
>     at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:678)
>     at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:636)
>     at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:882)
>     ... 3 more

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message