hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ramkrishna vasudevan <ramkrishna.s.vasude...@gmail.com>
Subject Re: Region server going down when deleting a table from HBase
Date Thu, 11 Jul 2013 16:08:47 GMT
Yes it creates the reader and then deletes the reader if the rename fails.


On Thu, Jul 11, 2013 at 8:09 PM, Anoop John <anoop.hbase@gmail.com> wrote:

> Ya as Ted said, if this warn is not there, need to see what happened
> to that file (just created)
>
> -Anoop-
>
> On Thu, Jul 11, 2013 at 8:07 PM, Anoop John <anoop.hbase@gmail.com> wrote:
>
> > After the flush opening the HFile reader. You are getting
> > FileNotFoundException!
> > *
> >
> > LOG
> > *.warn("Unable to rename " + path + " to " + dstPath);
> >
> > Do you see above warn in log?
> >
> > In code I can see even if this rename got failed we try to open the file
> > for read.   Also as part  of the region close, if we do a flush, we might
> > not really do the file open. It will be an unwanted input stream open.
> > Some one correct me if I am wrong pls.
> > -Anoop-
> >
> >  On Thu, Jul 11, 2013 at 6:54 PM, Sandeep L <sandeepvreddy@outlook.com
> >wrote:
> >
> >> Anoop,
> >> Following is stack trace
> >> ERROR org.apache.hadoop.hbase.executor.EventHandler: Caught throwable
> >> while processing event M_RS_CLOSE_REGIONjava.lang.RuntimeException:
> >> org.apache.hadoop.hbase.DroppedSnapshotException: region:
> >> TableName,better,1372421736221.55b13dd9eb08790e8e93757910209c21.
>  at
> >>
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:133)
> >>        at
> >> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:169)
> >>      at
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> >>        at
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> >>        at java.lang.Thread.run(Thread.java:722)Caused by:
> >> org.apache.hadoop.hbase.DroppedSnapshotException: region:
> >> TableName,better,1372421736221.55b13dd9eb08790e8e93757910209c21.
>  at
> >>
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1474)
> >>        at
> >>
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1353)
> >>        at
> >> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:905)
> >>    at
> org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:845)
> >>        at
> >>
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:119)
> >>        ... 4 moreCaused by: java.io.FileNotFoundException: File does not
> >> exist:
> >>
> /hbase/TableName/55b13dd9eb08790e8e93757910209c21/WPA/ba8ae96c466f4559901127ca24378020
> >>        at
> >>
> org.apache.hadoop.hdfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:1843)
> >>        at
> >>
> org.apache.hadoop.hdfs.DFSClient$DFSInputStream.<init>(DFSClient.java:1834)
> >>        at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:578)
> >>  at
> >>
> org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:154)
> >>        at
> >> org.apache.hadoop.fs.FilterFileSystem.open(FilterFileSystem.java:108)
> >>  at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:427)        at
> >>
> org.apache.hadoop.hbase.io.hfile.HFile.createReaderWithEncoding(HFile.java:547)
> >>        at
> >>
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.<init>(StoreFile.java:1245)
> >>        at
> >> org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:512)
> >>    at
> >>
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:602)
> >>        at
> >> org.apache.hadoop.hbase.regionserver.Store.commitFile(Store.java:791)
> >>  at
> org.apache.hadoop.hbase.regionserver.Store.access$500(Store.java:108)
> >>      at
> >>
> org.apache.hadoop.hbase.regionserver.Store$StoreFlusherImpl.commit(Store.java:2256)
> >>        at
> >>
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1455)
> >>        ... 8 more
> >> INFO org.apache.hadoop.ipc.HBaseServer: Stopping server on 60020
> >> Thanks,Sandeep.
> >>
> >> > Date: Thu, 11 Jul 2013 18:47:17 +0530
> >> > Subject: Re: Region server going down when deleting a table from HBase
> >> > From: anoop.hbase@gmail.com
> >> > To: user@hbase.apache.org
> >>  >
> >> > What is the full trace for DroppedSnapshotException ?  The caused by
> >> trace?
> >> >
> >> > -Anoop-
> >> >
> >> > On Thu, Jul 11, 2013 at 6:38 PM, Sandeep L <sandeepvreddy@outlook.com
> >> >wrote:
> >> >
> >> > > Hi,
> >> > > We are using hbase-0.94.1 with hadoop-1.0.2.
> >> > > Recently couple of time we faced a strange issue while deleting a
> >> table.
> >> > > Whenever we are deleting a table from HBase shell using disable and
> >> drop
> >> > > command at least one or two region servers are going down suddenly.
> >> > > I observed following error message in region server log:
> >> > > INFO org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED:
> >> > > Unrecoverable exception while closing region
> >> > > REGION_NAMEjava.lang.RuntimeException:
> >> > > org.apache.hadoop.hbase.DroppedSnapshotException: region:
> REGION_NAME
> >> > > Previously also we used to disable and drop tables, but we never
> seen
> >> this
> >> > > kind of issue.
> >> > >
> >> > > Thanks,Sandeep.
> >>
> >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message