hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shrijeet Paliwal <shrij...@rocketfuel.com>
Subject Re: A dropped table has region still deployed on region server
Date Fri, 20 Apr 2012 18:36:02 GMT
What happened in region server :

2012-04-19 19:32:31,752 INFO
org.apache.hadoop.hbase.regionserver.SplitRequest: Successful rollback of
failed split of
TestUserBTProfile,141018963342131767,1334870127822.5fe2189f32a52f3f2bce55a92ae5df9d.
2012-04-19 19:50:10,392 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Flushing
.META.,,1.1028785192
2012-04-19 19:50:10,529 INFO org.apache.hadoop.hbase.regionserver.Store:
Added hdfs://
master-hadoop.example.com:8020/hbase/.META./1028785192/info/270b515bf7d34ab88552f0e244419a0d,
entries=823, sequenceid=1825092, memsize=243.3k, filesize=102.2k
2012-04-19 19:50:10,535 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Finished memstore flush of ~483.5k/495080, currentsize=0.0/0 for region
.META.,,1.1028785192 in 88ms, sequenceid=1825092, compaction requested=false
2012-04-19 19:50:10,614 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Starting compaction on info in region .META.,,1.1028785192
2012-04-19 19:50:10,615 INFO org.apache.hadoop.hbase.regionserver.Store:
Starting compaction of 2 file(s) in info of .META.,,1.1028785192 into
tmpdir=hdfs://master-hadoop.example.com:8020/hbase/.META./1028785192/.tmp,
seqid=1825092, totalSize=541.0k
2012-04-19 19:50:10,813 INFO org.apache.hadoop.hbase.regionserver.Store:
Renaming compacted file at hdfs://
master-hadoop.example.com:8020/hbase/.META./1028785192/.tmp/5f7bea0f8f8142109c0ef41ada5a509eto
hdfs://
master-hadoop.example.com:8020/hbase/.META./1028785192/info/5f7bea0f8f8142109c0ef41ada5a509e2012-04-1919:50:10,927
INFO org.apache.hadoop.hbase.regionserver.Store: Completed
major compaction of 2 file(s) in info of .META.,,1.1028785192 into
5f7bea0f8f8142109c0ef41ada5a509e, size=442.0k; total size for store is
442.0k2012-04-19 19:50:10,927 INFO
org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
completed compaction: regionName=.META.,,1.1028785192, storeName=info,
fileCount=2, fileSize=541.0k, priority=1, time=6327679323413783;
duration=0sec2012-04-19 20:04:14,620 INFO
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush of region
userprofile,671599294440030433,1334171984301.075898f82c75fa049e7bfb88b7ca567d.
due to global heap pressure



On Fri, Apr 20, 2012 at 11:23 AM, Ted Yu <yuzhihong@gmail.com> wrote:

> bq. Successful rollback of failed split of
> Can you look for 'TestUserBTProfile,141018963342131767' in region / master
> logs to see what happened next ?
>
> Thanks
>
> On Fri, Apr 20, 2012 at 10:33 AM, Shrijeet Paliwal
> <shrijeet@rocketfuel.com>wrote:
>
> > Here are the version details:
> >
> > Hadoop : CDH3u0
> > HBase: 0.92.1 (last commit sha :
> 355e14491c78b56d9f2e270e9995d666d4ce6767)
> >
> > On Fri, Apr 20, 2012 at 10:26 AM, Shrijeet Paliwal
> > <shrijeet@rocketfuel.com>wrote:
> >
> > > Hi All,
> > >
> > > hbck is reporting this error:
> > >
> > > ERROR: Region UNKNOWN_REGION on ip-4.example.com,60020,1334705737039,
> > > key=5fe2189f32a52f3f2bce55a92ae5df9d, not on HDFS or in META but
> deployed
> > > on ip-4.example.com,60020,1334705737039
> > >
> > > Region server log :
> > > http://pastie.org/3823850
> > >
> > > Namenode audit log:
> > > http://pastie.org/3823867
> > >
> > > Master log:
> > > http://pastie.org/3823871
> > >
> > > From the region server log it seems there was a hiccup during a split
> > > event as seen here:
> > >
> > > 2189f32a52f3f2bce55a92ae5df9d.; Took too long to split the files and
> > > create the references, aborting split
> > > java.io.IOException: Took too long to split the files and create the
> > > references, aborting split
> > > >-at
> > >
> >
> org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFiles(SplitTransaction.java:613)
> > > >-at
> > >
> >
> org.apache.hadoop.hbase.regionserver.SplitTransaction.createDaughters(SplitTransaction.java:285)
> > > >-at
> > >
> >
> org.apache.hadoop.hbase.regionserver.SplitTransaction.execute(SplitTransaction.java:450)
> > > >-at
> > >
> >
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:67)
> > > >-at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > >-at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > >-at java.lang.Thread.run(Thread.java:619)
> > > 2012-04-19 19:32:31,690 INFO
> > > org.apache.hadoop.hbase.regionserver.StoreFile$Reader: Loaded ROWCOL
> > > CompoundBloomFilter metadata for 877edbbabbf74e2e94888ddb29fb1f1b
> > > 2012-04-19 19:32:31,730 INFO
> > > org.apache.hadoop.hbase.regionserver.SplitTransaction: Cleaned up old
> > > failed split transaction detritus: hdfs://
> > >
> >
> master-hadoop.example.com:8020/hbase/TestUserBTProfile/5fe2189f32a52f3f2bce55a92ae5df9d/splits
> > > 2012-04-19 19:32:31,731 INFO
> > org.apache.hadoop.hbase.regionserver.HRegion:
> > > Onlined
> > >
> >
> TestUserBTProfile,141018963342131767,1334870127822.5fe2189f32a52f3f2bce55a92ae5df9d.;
> > > next sequenceid=1823595
> > > 2012-04-19 19:32:31,752 INFO
> > > org.apache.hadoop.hbase.regionserver.SplitRequest: Successful rollback
> of
> > > failed split of
> > >
> >
> TestUserBTProfile,141018963342131767,1334870127822.5fe2189f32a52f3f2bce55a92ae5df9d.
> > >
> > > -Shrijeet
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message