hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Issues with HBase Regions in Transition
Date Wed, 02 Dec 2015 20:57:09 GMT
>From log snippet:

Class org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion
not found

I assume the above class is in the trafodion jar (just a guess)
This can explain why regions would be stuck in transition.

For upgrade, consider replacing the trafodion jar instead of removing it.

Cheers

On Wed, Dec 2, 2015 at 12:51 PM, Amanda Moran <amanda.moran@esgyn.com>
wrote:

> Hi there Ted-
>
> Thanks for response. Update: I copied my trafodion.jar files back to
> ../hbase/lib , did a restart and then hbase is back online... I then
> removed them again (just to make sure) did a restart .. and then its back
> in the region in transition. Seems like we should be able to remove our
> files and do a restart and have hbase be back online... Is this incorrect?
>
> Log snippet:
>
> 2015-12-02 19:48:41,614 ERROR
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Failed open
> of region=hbase:meta,,1.1588230740, starting to roll back the global
> memstore size.
> java.lang.IllegalStateException: Could not instantiate a region instance.
>         at
> org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:5486)
>         at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5793)
>         at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5765)
>         at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5721)
>         at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5672)
>         at
>
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:356)
>         at
>
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:126)
>         at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
>         at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Class
> org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion not
> found
>         at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2112)
>         at
> org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:5475)
>         ... 10 more
> Caused by: java.lang.ClassNotFoundException: Class
> org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion not
> found
>         at
>
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2018)
>         at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2110)
>         ... 11 more
> 2015-12-02 19:48:41,627 INFO
> org.apache.hadoop.hbase.coordination.ZkOpenRegionCoordination: Opening of
> region {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '',
> ENDKEY => ''} failed, transitioning from OPENING to FAILED_OPEN in ZK,
> expecting version 4
> 2015-12-02 19:48:41,673 INFO
> org.apache.hadoop.hbase.regionserver.RSRpcServices: Open
> hbase:meta,,1.1588230740
> 2015-12-02 19:48:41,679 ERROR
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Failed open
> of region=hbase:meta,,1.1588230740, starting to roll back the global
> memstore size.
> java.lang.IllegalStateException: Could not instantiate a region instance.
>         at
> org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:5486)
>         at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5793)
>         at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5765)
>         at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5721)
>         at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5672)
>         at
>
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:356)
>         at
>
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:126)
>         at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
>         at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Class
> org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion not
> found
>         at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2112)
>         at
> org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:5475)
>         ... 10 more
> Caused by: java.lang.ClassNotFoundException: Class
> org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion not
> found
>         at
>
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2018)
>         at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2110)
>         ... 11 more
>
>
> Let me know if you need more from the logs.
>
> THANKS!
>
> On Wed, Dec 2, 2015 at 12:37 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>
> > Do you mind pastebin snippet of region server log where the region stuck
> in
> > transition was hosted ?
> >
> > This would give us some clue.
> >
> > Cheers
> >
> > On Wed, Dec 2, 2015 at 12:14 PM, Amanda Moran <amanda.moran@esgyn.com>
> > wrote:
> >
> > > Hi there All-
> > >
> > > I apologize if this issue has been raised before... I have done ALOT of
> > > googling on this issues... and finally decided to just write here to
> the
> > > source :)
> > >
> > > Background:
> > >
> > > I work on the incubating Apache Trafodion project.
> > >
> > > I am currently using HBase 1.0 with Cloudera 5.4.5.
> > >
> > > I have tried many times to upgrade my cloudera cluster (to get the
> latest
> > > version of HBase) and have always failed due to my HBase master failing
> > to
> > > come backup do to a Region suck in Transition. It is always the
> > hbase:meta.
> > >
> > > In the past I just assumed that the upgrade feature just didn't
> actually
> > > work... and so even though frustrating... I just let it go.
> > >
> > > But now I have come across this issue again, in a slightly different
> > way...
> > > and this makes me think it is an HBase problem.
> > >
> > > I am preparing for an upgrade, and because of this I decided to remove
> > the
> > > "Trafodion".jar files that are located in the ../hbase/lib directory. I
> > > thought this might be the cause of not being able to upgrade, because I
> > was
> > > leaving the trafodion files in this directory. I removed them, then
> did a
> > > deploy client config (since some configs were stale), and then did a
> > > restart. Now I have a region stuck in transition, even before the
> > upgrade!
> > >
> > > Everything I have seen on the web, has not worked in the past. How do I
> > get
> > > pass this issue? Do I have no choice but to do a full
> uninstall/reinstall
> > > and lose my data?
> > >
> > > Thanks all for any help!
> > >
> > >
> > > --
> > > Thanks,
> > >
> > > Amanda Moran
> > >
> >
>
>
>
> --
> Thanks,
>
> Amanda Moran
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message