hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jerry He <jerry...@gmail.com>
Subject Re: the Exception of "VisibilityLabelsCache not yet instantiated" happen when I set the class "ExpAsStringVisibilityLabelServiceImpl" as Implement of "VisibilityLabelService".
Date Mon, 02 Feb 2015 18:54:35 GMT
Hi,

All the default scan label generators uses the VisibilityLabelCache to get
user labels somewhere.  But the VisibilityLableCache implementation is for
the DefaultVisibilityLabelServiceImpl mostly.
Even if we add the method createAndGet() for the VisibilityLableCache
to ExpAsStringVisibilityLabelServiceImpl,
it will not work -- the cache is always empty because
ExpAsStringVisibilityLabelServiceImpl does not use it.
We probably need a test SLG like ExpAsStringVisibilitySLG for
ExpAsStringVisibilityLabelServiceImpl?

Jerry

On Mon, Feb 2, 2015 at 9:20 AM, ramkrishna vasudevan <
ramkrishna.s.vasudevan@gmail.com> wrote:

> Hi
>
> Just for confirmation - you are trying to use cell level visibility tags -
> correct?  Cell Level ACLs are different from visibility tags.
>
> Coming to ExpAsStringVisibilityLabelServiceImpl - it is currently a test
> implementation and not a real src implementation. The idea for this class
> is to show how the visibility labels can be used without representing them
> as ordinals - which is internally stored in Zookeeper.
>
> Please file a JIRA for the issue that you had specified related to the
> init() API but the reason for this class to be in 'test' is that it is
> purely an example implementation.  Also what is your use case?  Do you want
> the visibility labels to be stored as Strings rather than ordinals?
>
> The reason why you may need to store as strings rather than ordinals is for
> cases where you two have clusters  with replication and you cannot have the
> same set of labels defined in both the clusters.
>
> BTW thanks a lot for reporting this bug/issue.
>
> Regards
> Ram
>
> On Mon, Feb 2, 2015 at 10:21 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>
> > This looks like a bug.
> >
> > Mind filing an issue and attach your patch ?
> >
> > Thanks
> >
> > On Sun, Feb 1, 2015 at 11:13 PM, 15810705424 <15810705424@163.com>
> wrote:
> >
> > > Hi, all:
> > >      Not long ago, I wanted to test Cell-Level ACLs by using tags and
> use
> > > the class "ExpAsStringVisibilityLabelServiceImpl" instead of  the
> default
> > > class "DefaultVisibilityLabelServiceImpl". set as fllows in
> > hbase-site.xml.
> > > My HBase version is  0.99.2.
> > > But when I restarted my cluster, the exception happened:
> > >
> > > 2015-01-29 00:32:50,455 ERROR [RS_OPEN_META-asiaB:60030-0]
> > > coprocessor.CoprocessorHost: The coprocessor
> > > org.apache.hadoop.hbase.security.visibility.VisibilityController threw
> > > java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
> > >
> > > java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
> > >
> > >         at
> > >
> >
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131)
> > >
> > >         at
> > >
> >
> org.apache.hadoop.hbase.security.visibility.VisibilityUtils.getScanLabelGenerators(VisibilityUtils.java:200)
> > >
> > >         at
> > >
> >
> org.apache.hadoop.hbase.security.visibility.ExpAsStringVisibilityLabelServiceImpl.init(ExpAsStringVisibilityLabelServiceImpl.java:335)
> > >
> > >         at
> > >
> >
> org.apache.hadoop.hbase.security.visibility.VisibilityController.initVisibilityLabelService(VisibilityController.java:260)
> > >
> > >         at
> > >
> >
> org.apache.hadoop.hbase.security.visibility.VisibilityController.postOpen(VisibilityController.java:246)
> > >
> > >         at
> > >
> >
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$2.call(RegionCoprocessorHost.java:346)
> > >
> > >         at
> > >
> >
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionOperation.call(RegionCoprocessorHost.java:1575)
> > >
> > >         at
> > >
> >
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1650)
> > >
> > >         at
> > >
> >
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1607)
> > >
> > >         at
> > >
> >
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postOpen(RegionCoprocessorHost.java:342)
> > >
> > >         at
> > >
> >
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:797)
> > >
> > >         at
> > >
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:725)
> > >
> > >         at
> > >
> >
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4797)
> > >
> > >         at
> > >
> >
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4768)
> > >
> > >         at
> > >
> >
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4739)
> > >
> > >         at
> > >
> >
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4695)
> > >
> > >         at
> > >
> >
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4646)
> > >
> > >         at
> > >
> >
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:356)
> > >
> > >         at
> > >
> >
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:126)
> > >
> > >         at
> > >
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> > >
> > >         at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > >
> > >         at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > >
> > >         at java.lang.Thread.run(Thread.java:745)
> > >
> > > Caused by: java.lang.reflect.InvocationTargetException
> > >
> > >         at
> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> > > Method)
> > >
> > >        at
> > >
> >
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> > >
> > >         at
> > >
> >
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> > >
> > >         at
> > java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> > >
> > >         at
> > >
> >
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129)
> > >
> > >         ... 22 more
> > >
> > > Caused by: java.lang.IllegalStateException: VisibilityLabelsCache not
> yet
> > > instantiated
> > >
> > >         at
> > >
> >
> org.apache.hadoop.hbase.security.visibility.VisibilityLabelsCache.get(VisibilityLabelsCache.java:106)
> > >
> > >         at
> > >
> >
> org.apache.hadoop.hbase.security.visibility.FeedUserAuthScanLabelGenerator.<init>(FeedUserAuthScanLabelGenerator.java:47)
> > >
> > >         ... 27 more
> > >
> > > then all regionservers exit as the exception.
> > >
> > > so I start to check code to find the cause:
> > >
> > > At first , I found the place where the exception of  "
> > > java.lang.IllegalStateException: VisibilityLabelsCache not yet
> > > instantiated"  exist.
> > >
> > > by this, It turn out to be that  the instance of
> "VisibilityLabelsCache"
> > >  had not yet instantiated when I used it .
> > >
> > > Secondly, I continue to find the serial called process:
> > >
> > > 1: class: ExpAsStringVisibilityLabelServiceImpl.java   method: init() .
> > >
> > > public void init(RegionCoprocessorEnvironment e) throws IOException {
> > >
> > >     this.scanLabelGenerators =
> > > VisibilityUtils.getScanLabelGenerators(this.conf);
> > >
> > >    ......
> > >
> > >     }
> > >
> > >   }
> > >
> > > 2: class: VisibilityUtils   method: getScanLabelGenerators()
> > >
> > >   public static List<ScanLabelGenerator>
> > > getScanLabelGenerators(Configuration conf) {
> > >
> > >     String slgClassesCommaSeparated =
> > > conf.get(VISIBILITY_LABEL_GENERATOR_CLASS);
> > >
> > >     List<ScanLabelGenerator> slgs = new
> ArrayList<ScanLabelGenerator>();
> > >
> > >     if (StringUtils.isNotEmpty(slgClassesCommaSeparated)) {
> > >
> > >       .......
> > >
> > >     }
> > >
> > >     // If no SLG is specified in conf, by default we'll add two SLGs
> > >
> > >     if (slgs.isEmpty()) {
> > >
> > >
> > >
> >
> slgs.add(ReflectionUtils.newInstance(FeedUserAuthScanLabelGenerator.class,
> > > conf));
> > >
> > >
> > >
> >
> slgs.add(ReflectionUtils.newInstance(DefinedSetFilterScanLabelGenerator.class,
> > > conf));
> > >
> > >     }
> > >
> > > }
> > >
> > > 3:class : FeedUserAuthScanLabelGenerator.java    constructer
> > > method: FeedUserAuthScanLabelGenerator()
> > >
> > >   public FeedUserAuthScanLabelGenerator() {
> > >
> > >     this.labelsCache = VisibilityLabelsCache.get();
> > >
> > >   }
> > >
> > > finally ,  I have found that  the instance of "VisibilityLabelsCache"
> had
> > > never not been inited.
> > >
> > > But the default set "DefaultVisibilityLabelServiceImpl" is ok, then I
> > > compare ExpAsStringVisibilityLabelServiceImpl with
> > > DefaultVisibilityLabelServiceImpl  ,  I found the default class of "
> > > DefaultVisibilityLabelServiceImpl "  called the method  of
> "createAndGet"
> > > to instance:
> > >
> > >   public void init(RegionCoprocessorEnvironment e) throws IOException {
> > >
> > >     ZooKeeperWatcher zk = e.getRegionServerServices().getZooKeeper();
> > >
> > >     try {
> > >
> > >       labelsCache = VisibilityLabelsCache.createAndGet(zk, this.conf);
> > >
> > >     } catch (IOException ioe) {
> > >
> > >       LOG.error("Error creating VisibilityLabelsCache", ioe);
> > >
> > >       throw ioe;
> > >
> > >     }
> > >
> > >     this.scanLabelGenerators =
> > > VisibilityUtils.getScanLabelGenerators(this.conf);
> > >
> > >     .........
> > >
> > > }
> > >
> > > So I think the class of "ExpAsStringVisibilityLabelServiceImpl" also
> lack
> > > of these codes,  then add the method "createAndGet" to the class "
> > > ExpAsStringVisibilityLabelServiceImpl" and rebuilt it,  replace it to
> the
> > > my cluster,  At last ,  My cluster is ok  .
> > >
> > > Next days, I fount this problem exists in 0.98.9.  But I found the
> class
> > ExpAsStringVisibilityLabelServiceImpl
> > >  is  always located in maven project: src/test, So I think it may be in
> > > test moment.
> > >
> > >  Is this a bug ?  Any comments/suggestions for work around?
> > >
> > >
> > > Thanks all.
> > >
> > >
> > >
> > >
> > >
> > > 来自网易手机号码邮箱了解更多 <http://shouji.163.com>
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message