hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jerry He <jerry...@gmail.com>
Subject Re: the Exception of "VisibilityLabelsCache not yet instantiated" happen when I set the class "ExpAsStringVisibilityLabelServiceImpl" as Implement of "VisibilityLabelService".
Date Mon, 02 Feb 2015 19:24:27 GMT
Hi,

There is a SimpleScanLabelGenerator in the package. This is works with
ExpAsStringVisibilityLabelServiceImpl.  And it is used in the unit test for
ExpAsStringVisibilityLabelServiceImpl.

Jerry

On Mon, Feb 2, 2015 at 10:54 AM, Jerry He <jerryjch@gmail.com> wrote:

> Hi,
>
> All the default scan label generators uses the VisibilityLabelCache to get
> user labels somewhere.  But the VisibilityLableCache implementation is for
> the DefaultVisibilityLabelServiceImpl mostly.
> Even if we add the method createAndGet() for the VisibilityLableCache to ExpAsStringVisibilityLabelServiceImpl,
> it will not work -- the cache is always empty because
> ExpAsStringVisibilityLabelServiceImpl does not use it.
> We probably need a test SLG like ExpAsStringVisibilitySLG for
> ExpAsStringVisibilityLabelServiceImpl?
>
> Jerry
>
> On Mon, Feb 2, 2015 at 9:20 AM, ramkrishna vasudevan <
> ramkrishna.s.vasudevan@gmail.com> wrote:
>
>> Hi
>>
>> Just for confirmation - you are trying to use cell level visibility tags -
>> correct?  Cell Level ACLs are different from visibility tags.
>>
>> Coming to ExpAsStringVisibilityLabelServiceImpl - it is currently a test
>> implementation and not a real src implementation. The idea for this class
>> is to show how the visibility labels can be used without representing them
>> as ordinals - which is internally stored in Zookeeper.
>>
>> Please file a JIRA for the issue that you had specified related to the
>> init() API but the reason for this class to be in 'test' is that it is
>> purely an example implementation.  Also what is your use case?  Do you
>> want
>> the visibility labels to be stored as Strings rather than ordinals?
>>
>> The reason why you may need to store as strings rather than ordinals is
>> for
>> cases where you two have clusters  with replication and you cannot have
>> the
>> same set of labels defined in both the clusters.
>>
>> BTW thanks a lot for reporting this bug/issue.
>>
>> Regards
>> Ram
>>
>> On Mon, Feb 2, 2015 at 10:21 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>>
>> > This looks like a bug.
>> >
>> > Mind filing an issue and attach your patch ?
>> >
>> > Thanks
>> >
>> > On Sun, Feb 1, 2015 at 11:13 PM, 15810705424 <15810705424@163.com>
>> wrote:
>> >
>> > > Hi, all:
>> > >      Not long ago, I wanted to test Cell-Level ACLs by using tags and
>> use
>> > > the class "ExpAsStringVisibilityLabelServiceImpl" instead of  the
>> default
>> > > class "DefaultVisibilityLabelServiceImpl". set as fllows in
>> > hbase-site.xml.
>> > > My HBase version is  0.99.2.
>> > > But when I restarted my cluster, the exception happened:
>> > >
>> > > 2015-01-29 00:32:50,455 ERROR [RS_OPEN_META-asiaB:60030-0]
>> > > coprocessor.CoprocessorHost: The coprocessor
>> > > org.apache.hadoop.hbase.security.visibility.VisibilityController threw
>> > > java.lang.RuntimeException:
>> java.lang.reflect.InvocationTargetException
>> > >
>> > > java.lang.RuntimeException:
>> java.lang.reflect.InvocationTargetException
>> > >
>> > >         at
>> > >
>> >
>> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131)
>> > >
>> > >         at
>> > >
>> >
>> org.apache.hadoop.hbase.security.visibility.VisibilityUtils.getScanLabelGenerators(VisibilityUtils.java:200)
>> > >
>> > >         at
>> > >
>> >
>> org.apache.hadoop.hbase.security.visibility.ExpAsStringVisibilityLabelServiceImpl.init(ExpAsStringVisibilityLabelServiceImpl.java:335)
>> > >
>> > >         at
>> > >
>> >
>> org.apache.hadoop.hbase.security.visibility.VisibilityController.initVisibilityLabelService(VisibilityController.java:260)
>> > >
>> > >         at
>> > >
>> >
>> org.apache.hadoop.hbase.security.visibility.VisibilityController.postOpen(VisibilityController.java:246)
>> > >
>> > >         at
>> > >
>> >
>> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$2.call(RegionCoprocessorHost.java:346)
>> > >
>> > >         at
>> > >
>> >
>> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionOperation.call(RegionCoprocessorHost.java:1575)
>> > >
>> > >         at
>> > >
>> >
>> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1650)
>> > >
>> > >         at
>> > >
>> >
>> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1607)
>> > >
>> > >         at
>> > >
>> >
>> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postOpen(RegionCoprocessorHost.java:342)
>> > >
>> > >         at
>> > >
>> >
>> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:797)
>> > >
>> > >         at
>> > >
>> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:725)
>> > >
>> > >         at
>> > >
>> >
>> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4797)
>> > >
>> > >         at
>> > >
>> >
>> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4768)
>> > >
>> > >         at
>> > >
>> >
>> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4739)
>> > >
>> > >         at
>> > >
>> >
>> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4695)
>> > >
>> > >         at
>> > >
>> >
>> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4646)
>> > >
>> > >         at
>> > >
>> >
>> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:356)
>> > >
>> > >         at
>> > >
>> >
>> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:126)
>> > >
>> > >         at
>> > >
>> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
>> > >
>> > >         at
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> > >
>> > >         at
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> > >
>> > >         at java.lang.Thread.run(Thread.java:745)
>> > >
>> > > Caused by: java.lang.reflect.InvocationTargetException
>> > >
>> > >         at
>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>> > > Method)
>> > >
>> > >        at
>> > >
>> >
>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>> > >
>> > >         at
>> > >
>> >
>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>> > >
>> > >         at
>> > java.lang.reflect.Constructor.newInstance(Constructor.java:526)
>> > >
>> > >         at
>> > >
>> >
>> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129)
>> > >
>> > >         ... 22 more
>> > >
>> > > Caused by: java.lang.IllegalStateException: VisibilityLabelsCache not
>> yet
>> > > instantiated
>> > >
>> > >         at
>> > >
>> >
>> org.apache.hadoop.hbase.security.visibility.VisibilityLabelsCache.get(VisibilityLabelsCache.java:106)
>> > >
>> > >         at
>> > >
>> >
>> org.apache.hadoop.hbase.security.visibility.FeedUserAuthScanLabelGenerator.<init>(FeedUserAuthScanLabelGenerator.java:47)
>> > >
>> > >         ... 27 more
>> > >
>> > > then all regionservers exit as the exception.
>> > >
>> > > so I start to check code to find the cause:
>> > >
>> > > At first , I found the place where the exception of  "
>> > > java.lang.IllegalStateException: VisibilityLabelsCache not yet
>> > > instantiated"  exist.
>> > >
>> > > by this, It turn out to be that  the instance of
>> "VisibilityLabelsCache"
>> > >  had not yet instantiated when I used it .
>> > >
>> > > Secondly, I continue to find the serial called process:
>> > >
>> > > 1: class: ExpAsStringVisibilityLabelServiceImpl.java   method: init()
>> .
>> > >
>> > > public void init(RegionCoprocessorEnvironment e) throws IOException {
>> > >
>> > >     this.scanLabelGenerators =
>> > > VisibilityUtils.getScanLabelGenerators(this.conf);
>> > >
>> > >    ......
>> > >
>> > >     }
>> > >
>> > >   }
>> > >
>> > > 2: class: VisibilityUtils   method: getScanLabelGenerators()
>> > >
>> > >   public static List<ScanLabelGenerator>
>> > > getScanLabelGenerators(Configuration conf) {
>> > >
>> > >     String slgClassesCommaSeparated =
>> > > conf.get(VISIBILITY_LABEL_GENERATOR_CLASS);
>> > >
>> > >     List<ScanLabelGenerator> slgs = new
>> ArrayList<ScanLabelGenerator>();
>> > >
>> > >     if (StringUtils.isNotEmpty(slgClassesCommaSeparated)) {
>> > >
>> > >       .......
>> > >
>> > >     }
>> > >
>> > >     // If no SLG is specified in conf, by default we'll add two SLGs
>> > >
>> > >     if (slgs.isEmpty()) {
>> > >
>> > >
>> > >
>> >
>> slgs.add(ReflectionUtils.newInstance(FeedUserAuthScanLabelGenerator.class,
>> > > conf));
>> > >
>> > >
>> > >
>> >
>> slgs.add(ReflectionUtils.newInstance(DefinedSetFilterScanLabelGenerator.class,
>> > > conf));
>> > >
>> > >     }
>> > >
>> > > }
>> > >
>> > > 3:class : FeedUserAuthScanLabelGenerator.java    constructer
>> > > method: FeedUserAuthScanLabelGenerator()
>> > >
>> > >   public FeedUserAuthScanLabelGenerator() {
>> > >
>> > >     this.labelsCache = VisibilityLabelsCache.get();
>> > >
>> > >   }
>> > >
>> > > finally ,  I have found that  the instance of "VisibilityLabelsCache"
>> had
>> > > never not been inited.
>> > >
>> > > But the default set "DefaultVisibilityLabelServiceImpl" is ok, then I
>> > > compare ExpAsStringVisibilityLabelServiceImpl with
>> > > DefaultVisibilityLabelServiceImpl  ,  I found the default class of "
>> > > DefaultVisibilityLabelServiceImpl "  called the method  of
>> "createAndGet"
>> > > to instance:
>> > >
>> > >   public void init(RegionCoprocessorEnvironment e) throws IOException
>> {
>> > >
>> > >     ZooKeeperWatcher zk = e.getRegionServerServices().getZooKeeper();
>> > >
>> > >     try {
>> > >
>> > >       labelsCache = VisibilityLabelsCache.createAndGet(zk, this.conf);
>> > >
>> > >     } catch (IOException ioe) {
>> > >
>> > >       LOG.error("Error creating VisibilityLabelsCache", ioe);
>> > >
>> > >       throw ioe;
>> > >
>> > >     }
>> > >
>> > >     this.scanLabelGenerators =
>> > > VisibilityUtils.getScanLabelGenerators(this.conf);
>> > >
>> > >     .........
>> > >
>> > > }
>> > >
>> > > So I think the class of "ExpAsStringVisibilityLabelServiceImpl" also
>> lack
>> > > of these codes,  then add the method "createAndGet" to the class "
>> > > ExpAsStringVisibilityLabelServiceImpl" and rebuilt it,  replace it to
>> the
>> > > my cluster,  At last ,  My cluster is ok  .
>> > >
>> > > Next days, I fount this problem exists in 0.98.9.  But I found the
>> class
>> > ExpAsStringVisibilityLabelServiceImpl
>> > >  is  always located in maven project: src/test, So I think it may be
>> in
>> > > test moment.
>> > >
>> > >  Is this a bug ?  Any comments/suggestions for work around?
>> > >
>> > >
>> > > Thanks all.
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > 来自网易手机号码邮箱了解更多 <http://shouji.163.com>
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message