hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anoop John <anoop.hb...@gmail.com>
Subject Re: Unexpected behaviour when VisibilityController coprocessor is used
Date Tue, 13 Oct 2015 03:39:54 GMT
Yes as such there is not mandatory to use AC along with VC.  It can be used
alone..
I believe u r getting the bug HBASE-13734.  This is fixed in 98.13 only.
Just change ur version from 98.6 to 98.13 and test once.   Let us know how
is it then.

-Anoop-

On Tue, Oct 13, 2015 at 9:01 AM, ramkrishna vasudevan <
ramkrishna.s.vasudevan@gmail.com> wrote:

> I think, even with only configuring VisibilityController there should not
> be a different behaviour, considering the fact that there are no visibility
> labels.  With just VisibilityController configured and doing puts and scans
> using super user let me check what is happening.
>
> Regards
> Ram
>
> On Tue, Oct 13, 2015 at 8:47 AM, Anoop John <anoop.hbase@gmail.com> wrote:
>
> > Hi Suresh
> >    You said abt doing test as an HBase super user.  You mean even when
> scan
> > is issues as a super user, u are not getting the rows back?
> >
> > -Anoop-
> >
> > On Tue, Oct 13, 2015 at 4:06 AM, Ted Yu <yuzhihong@gmail.com> wrote:
> >
> > > Convention is to put AccessController ahead of VisibilityController in
> > > hbase-site.xml
> > >
> > > Took a quick pass over region server log but haven't found much yet.
> > >
> > > FYI
> > >
> > > On Mon, Oct 12, 2015 at 3:28 PM, Suresh Subbiah <
> > > suresh.subbiah60@gmail.com>
> > > wrote:
> > >
> > > > Hi Ted,
> > > >
> > > > Thank you. Yes HDFS cluster has also been kerberized. BTW, this is a
> > > > "cluster" with only one node.
> > > >
> > > > Master hbase-site.xml, RS hbase-site.ml and RS log for the time
> > interval
> > > > test was run is attached
> > > >
> > > > http://pastebin.com/zuqCC4xG
> > > > http://pastebin.com/88Wx0KDf
> > > > http://pastebin.com/QZqihN1W
> > > >
> > > > Will try deploying 1.1.2 next.
> > > >
> > > > Thanks
> > > > Suresh
> > > >
> > > >
> > > >
> > > > On Mon, Oct 12, 2015 at 3:46 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> > > >
> > > > > bq. cluster enabled for secure HBase with kerberos
> > > > >
> > > > > I assume your hdfs cluster has also been kerberized.
> > > > >
> > > > > Please pastebin the complete hbase-site.xml
> > > > >
> > > > > Please turn on DEBUG logging and pastebin the region server log
> which
> > > > hosts
> > > > > visibilityTest
> > > > >
> > > > > BTW if possible, can you deploy 1.1.2 ?
> > > > >
> > > > > Cheers
> > > > >
> > > > > On Mon, Oct 12, 2015 at 1:14 PM, Suresh Subbiah <
> > > > > suresh.subbiah60@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Ted,
> > > > > >
> > > > > > I understand that using VisibilityController on an unsercure
> > cluster
> > > is
> > > > > of
> > > > > > limited value. I am still in the early stages of my task. I
am
> > logged
> > > > in
> > > > > as
> > > > > > HBase super user and was simply checking if rows could be
> accessed.
> > > > > >
> > > > > > With my colleague's help we did get the cluster enabled for
> secure
> > > > HBase
> > > > > > with kerberos. I repeated the test to get the same result. Our
> > > cluster
> > > > is
> > > > > > on 1.0. Do you think I may be doing something incorrectly? What
> > > > > information
> > > > > > can I send to help ensure that I have not made a mistake.
> > > > > >
> > > > > > Thanks
> > > > > > Suresh
> > > > > >
> > > > > > hbase shell
> > > > > > 15/10/12 14:35:09 INFO Configuration.deprecation:
> hadoop.native.lib
> > > is
> > > > > > deprecated. Instead, use io.native.lib.available
> > > > > > HBase Shell; enter 'help<RETURN>' for list of supported
commands.
> > > > > > Type "exit<RETURN>" to leave the HBase Shell
> > > > > > Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul  6 16:59:55 PDT 2015
> > > > > >
> > > > > > hbase(main):001:0> create 'visibilityTest', 'f1'
> > > > > > 0 row(s) in 0.7780 seconds
> > > > > >
> > > > > > => Hbase::Table - visibilityTest
> > > > > > hbase(main):002:0> put 'visibilityTest', 'r1', 'f1:c1', 'value1'
> > > > > > 0 row(s) in 0.1300 seconds
> > > > > >
> > > > > > hbase(main):003:0> deleteall 'visibilityTest', 'r1'
> > > > > > 0 row(s) in 0.0330 seconds
> > > > > >
> > > > > > hbase(main):004:0> put 'visibilityTest', 'r1', 'f1:c1', 'value2'
> > > > > > 0 row(s) in 0.0150 seconds
> > > > > >
> > > > > > hbase(main):005:0> scan 'visibilityTest'
> > > > > > ROW                   COLUMN+CELL
> > > > > >
> > > > > > 0 row(s) in 0.0550 seconds
> > > > > >
> > > > > > hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}
> > > > > > ROW                   COLUMN+CELL
> > > > > >
> > > > > >  r1                   column=f1:, timestamp=1444660561138,
> > > > > > type=DeleteFamily
> > > > > >  r1                   column=f1:c1, timestamp=1444660576868,
> > > > value=value2
> > > > > >
> > > > > > 1 row(s) in 0.0370 seconds
> > > > > >
> > > > > > -----------------------------------------------------
> > > > > > <property>
> > > > > >     <name>hbase.coprocessor.master.classes</name>
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.access.AccessController</value>
> > > > > >   </property>
> > > > > >
> > > > > > <property>
> > > > > >     <name>hbase.coprocessor.region.classes</name>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController</value>
> > > > > >   </property>
> > > > > >
> > > > > > --------------------------------------------------------
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Sat, Oct 10, 2015 at 9:51 PM, Ted Yu <yuzhihong@gmail.com>
> > wrote:
> > > > > >
> > > > > > > To my understanding, VisibilityController is used in a
secure
> > > > cluster.
> > > > > > > Without security, how do you enforce that only select user(s)
> can
> > > > > access
> > > > > > > certain cells ?
> > > > > > >
> > > > > > > Please see the following sections in refguide:
> > > > > > >
> > > > > > > http://hbase.apache.org/book.html#hbase.secure.configuration
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://hbase.apache.org/book.html#_server_side_configuration_for_simple_user_access_operation
> > > > > > >
> > > > > > > On Sat, Oct 10, 2015 at 7:40 PM, Suresh Subbiah <
> > > > > > > suresh.subbiah60@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Ted,
> > > > > > > >
> > > > > > > > Thank you for your response.
> > > > > > > > I found a machine with HBase 1.0.0 and tried the script
with
> > all
> > > 6
> > > > > > > coprocs
> > > > > > > > you listed (2 in master, and 4 in RS). I still do
not see the
> > row
> > > > > after
> > > > > > > the
> > > > > > > > second scan.
> > > > > > > >
> > > > > > > > However my cluster is not secure enabled I think.
Is that
> > > > necessary?
> > > > > I
> > > > > > am
> > > > > > > > not sure how to do that, though I can ask other members
of my
> > > team
> > > > > and
> > > > > > > try
> > > > > > > > it if that will help.
> > > > > > > >
> > > > > > > > It will be ideal if we could get this to work on a
1.0 based
> > > > version.
> > > > > > > > Moving to 1.1 will take more time since we have some
> > > dependencies.
> > > > > > > >
> > > > > > > > Thank you
> > > > > > > > Suresh
> > > > > > > >
> > > > > > > > 15/10/10 19:20:44 INFO Configuration.deprecation:
> > > hadoop.native.lib
> > > > > is
> > > > > > > > deprecated. Instead, use io.native.lib.available
> > > > > > > > HBase Shell; enter 'help<RETURN>' for list of
supported
> > commands.
> > > > > > > > Type "exit<RETURN>" to leave the HBase Shell
> > > > > > > > Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul  6 16:59:55
PDT
> 2015
> > > > > > > >
> > > > > > > > *hbase(main):001:0> create 'visibilityTest', 'f1'
*
> > > > > > > > *0 row(s) in 0.5460 seconds*
> > > > > > > >
> > > > > > > > *=> Hbase::Table - visibilityTest*
> > > > > > > > *hbase(main):002:0> put 'visibilityTest', 'r1',
'f1:c1',
> > > 'value1' *
> > > > > > > > *0 row(s) in 0.0670 seconds*
> > > > > > > >
> > > > > > > > *hbase(main):003:0> deleteall 'visibilityTest',
'r1' *
> > > > > > > > *0 row(s) in 0.0090 seconds*
> > > > > > > >
> > > > > > > > *hbase(main):004:0> put 'visibilityTest', 'r1',
'f1:c1',
> > > 'value2'*
> > > > > > > > *0 row(s) in 0.0040 seconds*
> > > > > > > >
> > > > > > > > *hbase(main):005:0> scan 'visibilityTest'*
> > > > > > > > *ROW                   COLUMN+CELL
> > > > > > > >       *
> > > > > > > > *0 row(s) in 0.0160 seconds*
> > > > > > > >
> > > > > > > > *hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}*
> > > > > > > > *ROW                   COLUMN+CELL
> > > > > > > >       *
> > > > > > > > * r1                   column=f1:, timestamp=1444530064056,
> > > > > > > > type=DeleteFamily    *
> > > > > > > > * r1                   column=f1:c1, timestamp=1444530064084,
> > > > > > > value=value2
> > > > > > > >       *
> > > > > > > > *1 row(s) in 0.0580 seconds*
> > > > > > > >
> > > > > > > > *hbase(main):007:0> exit*
> > > > > > > >
> > > > > > > >
> > > > > > > > On Sat, Oct 10, 2015 at 7:26 PM, Ted Yu <yuzhihong@gmail.com
> >
> > > > wrote:
> > > > > > > >
> > > > > > > > > I tried the sequence of commands from your example
on a
> > secure
> > > > > 1.1.2
> > > > > > > > > cluster with the following config:
> > > > > > > > >
> > > > > > > > >     <property>
> > > > > > > > >       <name>hbase.coprocessor.master.classes</name>
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > > >     </property>
> > > > > > > > >     <property>
> > > > > > > > >       <name>hbase.coprocessor.region.classes</name>
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > > >     </property>
> > > > > > > > >
> > > > > > > > > I got:
> > > > > > > > >
> > > > > > > > > hbase(main):005:0> scan 'visibilityTest'
> > > > > > > > > ROW                                         
 COLUMN+CELL
> > > > > > > > >  r1                                         
 column=f1:c1,
> > > > > > > > > timestamp=1444522994981, value=value2
> > > > > > > > > 1 row(s) in 0.1020 seconds
> > > > > > > > >
> > > > > > > > > Can you try again with 0.98.15 release whose
vote passed
> > Friday
> > > > to
> > > > > > see
> > > > > > > if
> > > > > > > > > what you observed can be reproduced ?
> > > > > > > > >
> > > > > > > > > Cheers
> > > > > > > > >
> > > > > > > > > On Sat, Oct 10, 2015 at 3:58 PM, Suresh Subbiah
<
> > > > > > > > > suresh.subbiah60@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > > When I run the following script from hbase
shell the last
> > > scan
> > > > > > > returns
> > > > > > > > no
> > > > > > > > > > rows
> > > > > > > > > >
> > > > > > > > > > create 'visibilityTest', 'f1'
> > > > > > > > > > put 'visibilityTest', 'r1', 'f1:c1', 'value1'
> > > > > > > > > > deleteall 'visibilityTest', 'r1'
> > > > > > > > > > put 'visibilityTest', 'r1', 'f1:c1', 'value2'
> > > > > > > > > > scan 'visibilityTest'
> > > > > > > > > >
> > > > > > > > > > *hbase(main):013:0> scan 'visibilityTest'*
> > > > > > > > > > *ROW                   COLUMN+CELL
> > > > > > > > > >       *
> > > > > > > > > > *0 row(s) in 0.0100 seconds*
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > However if I run
> > > > > > > > > > scan 'visibilityTest' , {RAW=>TRUE}
> > > > > > > > > >
> > > > > > > > > > I see that the second row that I put is
indeed there and
> > has
> > > a
> > > > > > > > timestamp
> > > > > > > > > > value higher that the previous delete
> > > > > > > > > >
> > > > > > > > > > *hbase(main):014:0> scan 'visibilityTest',
{RAW=>TRUE}*
> > > > > > > > > > *ROW                   COLUMN+CELL
> > > > > > > > > >       *
> > > > > > > > > > * r1                   column=f1:,
> timestamp=1444516578296,
> > > > > > > > > > type=DeleteFamily    *
> > > > > > > > > > * r1                   column=f1:c1,
> > timestamp=1444516647655,
> > > > > > > > > value=value2
> > > > > > > > > >       *
> > > > > > > > > > *1 row(s) in 0.0110 seconds*
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > This is on hbase 0.98.6.  Problem is seen
only when
> > > > > hbase-site.xml
> > > > > > > has
> > > > > > > > > > these lines. No other coprocessors were
used during this
> > > test.
> > > > > > > > > >
> > > > > > > > > > <property>
> > > > > > > > > >     <name>hbase.coprocessor.region.classes</name>
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > > > >    </property>
> > > > > > > > > >    <property>
> > > > > > > > > >      <name>hbase.coprocessor.master.classes</name>
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > > > >    </property>
> > > > > > > > > >    <property>
> > > > > > > > > >      <name>hfile.format.version</name>
> > > > > > > > > >      <value>3</value>
> > > > > > > > > >    </property>
> > > > > > > > > >
> > > > > > > > > > Any suggestions of what I may be doing incorrectly?
Or is
> > > this
> > > > a
> > > > > > bug?
> > > > > > > > > >
> > > > > > > > > > Thank you
> > > > > > > > > > Suresh
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message