hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suresh Subbiah <suresh.subbia...@gmail.com>
Subject Re: Unexpected behaviour when VisibilityController coprocessor is used
Date Tue, 13 Oct 2015 04:23:06 GMT
Thank you Anoop, Ram, Ted.
Yes it is likely due to https://issues.apache.org/jira/browse/HBASE-13734

Narendra Goyal from our team verified a short while ago that 1.0.2 indeed
does not have this problem.
This is for the Trafodion SQL engine. We have some transactional coprocs
that run on 1.0.0 but are yet to be tested with 1.0.2.
We will do that now.

Thank you very much for your help. At the Trafodion incubation user dlist
we try to learn from you'll and be as responsive and helpful. Thank you for
being such an excellent role model.

BTW we were able to use the suggestions from the previous round of email
exchanges and use HFile utility to see the tags associated with a
visibility label.

Thank you
Suresh



On Mon, Oct 12, 2015 at 10:43 PM, ramkrishna vasudevan <
ramkrishna.s.vasudevan@gmail.com> wrote:

> I tried it on the latest trunk and this issue is not there. So as Anoop
> said the latest version of 0.98 should be solving this problem.
> @Suresh
> Let us know if you still find the issue in later versions of 0.98 and we
> can work on it to solve the problem.
>
> Regards
> Ram
>
> On Tue, Oct 13, 2015 at 9:09 AM, Anoop John <anoop.hbase@gmail.com> wrote:
>
> > Yes as such there is not mandatory to use AC along with VC.  It can be
> used
> > alone..
> > I believe u r getting the bug HBASE-13734.  This is fixed in 98.13 only.
> > Just change ur version from 98.6 to 98.13 and test once.   Let us know
> how
> > is it then.
> >
> > -Anoop-
> >
> > On Tue, Oct 13, 2015 at 9:01 AM, ramkrishna vasudevan <
> > ramkrishna.s.vasudevan@gmail.com> wrote:
> >
> > > I think, even with only configuring VisibilityController there should
> not
> > > be a different behaviour, considering the fact that there are no
> > visibility
> > > labels.  With just VisibilityController configured and doing puts and
> > scans
> > > using super user let me check what is happening.
> > >
> > > Regards
> > > Ram
> > >
> > > On Tue, Oct 13, 2015 at 8:47 AM, Anoop John <anoop.hbase@gmail.com>
> > wrote:
> > >
> > > > Hi Suresh
> > > >    You said abt doing test as an HBase super user.  You mean even
> when
> > > scan
> > > > is issues as a super user, u are not getting the rows back?
> > > >
> > > > -Anoop-
> > > >
> > > > On Tue, Oct 13, 2015 at 4:06 AM, Ted Yu <yuzhihong@gmail.com> wrote:
> > > >
> > > > > Convention is to put AccessController ahead of VisibilityController
> > in
> > > > > hbase-site.xml
> > > > >
> > > > > Took a quick pass over region server log but haven't found much
> yet.
> > > > >
> > > > > FYI
> > > > >
> > > > > On Mon, Oct 12, 2015 at 3:28 PM, Suresh Subbiah <
> > > > > suresh.subbiah60@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Ted,
> > > > > >
> > > > > > Thank you. Yes HDFS cluster has also been kerberized. BTW, this
> is
> > a
> > > > > > "cluster" with only one node.
> > > > > >
> > > > > > Master hbase-site.xml, RS hbase-site.ml and RS log for the time
> > > > interval
> > > > > > test was run is attached
> > > > > >
> > > > > > http://pastebin.com/zuqCC4xG
> > > > > > http://pastebin.com/88Wx0KDf
> > > > > > http://pastebin.com/QZqihN1W
> > > > > >
> > > > > > Will try deploying 1.1.2 next.
> > > > > >
> > > > > > Thanks
> > > > > > Suresh
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Mon, Oct 12, 2015 at 3:46 PM, Ted Yu <yuzhihong@gmail.com>
> > wrote:
> > > > > >
> > > > > > > bq. cluster enabled for secure HBase with kerberos
> > > > > > >
> > > > > > > I assume your hdfs cluster has also been kerberized.
> > > > > > >
> > > > > > > Please pastebin the complete hbase-site.xml
> > > > > > >
> > > > > > > Please turn on DEBUG logging and pastebin the region server
log
> > > which
> > > > > > hosts
> > > > > > > visibilityTest
> > > > > > >
> > > > > > > BTW if possible, can you deploy 1.1.2 ?
> > > > > > >
> > > > > > > Cheers
> > > > > > >
> > > > > > > On Mon, Oct 12, 2015 at 1:14 PM, Suresh Subbiah <
> > > > > > > suresh.subbiah60@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Ted,
> > > > > > > >
> > > > > > > > I understand that using VisibilityController on an
unsercure
> > > > cluster
> > > > > is
> > > > > > > of
> > > > > > > > limited value. I am still in the early stages of my
task. I
> am
> > > > logged
> > > > > > in
> > > > > > > as
> > > > > > > > HBase super user and was simply checking if rows could
be
> > > accessed.
> > > > > > > >
> > > > > > > > With my colleague's help we did get the cluster enabled
for
> > > secure
> > > > > > HBase
> > > > > > > > with kerberos. I repeated the test to get the same
result.
> Our
> > > > > cluster
> > > > > > is
> > > > > > > > on 1.0. Do you think I may be doing something incorrectly?
> What
> > > > > > > information
> > > > > > > > can I send to help ensure that I have not made a mistake.
> > > > > > > >
> > > > > > > > Thanks
> > > > > > > > Suresh
> > > > > > > >
> > > > > > > > hbase shell
> > > > > > > > 15/10/12 14:35:09 INFO Configuration.deprecation:
> > > hadoop.native.lib
> > > > > is
> > > > > > > > deprecated. Instead, use io.native.lib.available
> > > > > > > > HBase Shell; enter 'help<RETURN>' for list of
supported
> > commands.
> > > > > > > > Type "exit<RETURN>" to leave the HBase Shell
> > > > > > > > Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul  6 16:59:55
PDT
> 2015
> > > > > > > >
> > > > > > > > hbase(main):001:0> create 'visibilityTest', 'f1'
> > > > > > > > 0 row(s) in 0.7780 seconds
> > > > > > > >
> > > > > > > > => Hbase::Table - visibilityTest
> > > > > > > > hbase(main):002:0> put 'visibilityTest', 'r1',
'f1:c1',
> > 'value1'
> > > > > > > > 0 row(s) in 0.1300 seconds
> > > > > > > >
> > > > > > > > hbase(main):003:0> deleteall 'visibilityTest',
'r1'
> > > > > > > > 0 row(s) in 0.0330 seconds
> > > > > > > >
> > > > > > > > hbase(main):004:0> put 'visibilityTest', 'r1',
'f1:c1',
> > 'value2'
> > > > > > > > 0 row(s) in 0.0150 seconds
> > > > > > > >
> > > > > > > > hbase(main):005:0> scan 'visibilityTest'
> > > > > > > > ROW                   COLUMN+CELL
> > > > > > > >
> > > > > > > > 0 row(s) in 0.0550 seconds
> > > > > > > >
> > > > > > > > hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}
> > > > > > > > ROW                   COLUMN+CELL
> > > > > > > >
> > > > > > > >  r1                   column=f1:, timestamp=1444660561138,
> > > > > > > > type=DeleteFamily
> > > > > > > >  r1                   column=f1:c1, timestamp=1444660576868,
> > > > > > value=value2
> > > > > > > >
> > > > > > > > 1 row(s) in 0.0370 seconds
> > > > > > > >
> > > > > > > > -----------------------------------------------------
> > > > > > > > <property>
> > > > > > > >     <name>hbase.coprocessor.master.classes</name>
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.access.AccessController</value>
> > > > > > > >   </property>
> > > > > > > >
> > > > > > > > <property>
> > > > > > > >     <name>hbase.coprocessor.region.classes</name>
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController</value>
> > > > > > > >   </property>
> > > > > > > >
> > > > > > > > --------------------------------------------------------
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Sat, Oct 10, 2015 at 9:51 PM, Ted Yu <yuzhihong@gmail.com
> >
> > > > wrote:
> > > > > > > >
> > > > > > > > > To my understanding, VisibilityController is
used in a
> secure
> > > > > > cluster.
> > > > > > > > > Without security, how do you enforce that only
select
> user(s)
> > > can
> > > > > > > access
> > > > > > > > > certain cells ?
> > > > > > > > >
> > > > > > > > > Please see the following sections in refguide:
> > > > > > > > >
> > > > > > > > >
> http://hbase.apache.org/book.html#hbase.secure.configuration
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://hbase.apache.org/book.html#_server_side_configuration_for_simple_user_access_operation
> > > > > > > > >
> > > > > > > > > On Sat, Oct 10, 2015 at 7:40 PM, Suresh Subbiah
<
> > > > > > > > > suresh.subbiah60@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi Ted,
> > > > > > > > > >
> > > > > > > > > > Thank you for your response.
> > > > > > > > > > I found a machine with HBase 1.0.0 and tried
the script
> > with
> > > > all
> > > > > 6
> > > > > > > > > coprocs
> > > > > > > > > > you listed (2 in master, and 4 in RS). I
still do not see
> > the
> > > > row
> > > > > > > after
> > > > > > > > > the
> > > > > > > > > > second scan.
> > > > > > > > > >
> > > > > > > > > > However my cluster is not secure enabled
I think. Is that
> > > > > > necessary?
> > > > > > > I
> > > > > > > > am
> > > > > > > > > > not sure how to do that, though I can ask
other members
> of
> > my
> > > > > team
> > > > > > > and
> > > > > > > > > try
> > > > > > > > > > it if that will help.
> > > > > > > > > >
> > > > > > > > > > It will be ideal if we could get this to
work on a 1.0
> > based
> > > > > > version.
> > > > > > > > > > Moving to 1.1 will take more time since
we have some
> > > > > dependencies.
> > > > > > > > > >
> > > > > > > > > > Thank you
> > > > > > > > > > Suresh
> > > > > > > > > >
> > > > > > > > > > 15/10/10 19:20:44 INFO Configuration.deprecation:
> > > > > hadoop.native.lib
> > > > > > > is
> > > > > > > > > > deprecated. Instead, use io.native.lib.available
> > > > > > > > > > HBase Shell; enter 'help<RETURN>'
for list of supported
> > > > commands.
> > > > > > > > > > Type "exit<RETURN>" to leave the HBase
Shell
> > > > > > > > > > Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul
 6 16:59:55 PDT
> > > 2015
> > > > > > > > > >
> > > > > > > > > > *hbase(main):001:0> create 'visibilityTest',
'f1' *
> > > > > > > > > > *0 row(s) in 0.5460 seconds*
> > > > > > > > > >
> > > > > > > > > > *=> Hbase::Table - visibilityTest*
> > > > > > > > > > *hbase(main):002:0> put 'visibilityTest',
'r1', 'f1:c1',
> > > > > 'value1' *
> > > > > > > > > > *0 row(s) in 0.0670 seconds*
> > > > > > > > > >
> > > > > > > > > > *hbase(main):003:0> deleteall 'visibilityTest',
'r1' *
> > > > > > > > > > *0 row(s) in 0.0090 seconds*
> > > > > > > > > >
> > > > > > > > > > *hbase(main):004:0> put 'visibilityTest',
'r1', 'f1:c1',
> > > > > 'value2'*
> > > > > > > > > > *0 row(s) in 0.0040 seconds*
> > > > > > > > > >
> > > > > > > > > > *hbase(main):005:0> scan 'visibilityTest'*
> > > > > > > > > > *ROW                   COLUMN+CELL
> > > > > > > > > >       *
> > > > > > > > > > *0 row(s) in 0.0160 seconds*
> > > > > > > > > >
> > > > > > > > > > *hbase(main):006:0> scan 'visibilityTest',
{RAW=>TRUE}*
> > > > > > > > > > *ROW                   COLUMN+CELL
> > > > > > > > > >       *
> > > > > > > > > > * r1                   column=f1:,
> timestamp=1444530064056,
> > > > > > > > > > type=DeleteFamily    *
> > > > > > > > > > * r1                   column=f1:c1,
> > timestamp=1444530064084,
> > > > > > > > > value=value2
> > > > > > > > > >       *
> > > > > > > > > > *1 row(s) in 0.0580 seconds*
> > > > > > > > > >
> > > > > > > > > > *hbase(main):007:0> exit*
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Sat, Oct 10, 2015 at 7:26 PM, Ted Yu
<
> > yuzhihong@gmail.com
> > > >
> > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > I tried the sequence of commands from
your example on a
> > > > secure
> > > > > > > 1.1.2
> > > > > > > > > > > cluster with the following config:
> > > > > > > > > > >
> > > > > > > > > > >     <property>
> > > > > > > > > > >       <name>hbase.coprocessor.master.classes</name>
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > > > > >     </property>
> > > > > > > > > > >     <property>
> > > > > > > > > > >       <name>hbase.coprocessor.region.classes</name>
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > > > > >     </property>
> > > > > > > > > > >
> > > > > > > > > > > I got:
> > > > > > > > > > >
> > > > > > > > > > > hbase(main):005:0> scan 'visibilityTest'
> > > > > > > > > > > ROW
>  COLUMN+CELL
> > > > > > > > > > >  r1
> >  column=f1:c1,
> > > > > > > > > > > timestamp=1444522994981, value=value2
> > > > > > > > > > > 1 row(s) in 0.1020 seconds
> > > > > > > > > > >
> > > > > > > > > > > Can you try again with 0.98.15 release
whose vote
> passed
> > > > Friday
> > > > > > to
> > > > > > > > see
> > > > > > > > > if
> > > > > > > > > > > what you observed can be reproduced
?
> > > > > > > > > > >
> > > > > > > > > > > Cheers
> > > > > > > > > > >
> > > > > > > > > > > On Sat, Oct 10, 2015 at 3:58 PM, Suresh
Subbiah <
> > > > > > > > > > > suresh.subbiah60@gmail.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hi,
> > > > > > > > > > > >
> > > > > > > > > > > > When I run the following script
from hbase shell the
> > last
> > > > > scan
> > > > > > > > > returns
> > > > > > > > > > no
> > > > > > > > > > > > rows
> > > > > > > > > > > >
> > > > > > > > > > > > create 'visibilityTest', 'f1'
> > > > > > > > > > > > put 'visibilityTest', 'r1', 'f1:c1',
'value1'
> > > > > > > > > > > > deleteall 'visibilityTest', 'r1'
> > > > > > > > > > > > put 'visibilityTest', 'r1', 'f1:c1',
'value2'
> > > > > > > > > > > > scan 'visibilityTest'
> > > > > > > > > > > >
> > > > > > > > > > > > *hbase(main):013:0> scan 'visibilityTest'*
> > > > > > > > > > > > *ROW                   COLUMN+CELL
> > > > > > > > > > > >       *
> > > > > > > > > > > > *0 row(s) in 0.0100 seconds*
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > However if I run
> > > > > > > > > > > > scan 'visibilityTest' , {RAW=>TRUE}
> > > > > > > > > > > >
> > > > > > > > > > > > I see that the second row that
I put is indeed there
> > and
> > > > has
> > > > > a
> > > > > > > > > > timestamp
> > > > > > > > > > > > value higher that the previous
delete
> > > > > > > > > > > >
> > > > > > > > > > > > *hbase(main):014:0> scan 'visibilityTest',
> {RAW=>TRUE}*
> > > > > > > > > > > > *ROW                   COLUMN+CELL
> > > > > > > > > > > >       *
> > > > > > > > > > > > * r1                   column=f1:,
> > > timestamp=1444516578296,
> > > > > > > > > > > > type=DeleteFamily    *
> > > > > > > > > > > > * r1                   column=f1:c1,
> > > > timestamp=1444516647655,
> > > > > > > > > > > value=value2
> > > > > > > > > > > >       *
> > > > > > > > > > > > *1 row(s) in 0.0110 seconds*
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > This is on hbase 0.98.6.  Problem
is seen only when
> > > > > > > hbase-site.xml
> > > > > > > > > has
> > > > > > > > > > > > these lines. No other coprocessors
were used during
> > this
> > > > > test.
> > > > > > > > > > > >
> > > > > > > > > > > > <property>
> > > > > > > > > > > >     <name>hbase.coprocessor.region.classes</name>
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > > > > > >    </property>
> > > > > > > > > > > >    <property>
> > > > > > > > > > > >      <name>hbase.coprocessor.master.classes</name>
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> <value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
> > > > > > > > > > > >    </property>
> > > > > > > > > > > >    <property>
> > > > > > > > > > > >      <name>hfile.format.version</name>
> > > > > > > > > > > >      <value>3</value>
> > > > > > > > > > > >    </property>
> > > > > > > > > > > >
> > > > > > > > > > > > Any suggestions of what I may
be doing incorrectly?
> Or
> > is
> > > > > this
> > > > > > a
> > > > > > > > bug?
> > > > > > > > > > > >
> > > > > > > > > > > > Thank you
> > > > > > > > > > > > Suresh
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message