hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anoop John <anoop.hb...@gmail.com>
Subject Re: Unexpected behaviour when VisibilityController coprocessor is used
Date Tue, 13 Oct 2015 05:23:05 GMT
Its our pleasure to help you guys.  No problem. Do write back with issues
if u find any.  :-)


Anoop

On Tuesday, October 13, 2015, Suresh Subbiah <suresh.subbiah60@gmail.com>
wrote:
> Thank you Anoop, Ram, Ted.
> Yes it is likely due to https://issues.apache.org/jira/browse/HBASE-13734
>
> Narendra Goyal from our team verified a short while ago that 1.0.2 indeed
> does not have this problem.
> This is for the Trafodion SQL engine. We have some transactional coprocs
> that run on 1.0.0 but are yet to be tested with 1.0.2.
> We will do that now.
>
> Thank you very much for your help. At the Trafodion incubation user dlist
> we try to learn from you'll and be as responsive and helpful. Thank you
for
> being such an excellent role model.
>
> BTW we were able to use the suggestions from the previous round of email
> exchanges and use HFile utility to see the tags associated with a
> visibility label.
>
> Thank you
> Suresh
>
>
>
> On Mon, Oct 12, 2015 at 10:43 PM, ramkrishna vasudevan <
> ramkrishna.s.vasudevan@gmail.com> wrote:
>
>> I tried it on the latest trunk and this issue is not there. So as Anoop
>> said the latest version of 0.98 should be solving this problem.
>> @Suresh
>> Let us know if you still find the issue in later versions of 0.98 and we
>> can work on it to solve the problem.
>>
>> Regards
>> Ram
>>
>> On Tue, Oct 13, 2015 at 9:09 AM, Anoop John <anoop.hbase@gmail.com>
wrote:
>>
>> > Yes as such there is not mandatory to use AC along with VC.  It can be
>> used
>> > alone..
>> > I believe u r getting the bug HBASE-13734.  This is fixed in 98.13
only.
>> > Just change ur version from 98.6 to 98.13 and test once.   Let us know
>> how
>> > is it then.
>> >
>> > -Anoop-
>> >
>> > On Tue, Oct 13, 2015 at 9:01 AM, ramkrishna vasudevan <
>> > ramkrishna.s.vasudevan@gmail.com> wrote:
>> >
>> > > I think, even with only configuring VisibilityController there should
>> not
>> > > be a different behaviour, considering the fact that there are no
>> > visibility
>> > > labels.  With just VisibilityController configured and doing puts and
>> > scans
>> > > using super user let me check what is happening.
>> > >
>> > > Regards
>> > > Ram
>> > >
>> > > On Tue, Oct 13, 2015 at 8:47 AM, Anoop John <anoop.hbase@gmail.com>
>> > wrote:
>> > >
>> > > > Hi Suresh
>> > > >    You said abt doing test as an HBase super user.  You mean even
>> when
>> > > scan
>> > > > is issues as a super user, u are not getting the rows back?
>> > > >
>> > > > -Anoop-
>> > > >
>> > > > On Tue, Oct 13, 2015 at 4:06 AM, Ted Yu <yuzhihong@gmail.com>
wrote:
>> > > >
>> > > > > Convention is to put AccessController ahead of
VisibilityController
>> > in
>> > > > > hbase-site.xml
>> > > > >
>> > > > > Took a quick pass over region server log but haven't found much
>> yet.
>> > > > >
>> > > > > FYI
>> > > > >
>> > > > > On Mon, Oct 12, 2015 at 3:28 PM, Suresh Subbiah <
>> > > > > suresh.subbiah60@gmail.com>
>> > > > > wrote:
>> > > > >
>> > > > > > Hi Ted,
>> > > > > >
>> > > > > > Thank you. Yes HDFS cluster has also been kerberized. BTW,
this
>> is
>> > a
>> > > > > > "cluster" with only one node.
>> > > > > >
>> > > > > > Master hbase-site.xml, RS hbase-site.ml and RS log for the
time
>> > > > interval
>> > > > > > test was run is attached
>> > > > > >
>> > > > > > http://pastebin.com/zuqCC4xG
>> > > > > > http://pastebin.com/88Wx0KDf
>> > > > > > http://pastebin.com/QZqihN1W
>> > > > > >
>> > > > > > Will try deploying 1.1.2 next.
>> > > > > >
>> > > > > > Thanks
>> > > > > > Suresh
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > On Mon, Oct 12, 2015 at 3:46 PM, Ted Yu <yuzhihong@gmail.com>
>> > wrote:
>> > > > > >
>> > > > > > > bq. cluster enabled for secure HBase with kerberos
>> > > > > > >
>> > > > > > > I assume your hdfs cluster has also been kerberized.
>> > > > > > >
>> > > > > > > Please pastebin the complete hbase-site.xml
>> > > > > > >
>> > > > > > > Please turn on DEBUG logging and pastebin the region
server
log
>> > > which
>> > > > > > hosts
>> > > > > > > visibilityTest
>> > > > > > >
>> > > > > > > BTW if possible, can you deploy 1.1.2 ?
>> > > > > > >
>> > > > > > > Cheers
>> > > > > > >
>> > > > > > > On Mon, Oct 12, 2015 at 1:14 PM, Suresh Subbiah <
>> > > > > > > suresh.subbiah60@gmail.com>
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > > > Hi Ted,
>> > > > > > > >
>> > > > > > > > I understand that using VisibilityController on
an
unsercure
>> > > > cluster
>> > > > > is
>> > > > > > > of
>> > > > > > > > limited value. I am still in the early stages
of my task. I
>> am
>> > > > logged
>> > > > > > in
>> > > > > > > as
>> > > > > > > > HBase super user and was simply checking if rows
could be
>> > > accessed.
>> > > > > > > >
>> > > > > > > > With my colleague's help we did get the cluster
enabled for
>> > > secure
>> > > > > > HBase
>> > > > > > > > with kerberos. I repeated the test to get the
same result.
>> Our
>> > > > > cluster
>> > > > > > is
>> > > > > > > > on 1.0. Do you think I may be doing something
incorrectly?
>> What
>> > > > > > > information
>> > > > > > > > can I send to help ensure that I have not made
a mistake.
>> > > > > > > >
>> > > > > > > > Thanks
>> > > > > > > > Suresh
>> > > > > > > >
>> > > > > > > > hbase shell
>> > > > > > > > 15/10/12 14:35:09 INFO Configuration.deprecation:
>> > > hadoop.native.lib
>> > > > > is
>> > > > > > > > deprecated. Instead, use io.native.lib.available
>> > > > > > > > HBase Shell; enter 'help<RETURN>' for list
of supported
>> > commands.
>> > > > > > > > Type "exit<RETURN>" to leave the HBase Shell
>> > > > > > > > Version 1.0.0-cdh5.4.4, rUnknown, Mon Jul  6 16:59:55
PDT
>> 2015
>> > > > > > > >
>> > > > > > > > hbase(main):001:0> create 'visibilityTest',
'f1'
>> > > > > > > > 0 row(s) in 0.7780 seconds
>> > > > > > > >
>> > > > > > > > => Hbase::Table - visibilityTest
>> > > > > > > > hbase(main):002:0> put 'visibilityTest', 'r1',
'f1:c1',
>> > 'value1'
>> > > > > > > > 0 row(s) in 0.1300 seconds
>> > > > > > > >
>> > > > > > > > hbase(main):003:0> deleteall 'visibilityTest',
'r1'
>> > > > > > > > 0 row(s) in 0.0330 seconds
>> > > > > > > >
>> > > > > > > > hbase(main):004:0> put 'visibilityTest', 'r1',
'f1:c1',
>> > 'value2'
>> > > > > > > > 0 row(s) in 0.0150 seconds
>> > > > > > > >
>> > > > > > > > hbase(main):005:0> scan 'visibilityTest'
>> > > > > > > > ROW                   COLUMN+CELL
>> > > > > > > >
>> > > > > > > > 0 row(s) in 0.0550 seconds
>> > > > > > > >
>> > > > > > > > hbase(main):006:0> scan 'visibilityTest', {RAW=>TRUE}
>> > > > > > > > ROW                   COLUMN+CELL
>> > > > > > > >
>> > > > > > > >  r1                   column=f1:, timestamp=1444660561138,
>> > > > > > > > type=DeleteFamily
>> > > > > > > >  r1                   column=f1:c1,
timestamp=1444660576868,
>> > > > > > value=value2
>> > > > > > > >
>> > > > > > > > 1 row(s) in 0.0370 seconds
>> > > > > > > >
>> > > > > > > > -----------------------------------------------------
>> > > > > > > > <property>
>> > > > > > > >     <name>hbase.coprocessor.master.classes</name>
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
<value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.access.AccessController</value>
>> > > > > > > >   </property>
>> > > > > > > >
>> > > > > > > > <property>
>> > > > > > > >     <name>hbase.coprocessor.region.classes</name>
>> > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
<value>org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController</value>
>> > > > > > > >   </property>
>> > > > > > > >
>> > > > > > > > --------------------------------------------------------
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > On Sat, Oct 10, 2015 at 9:51 PM, Ted Yu <
yuzhihong@gmail.com
>> >
>> > > > wrote:
>> > > > > > > >
>> > > > > > > > > To my understanding, VisibilityController
is used in a
>> secure
>> > > > > > cluster.
>> > > > > > > > > Without security, how do you enforce that
only select
>> user(s)
>> > > can
>> > > > > > > access
>> > > > > > > > > certain cells ?
>> > > > > > > > >
>> > > > > > > > > Please see the following sections in refguide:
>> > > > > > > > >
>> > > > > > > > >
>> http://hbase.apache.org/book.html#hbase.secure.configuration
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
http://hbase.apache.org/book.html#_server_side_configuration_for_simple_user_access_operation
>> > > > > > > > >
>> > > > > > > > > On Sat, Oct 10, 2015 at 7:40 PM, Suresh Subbiah
<
>> > > > > > > > > suresh.subbiah60@gmail.com>
>> > > > > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > > > Hi Ted,
>> > > > > > > > > >
>> > > > > > > > > > Thank you for your response.
>> > > > > > > > > > I found a machine with HBase 1.0.0 and
tried the script
>> > with
>> > > > all
>> > > > > 6
>> > > > > > > > > coprocs
>> > > > > > > > > > you listed (2 in master, and 4 in RS).
I still do not
see
>> > the
>> > > > row
>> > > > > > > after
>> > > > > > > > > the
>> > > > > > > > > > second scan.
>> > > > > > > > > >
>> > > > > > > > > > However my cluster is not secure enabled
I think. Is
that
>> > > > > > necessary?
>> > > > > > > I
>> > > > > > > > am
>> > > > > > > > > > not sure how to do that, though I can
ask other members
>> of
>> > my
>> > > > > team
>> > > > > > > and
>> > > > > > > > > try
>> > > > > > > > > > it if that will help.
>> > > > > > > > > >
>> > > > > > > > > > It will be ideal if we could get this
to work on a 1.0
>> > based
>> > > > > > version.
>> > > > > > > > > > Moving to 1.1 will take more time since
we have some
>> > > > > dependencies.
>> > > > > > > > > >
>> > > > > > > > > > Thank you
>> > > > > > > > > > Suresh
>> > > > > > > > > >
>> > > > > > > > > > 15/10/10 19:20:44 INFO Configuration.deprecation:
>> > > > > hadoop.native.lib
>> > > > > > > is
>> > > > > > > > > > deprecated. Instead, use io.native.lib.available
>> > > > > > > > > > HBase Shell; enter 'help<RETURN>'
for list of supported
>> > > > commands.
>> > > > > > > > > > Type "exit<RETURN>" to leave the
HBase Shell
>> > > > > > > > > > Version 1.0.0-cdh5.4.4, rUnknown, Mon
Jul  6 16:59:55
PDT
>> > > 2015
>> > > > > > > > > >
>> > > > > > > > > > *hbase(main):001:0> create 'visibilityTest',
'f1' *
>> > > > > > > > > > *0 row(s) in 0.5460 seconds*
>> > > > > > > > > >
>> > > > > > > > > > *=> Hbase::Table - visibilityTest*
>> > > > > > > > > > *hbase(main):002:0> put 'visibilityTest',
'r1',
'f1:c1',
>> > > > > 'value1' *
>> > > > > > > > > > *0 row(s) in 0.0670 seconds*
>> > > > > > > > > >
>> > > > > > > > > > *hbase(main):003:0> deleteall 'visibilityTest',
'r1' *
>> > > > > > > > > > *0 row(s) in 0.0090 seconds*
>> > > > > > > > > >
>> > > > > > > > > > *hbase(main):004:0> put 'visibilityTest',
'r1',
'f1:c1',
>> > > > > 'value2'*
>> > > > > > > > > > *0 row(s) in 0.0040 seconds*
>> > > > > > > > > >
>> > > > > > > > > > *hbase(main):005:0> scan 'visibilityTest'*
>> > > > > > > > > > *ROW                   COLUMN+CELL
>> > > > > > > > > >       *
>> > > > > > > > > > *0 row(s) in 0.0160 seconds*
>> > > > > > > > > >
>> > > > > > > > > > *hbase(main):006:0> scan 'visibilityTest',
{RAW=>TRUE}*
>> > > > > > > > > > *ROW                   COLUMN+CELL
>> > > > > > > > > >       *
>> > > > > > > > > > * r1                   column=f1:,
>> timestamp=1444530064056,
>> > > > > > > > > > type=DeleteFamily    *
>> > > > > > > > > > * r1                   column=f1:c1,
>> > timestamp=1444530064084,
>> > > > > > > > > value=value2
>> > > > > > > > > >       *
>> > > > > > > > > > *1 row(s) in 0.0580 seconds*
>> > > > > > > > > >
>> > > > > > > > > > *hbase(main):007:0> exit*
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > On Sat, Oct 10, 2015 at 7:26 PM, Ted
Yu <
>> > yuzhihong@gmail.com
>> > > >
>> > > > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > > I tried the sequence of commands
from your example
on a
>> > > > secure
>> > > > > > > 1.1.2
>> > > > > > > > > > > cluster with the following config:
>> > > > > > > > > > >
>> > > > > > > > > > >     <property>
>> > > > > > > > > > >       <name>hbase.coprocessor.master.classes</name>
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
<value>org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
>> > > > > > > > > > >     </property>
>> > > > > > > > > > >     <property>
>> > > > > > > > > > >       <name>hbase.coprocessor.region.classes</name>
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
<value>org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
>> > > > > > > > > > >     </property>
>> > > > > > > > > > >
>> > > > > > > > > > > I got:
>> > > > > > > > > > >
>> > > > > > > > > > > hbase(main):005:0> scan 'visibilityTest'
>> > > > > > > > > > > ROW
>>  COLUMN+CELL
>> > > > > > > > > > >  r1
>> >  column=f1:c1,
>> > > > > > > > > > > timestamp=1444522994981, value=value2
>> > > > > > > > > > > 1 row(s) in 0.1020 seconds
>> > > > > > > > > > >
>> > > > > > > > > > > Can you try again with 0.98.15
release whose vote
>> passed
>> > > > Friday
>> > > > > > to
>> > > > > > > > see
>> > > > > > > > > if
>> > > > > > > > > > > what you observed can be reproduced
?
>> > > > > > > > > > >
>> > > > > > > > > > > Cheers
>> > > > > > > > > > >
>> > > > > > > > > > > On Sat, Oct 10, 2015 at 3:58 PM,
Suresh Subbiah <
>> > > > > > > > > > > suresh.subbiah60@gmail.com>
>> > > > > > > > > > > wrote:
>> > > > > > > > > > >
>> > > > > > > > > > > > Hi,
>> > > > > > > > > > > >
>> > > > > > > > > > > > When I run the following script
from hbase shell
the
>> > last
>> > > > > scan
>> > > > > > > > > returns
>> > > > > > > > > > no
>> > > > > > > > > > > > rows
>> > > > > > > > > > > >
>> > > > > > > > > > > > create 'visibilityTest', 'f1'
>> > > > > > > > > > > > put 'visibilityTest', 'r1',
'f1:c1', 'value1'
>> > > > > > > > > > > > deleteall 'visibilityTest',
'r1'
>> > > > > > > > > > > > put 'visibilityTest', 'r1',
'f1:c1', 'value2'
>> > > > > > > > > > > > scan 'visibilityTest'
>> > > > > > > > > > > >
>> > > > > > > > > > > > *hbase(main):013:0> scan
'visibilityTest'*
>> > > > > > > > > > > > *ROW                   COLUMN+CELL
>> > > > > > > > > > > >       *
>> > > > > > > > > > > > *0 row(s) in 0.0100 seconds*
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > > However if I run
>> > > > > > > > > > > > scan 'visibilityTest' , {RAW=>TRUE}
>> > > > > > > > > > > >
>> > > > > > > > > > > > I see that the second row
that I put is indeed
there
>> > and
>> > > > has
>> > > > > a
>> > > > > > > > > > timestamp
>> > > > > > > > > > > > value higher that the previous
delete
>> > > > > > > > > > > >
>> > > > > > > > > > > > *hbase(main):014:0> scan
'visibilityTest',
>> {RAW=>TRUE}*
>> > > > > > > > > > > > *ROW                   COLUMN+CELL
>> > > > > > > > > > > >       *
>> > > > > > > > > > > > * r1                   column=f1:,
>> > > timestamp=1444516578296,
>> > > > > > > > > > > > type=DeleteFamily    *
>> > > > > > > > > > > > * r1                   column=f1:c1,
>> > > > timestamp=1444516647655,
>> > > > > > > > > > > value=value2
>> > > > > > > > > > > >       *
>> > > > > > > > > > > > *1 row(s) in 0.0110 seconds*
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > > This is on hbase 0.98.6. 
Problem is seen only when
>> > > > > > > hbase-site.xml
>> > > > > > > > > has
>> > > > > > > > > > > > these lines. No other coprocessors
were used during
>> > this
>> > > > > test.
>> > > > > > > > > > > >
>> > > > > > > > > > > > <property>
>> > > > > > > > > > > >     <name>hbase.coprocessor.region.classes</name>
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
<value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
>> > > > > > > > > > > >    </property>
>> > > > > > > > > > > >    <property>
>> > > > > > > > > > > >      <name>hbase.coprocessor.master.classes</name>
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
<value>org.apache.hadoop.hbase.security.visibility.VisibilityController</value>
>> > > > > > > > > > > >    </property>
>> > > > > > > > > > > >    <property>
>> > > > > > > > > > > >      <name>hfile.format.version</name>
>> > > > > > > > > > > >      <value>3</value>
>> > > > > > > > > > > >    </property>
>> > > > > > > > > > > >
>> > > > > > > > > > > > Any suggestions of what I
may be doing incorrectly?
>> Or
>> > is
>> > > > > this
>> > > > > > a
>> > > > > > > > bug?
>> > > > > > > > > > > >
>> > > > > > > > > > > > Thank you
>> > > > > > > > > > > > Suresh
>> > > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message