hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Davidson <adam.david...@bigdatapartnership.com>
Subject Re: doAs with HBase Java API and Apache Ranger
Date Tue, 22 Dec 2015 16:15:05 GMT
Hi Jerry,

I think we have cracked it now, thanks. I noticed the proxy setting on the
HBase server was setup for the wrong user. It wasn't obvious until I
noticed in some debug logs that the impersonation was being forbidden which
helped me identify the issue. Ranger now reports the correct user in the
audit log, so that side all seems fine.

Regards,
Adam

On Tue, 22 Dec 2015 at 15:00 Jerry He <jerryjch@gmail.com> wrote:

> What is the ConectionCache in your code?  Is it org.apache.hadoop.hbase.
> util.ConnectionCache?
>
> As a sample on how to use the ConnectionCache, you can look at this:
>
> https://github.com/apache/hbase/blob/master/hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServletContainer.java
>
> and
>
> https://github.com/apache/hbase/blob/master/hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServlet.java
>
> Pay attention to the connectionCache.setEffectiveUser() call.  No doAs()
> call is used.
>
> An extra note.  You can look at the native HBase audit log in the log
> directory along side with the maser and region server logs.
> If the hbase native audit log shows the same as in Ranger, then you can
> rule out it is the Ranger plug issue.
>
> Hope this helps.
>
> Jerry
>
> On Mon, Dec 21, 2015 at 6:44 AM, Adam Davidson <
> adam.davidson@bigdatapartnership.com> wrote:
>
>> Hi,
>>
>> I'm working alongside Chris on this problem. As Jerry mentioned the user
>> context is obtained when the Connection is created, I've modified the code
>> as below;
>>
>> try {
>>       ugi.doAs(new PriviledgedExceptionAction<Void>() {
>>           @Override
>>           public Void run() throws Exception {
>>                 LOGGER.info("HBase put as user " +
>> ugi.getShortUserName());
>>                 connectionCache.getTable(tableName).put(put);
>>                 return null;
>>           }
>>     });
>>
>> where the connection cache is initialised at start up using the default
>> UserProvider with a secure Configuration. I believe the server side user
>> proxying has been configured.  However, the Ranger audit logs still show
>> the HBase operations being executed by the service user (i.e. the Kerberos
>> principal that runs the rest service). Assuming what I've done is correct,
>> is there anything else you can think of that might be wrong? Is this just a
>> matter for the Ranger team or whomever is responsible for the HBase Ranger
>> plugin?
>>
>> Best Regards,
>> Adam
>>
>> On Sun, 20 Dec 2015 at 03:40 Jerry He <jerryjch@gmail.com> wrote:
>>
>>> To answer your HBase question, user context is obtained when
>>> 'Connection' object is created.  'Table' shares what the 'Connection' has.
>>> Also like Ted mentioned, use ProxyUser needs additional config on the
>>> serer side.
>>>
>>> Jerry
>>>
>>> On Fri, Dec 18, 2015 at 9:37 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>>>
>>>> I talked with a Ranger developer who has read the thread on Ranger
>>>> mailing
>>>> list.
>>>>
>>>> Setting up proxy may require certain steps.
>>>>
>>>> I suggest responding to Bosco's question on the Ranger mailing list (by
>>>> providing related server log, e.g.) - Ranger developers have knowledge
>>>> about HBase.
>>>>
>>>> Cheers
>>>>
>>>> On Fri, Dec 18, 2015 at 9:24 AM, Chris Gent <
>>>> chris.gent@bigdatapartnership.com> wrote:
>>>>
>>>> > Hey Ted,
>>>> >
>>>> > Yeah - they suggested asking over here :-)
>>>> >
>>>> > I think the question is where the user context is set/comes from when
>>>> using
>>>> > the HBase API. It was suggested that it comes when the Table object
>>>> gets
>>>> > created? Or is it right back when the connection is established?
>>>> >
>>>> > --
>>>> > Chris
>>>> >
>>>> >
>>>> >
>>>> > On 18 December 2015 at 17:18, Ted Yu <yuzhihong@gmail.com> wrote:
>>>> >
>>>> > > Have you polled Ranger community with this question ?
>>>> > >
>>>> > > http://ranger.apache.org/mail-lists.html
>>>> > >
>>>> > > Cheers
>>>> > >
>>>> > > On Fri, Dec 18, 2015 at 9:04 AM, Chris Gent <
>>>> > > chris.gent@bigdatapartnership.com> wrote:
>>>> > >
>>>> > > > Hi,
>>>> > > >
>>>> > > > We have a webservice that performs reads/writes on HBase tables
>>>> and
>>>> > have
>>>> > > a
>>>> > > > requirement to authorize and audit table/column family access
>>>> using
>>>> > > Ranger.
>>>> > > >
>>>> > > > I've configured the reads/writes to be performed under doAs
to
>>>> try to
>>>> > > make
>>>> > > > this happen but the requests end up being authorized and audit
>>>> logged
>>>> > as
>>>> > > > the service user rather than the requestor.
>>>> > > >
>>>> > > >
>>>> > > > A snippet of the application code looks like this (doAsUser
is
>>>> the end
>>>> > > > user's username):
>>>> > > >
>>>> > > >
>>>> > > > UserGroupInformation ugi =
>>>> > UserGroupInformation.createProxyUser(doAsUser,
>>>> > > > UserGroupInformation.getLoginUser());
>>>> > > >
>>>> > > > try {
>>>> > > >       ugi.doAs(new PriviledgedExceptionAction<Void>()
{
>>>> > > >           @Override
>>>> > > >           public Void run() throws Exception {
>>>> > > >                 LOGGER.info("HBase put as user " +
>>>> > > ugi.getShortUserName());
>>>> > > >                 table.put(put);
>>>> > > >                 return null;
>>>> > > >           }
>>>> > > >     });
>>>> > > >
>>>> > > >
>>>> > > > Has anyone got experience with the HBase Ranger plugin and/or
come
>>>> > across
>>>> > > > this problem before and know the best way to solve it?
>>>> > > >
>>>> > > > For reference this is all running with HDP 2.3.2.
>>>> > > >
>>>> > > > Thanks in advance!
>>>> > > >
>>>> > > > --
>>>> > > > Chris
>>>> > > >
>>>> > > > --
>>>> > > >
>>>> > > >
>>>> > > > *NOTICE AND DISCLAIMER*
>>>> > > >
>>>> > > > This email (including attachments) is confidential. If you
are
>>>> not the
>>>> > > > intended recipient, notify the sender immediately, delete
this
>>>> email
>>>> > from
>>>> > > > your system and do not disclose or use for any purpose.
>>>> > > >
>>>> > > > Business Address: Eagle House, 163 City Road, London, EC1V
1NR.
>>>> United
>>>> > > > Kingdom
>>>> > > > Registered Office: Finsgate, 5-7 Cranwood Street, London,
EC1V
>>>> 9EE.
>>>> > > United
>>>> > > > Kingdom
>>>> > > > Big Data Partnership Limited is a company registered in England
&
>>>> Wales
>>>> > > > with Company No 7904824
>>>> > > >
>>>> > >
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > *Christopher Gent*
>>>> >
>>>> > *Managing Consultant*
>>>> > Big Data Partnership
>>>> > M: 07795 210205
>>>> > E: chris.gent@bigdatapartnership.com
>>>> >
>>>> > *NOTICE AND DISCLAIMER*
>>>> >
>>>> > This email (including attachments) is confidential. If you are not the
>>>> > intended recipient, notify the sender immediately, delete this email
>>>> from
>>>> > your system and do not disclose or use for any purpose.
>>>> >
>>>> > Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
>>>> > Kingdom
>>>> > Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE.
>>>> United
>>>> > Kingdom
>>>> > Big Data Partnership Limited is a company registered in England &
>>>> Wales
>>>> > with Company No 7904824
>>>> >
>>>> > --
>>>> >
>>>> >
>>>> > *NOTICE AND DISCLAIMER*
>>>> >
>>>> > This email (including attachments) is confidential. If you are not the
>>>> > intended recipient, notify the sender immediately, delete this email
>>>> from
>>>> > your system and do not disclose or use for any purpose.
>>>> >
>>>> > Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
>>>> > Kingdom
>>>> > Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE.
>>>> United
>>>> > Kingdom
>>>> > Big Data Partnership Limited is a company registered in England &
>>>> Wales
>>>> > with Company No 7904824
>>>> >
>>>>
>>>
>>>
>> *NOTICE AND DISCLAIMER*
>>
> This email (including attachments) is confidential. If you are not the
>> intended recipient, notify the sender immediately, delete this email from
>> your system and do not disclose or use for any purpose.
>>
>> Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
>> Kingdom
>> Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE.
>> United Kingdom
>> Big Data Partnership Limited is a company registered in England & Wales
>> with Company No 7904824
>>
>

-- 
 

*NOTICE AND DISCLAIMER*

This email (including attachments) is confidential. If you are not the 
intended recipient, notify the sender immediately, delete this email from 
your system and do not disclose or use for any purpose.

Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United 
Kingdom
Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. United 
Kingdom
Big Data Partnership Limited is a company registered in England & Wales 
with Company No 7904824

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message