hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Illya Yalovyy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-15076) Improve scalability of LDAP authentication provider group filter
Date Mon, 28 Nov 2016 19:55:58 GMT

    [ https://issues.apache.org/jira/browse/HIVE-15076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15702942#comment-15702942
] 

Illya Yalovyy commented on HIVE-15076:
--------------------------------------

[~ngangam]

Thank you for the feedback on this CR.

Here is some performance results (what I can share):
|| User member of # groups || GroupMembershipKeyFilter || UserMembershipKeyFilter ||
| 200 | 0.118 | 0.103 |
| 400 | 0.135 | 0.106 |
| 600 | 0.171 | 0.113 |
| 800 | 0.230 | 0.119 |
| 1000 | FAIL | 0.129 |

GroupMembershipKeyFilter fails with "javax.naming.SizeLimitExceededException: [LDAP: error
code 4 - Sizelimit Exceeded]" when number of groups greater than 800. The particular number
of groups when the default implementation fails depends on record size for each group, so
in real production it will be much less.

> Improve scalability of LDAP authentication provider group filter
> ----------------------------------------------------------------
>
>                 Key: HIVE-15076
>                 URL: https://issues.apache.org/jira/browse/HIVE-15076
>             Project: Hive
>          Issue Type: Improvement
>          Components: Authentication
>    Affects Versions: 2.1.0
>            Reporter: Illya Yalovyy
>            Assignee: Illya Yalovyy
>         Attachments: HIVE-15076.1.patch, HIVE-15076.2.patch
>
>
> Current implementation uses following algorithm:
> #   For a given user find all groups that user is a member of. (A list of LDAP groups
is constructed as a result of that request)
> #  Match this list of groups with provided group filter.
>  
> Time/Memory complexity of this approach is O(N) on client side, where N – is a number
of groups the user has membership in. On a large directory (800+ groups per user) we can observe
up to 2x performance degradation and failures because of size of LDAP response (LDAP: error
code 4 - Sizelimit Exceeded).
>  
> Some Directory Services (Microsoft Active Directory for instance) provide a virtual attribute
for User Object that contains a list of groups that user belongs to. This attribute can be
used to quickly determine whether this user passes or fails the group filter.   



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message