sentry-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SENTRY-2184) Performance Issue: MPath is queried for each MAuthzPathsMapping in full snapshot
Date Mon, 19 Mar 2018 21:51:00 GMT

    [ https://issues.apache.org/jira/browse/SENTRY-2184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405505#comment-16405505
] 

Hadoop QA commented on SENTRY-2184:
-----------------------------------

Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12915172/SENTRY-2184.001.patch against master.

{color:green}Overall:{color} +1 all checks pass

{color:green}SUCCESS:{color} all tests passed

Console output: https://builds.apache.org/job/PreCommit-SENTRY-Build/3703/console

This message is automatically generated.

> Performance Issue: MPath is queried for each MAuthzPathsMapping in full snapshot
> --------------------------------------------------------------------------------
>
>                 Key: SENTRY-2184
>                 URL: https://issues.apache.org/jira/browse/SENTRY-2184
>             Project: Sentry
>          Issue Type: Bug
>          Components: Sentry
>    Affects Versions: 2.1.0
>            Reporter: Na Li
>            Assignee: Na Li
>            Priority: Critical
>         Attachments: SENTRY-2184.001.patch
>
>
> MAuthzPathsMapping contains list of MPath instances. From log message, when getting path
full snapshot at SentryStore.retrieveFullPathsImageCore(), DataNucleus issues a query for
all MPath instances associated with each MAuthzPathsMapping. Therefore, getting full path
image may take a very long time.
> The solution is to get MPath in a batch when getting full path image.
> Log Message when DataNucleus issues a query for all MPath instances associated with each MAuthzPathsMapping
> {code:java}
> 1) Initially, all MAuthzPathsMapping entries for current snapshot is queried.
> 2018-03-14 11:51:23,999 (main) [DEBUG - org.datanucleus.util.Log4JLogger.debug(Log4JLogger.java:58)]
SELECT 'org.apache.sentry.provider.db.service.model.MAuthzPathsMapping' AS NUCLEUS_TYPE,A0.AUTHZ_OBJ_NAME,A0.AUTHZ_SNAPSHOT_ID,A0.CREATE_TIME_MS,A0.AUTHZ_OBJ_ID
FROM AUTHZ_PATHS_MAPPING A0 WHERE A0.AUTHZ_SNAPSHOT_ID = <1>
> 2) call authzToPaths.getPathStrings() causes MPath to be queried for each AUTHZ_OBJ_ID
> 2018-03-14 11:52:27,700 (main) [DEBUG - org.datanucleus.util.Log4JLogger.debug(Log4JLogger.java:58)]
SELECT 'org.apache.sentry.provider.db.service.model.MPath' AS NUCLEUS_TYPE,A0.PATH_NAME,A0.PATH_ID
FROM AUTHZ_PATH A0 WHERE A0.AUTHZ_OBJ_ID = <1>{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message