sentry-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexander Kolbasov (JIRA)" <>
Subject [jira] [Commented] (SENTRY-1964) HDFS sync does not need partition locations (usually)
Date Wed, 15 Nov 2017 00:51:00 GMT


Alexander Kolbasov commented on SENTRY-1964:

[~Tagar] What you are describing may be a reasonable thing to do, but I think the scope of
this JIRA is to optimize existing semantics.

> HDFS sync does not need partition locations (usually)
> -----------------------------------------------------
>                 Key: SENTRY-1964
>                 URL:
>             Project: Sentry
>          Issue Type: Improvement
>          Components: Sentry
>    Affects Versions: 2.0.0
>            Reporter: Na Li
>            Assignee: Na Li
>            Priority: Critical
>         Attachments: SENTRY-1964.001.patch
> Right now, sentry saves partition info from HMS and send it to HDFS. HDFS only needs
database and table info, and does not need partition info for ACL unless the partion location
is not sharing the same prefix of its table.
> The partition data amount is huge, and causes performance issue. We can optimize it by
not saving and not sending partition info if it shares the same path of its table. 

This message was sent by Atlassian JIRA

View raw message