sentry-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dapeng Sun (JIRA)" <>
Subject [jira] [Commented] (SENTRY-427) High availability for the SENTRY service
Date Fri, 19 Sep 2014 09:37:33 GMT


Dapeng Sun commented on SENTRY-427:

Hi, Prasad. Thanks very much for your comments!
{quote}sequence ID: Could you please add some details on how this global counter will be used
Okay. I will add a factory class for {{CommitContext}} , we can {{getCommitContext(UUID serverUUID,
long sequenceId)}} in the factory instead of {{new CommitContext(UUID serverUUID, long sequenceId)}},
the code will be like
  public CommitContext getCommitContext(UUID serverUUID, long sequenceId) {
    if (haEnabled) {
      try {
        return new CommitContext(serverUUID, counter.getGlobalSequenceId());
      } catch (Exception e) {
        LOGGER.error("Error in get global sequence Id, return the sequenceId in service!"
        return new CommitContext(serverUUID, sequenceId);
    } else {
      return new CommitContext(serverUUID, sequenceId);
{{counter.getGlobalSequenceId()}} will return the sequence id. How do you think about it?
{quote}Regarding the transactions, I guess the example given could very well be applicable
to a single Sentry server with multiple concurrent client (eg two different users from Hive).
As you mentioned in the document, with many-to-many relationship, locking a Sentry object
like role is equivalent to locking the mapping table rows for that object. I am not sure whether
optimistic transactions in JDO can solve that.. The pessimistic (transaction isolation level
set to serialized) would solve that but could be a huge performance penalty.
Another option could be to implement a rudimentary lock manager via ZooKeeper to lock logical
sentry objects like roles and privileges. This is what Hive does to handle table locking to
support concurrent DDL/DML and queries.{quote}
Yes, I’m agree with you that a single Sentry server with multiple concurrent client may
also cause the problem, and the ZooKeeper is also a good choice, but I think optimistic transaction
will minimal impact performance, I had tested the optimistic lock in local environment, relevant
unit tests will be add to the patch.
{quote}Audit Log support
IMO it should be okay to have separate audit logs for each Sentry service. The downstream
consumer of the audit log should be able to extract it from multiple sources. We are already
introducing ZK as requirement, to add Flume/Kafka and HDFS dependency on top of that would
make things too hard to setup and deploy.{quote}
Yes, I totally agree with you. Very appreciate your comments!

> High availability for the SENTRY service
> ----------------------------------------
>                 Key: SENTRY-427
>                 URL:
>             Project: Sentry
>          Issue Type: New Feature
>            Reporter: Dapeng Sun
>            Assignee: Dapeng Sun
>         Attachments: SENTRY_HA_Design.pdf

This message was sent by Atlassian JIRA

View raw message