hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "linden lin (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2014) [DAC] Audit
Date Mon, 07 Dec 2009 10:17:18 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12786849#action_12786849

linden lin commented on HBASE-2014:

@stack, User should have a another hbase instance for audit isn't a reasonable solution from
my view. Acquiring the enough detail, log4j or other logging solution is ok (leverage the
efforts in implementation). But my consideration is how to transfer the log to different kind
of sinks with efficient method.
My draft idea, I recommend using the distributional subscriber & receiver model for Hbase
audit. One Hbase server (or HRegion) is a subscriber (many subscribers) for the distributional
framework and receiver is the any sink which receives the interested content from distributional
framework. The key point is receiver can divide the subscriber's log for load balance (for
example, by topic name, topic name is IP address, table name, key range and so on). 
Thus, Hbase only needs to add a client plug-in for the distributional framework (message bus,
etc) and define the log title for router (it is static from design).

Normally, the auditing feature is disabled. When user want to enable this feature, he should
install the specific third-party router cluster (distributional, scalable framework), then
add the cluster address to Hbase configuration. Thus, Hbase cluster can be the subscribers
for the router cluster. The next things I think they are all customers' task. (Add receiver,
operate the log and so on)

Meanwhile, should we need to support dynamic subscriber and subscriber content in this version?

> [DAC] Audit
> -----------
>                 Key: HBASE-2014
>                 URL: https://issues.apache.org/jira/browse/HBASE-2014
>             Project: Hadoop HBase
>          Issue Type: Sub-task
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>             Fix For: 0.22.0
> Audit: Important actions taken by subjects should be logged for accountability, a chronological
record which enables the full reconstruction and examination of a sequence of events, e.g.
schema changes or data mutations. Logging activity should be protected from all subjects except
for a restricted set with administrative privilege, perhaps to only a single super-user.
> Support dynamic scaling transparently and support multi-tenant. Acquire enough detail
and support streamline auditing in time. Should be configurable on a per-table basis to avoid
this overhead where it is not wanted.
> Consider logging audit trails to an HBase table (bigtable type schemas are natural for
this) and also external options with Java library support - syslog, etc., or maybe commons-logging
is sufficient and punt to administrator to set up appropriate commons-logging/log4j configurations
for their needs.
> Consider integration with Scribe (http://developers.facebook.com/scribe/) or Chukwa (http://wiki.apache.org/hadoop/Chukwa).
> * Session information (Required)
> ** Client, server, When, How, Where.
> * Command information (Required)
> ** Command detail and intent
> ** Command result and why
> ** Data event (input and output interested data, depends on predefined policy) 
> *** Metadata, data detail, session identity and command identity, data direction, etc.
> ** Command Counts (optional)
> *** Execution duration
> *** Response/request data amount
> *** Resource usage
> * Node status
> ** Node resource counts
> ** Session status
> ** Abnormal events (Required)

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message