Mailing-List: contact hbase-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-dev@hadoop.apache.org
Message-ID: <1856639568.1260339558112.JavaMail.jira@brutus>
Date: Wed, 9 Dec 2009 06:19:18 +0000 (UTC)
From: "linden lin (JIRA)" <jira@apache.org>
To: hbase-dev@hadoop.apache.org
Subject: [jira] Commented: (HBASE-2014) [DAC] Audit
In-Reply-To: <2107153691.1259263359602.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12787937#action_12787937 ] 

linden lin commented on HBASE-2014:
-----------------------------------

I give my thoughts for reference.
Routing string is hierarchical and easily matched by order. 

The routing string is as follows:

{Event Type}.{Candidate Router Key}.{Sub Event Item}.{Other items}

{Event Type} indicates the main event type; {Candidate Router Key} is the consideration for scalability and performance; and {Sub Event Item} will be the more accurate type for filtering and routing; {Other items} is reserved for future.

{Event Type}:
1.	Session
2.	Command
3.	Data in Command
4.	Counts in Command
5.	Node Status: necessary status and abnormal events.

Relation (A->B: A depends on B):
Counts in Command->Data In Command->Command->Session;
Counts in Command->Command->Session;
Node Status;

{Candidate Router Key} (Only choose one):
1.	Object Name (Recommended, it is table name for Hbase, if there isn't table in the event, Object name is null. If client query metadata from Zookeeper, use hardcode table name to replace. Such as "Hbase Metadata".)
2.	HRegion Identity
3.	RegionServer IP
4.	Others.....

{Sub Event Item}: It depends on the {Event Type}.
1.	Session (in current Hbase version, it is the connection, establish connection and close connection)
       Session Login In : 
       Session Login Off:
2.	Command
       Command Request
       Command Response
3.	Data in Command (interested data from input and output command)
       Data metadata and content (only)
4.	Counts In Command
      Counts Set (only)
5.	Node Status
      Performance counts (Resource usage, Session amount, and other performance related counts)
      Abnormal events (defined by user, normally, it includes error event, huge request in a short time and 
      so on).


BTW, I suggest try to shorten the routing string and keep the capability of dynamic routing. Fox example:

2.ObjectName.2.Others  => it means Command.ObjectName.CommandResponse.Others (string to number is only for predefined type).


> [DAC] Audit
> -----------
>
>                 Key: HBASE-2014
>                 URL: https://issues.apache.org/jira/browse/HBASE-2014
>             Project: Hadoop HBase
>          Issue Type: Sub-task
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>             Fix For: 0.22.0
>
>
> Audit: Important actions taken by subjects should be logged for accountability, a chronological record which enables the full reconstruction and examination of a sequence of events, e.g. schema changes or data mutations. Logging activity should be protected from all subjects except for a restricted set with administrative privilege, perhaps to only a single super-user.
> Support dynamic scaling transparently and support multi-tenant. Acquire enough detail and support streamline auditing in time. Should be configurable on a per-table basis to avoid this overhead where it is not wanted.
> Consider logging audit trails to an HBase table (bigtable type schemas are natural for this) and also external options with Java library support - syslog, etc., or maybe commons-logging is sufficient and punt to administrator to set up appropriate commons-logging/log4j configurations for their needs.
> Consider integration with Scribe (http://developers.facebook.com/scribe/) or Chukwa (http://wiki.apache.org/hadoop/Chukwa).
> * Session information (Required)
> ** Client, server, When, How, Where.
> * Command information (Required)
> ** Command detail and intent
> ** Command result and why
> ** Data event (input and output interested data, depends on predefined policy) 
> *** Metadata, data detail, session identity and command identity, data direction, etc.
> ** Command Counts (optional)
> *** Execution duration
> *** Response/request data amount
> *** Resource usage
> * Node status
> ** Node resource counts
> ** Session status
> ** Abnormal events (Required)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.