cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinay Chella (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-12151) Audit logging for database activity
Date Sat, 17 Mar 2018 00:11:00 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16403146#comment-16403146
] 

Vinay Chella commented on CASSANDRA-12151:
------------------------------------------

[~djoshi3]

Implemented all the code reviews comments provided in this JIRA thread as well as Github PR.
Except one below

{quote}
Consider refactoring your code to add a netty handler that invokes an auditing interface.
The advantage of this approach would be that, when audit logging is disabled, you can take
this handler out of the netty pipeline. This way there is zero performance impact when the
audit is disabled. You can define a IAuditLogger interface that has sufficient contextual
information to log all queries. This will help make the audit logging implementation pluggable.
{quote}

I am creating a follow-up JIRA to discuss the more details on this.

On a high level, this changeset includes following changes

# Extended and reused FullQueryLogger in logging audit events
# Combined and Simplified FQL and AuditLog entry points in the request path
# AuditLogEntryType::allStatementsMap - Instead of creating an explicit map of statements,
type of statement is being added to the actual class itself. This makes new statements easy
to manage
# AuditLogFilter::loadFilters - Simplified filter loading logic, easy to add new filters if
needed
# CQL query auditing can now be filtered on user level.
# Added documentation in the doc folder
# Removed ConsistencyLevel in logging details
# Added more test cases
# Implemented code review comments provided in this JIRA as well as Github PR

\\

||[branch|https://github.com/vinaykumarchella/cassandra/tree/trunk_CASSANDRA-12151]||
|[PR for trunk|https://github.com/vinaykumarchella/cassandra/pull/2/commits]|
|[circleci|https://circleci.com/gh/vinaykumarchella/cassandra/tree/trunk_CASSANDRA-12151]|

\\

We ran cassandra stress test with this patch and attached stress test results. Here is the
high level summary

Note: Below tests are run on AWS i2.2xl instance.
\\
{{cass-stress cmd: write n=1000000 -rate threads=10 -graph file=CASSANDRA_12151-benchmark.html}}
||WRITE - Test Suite||Throughput||Latency Mean||Latency 95th||Latency 99th||
|trunk|13,925 op/s|0.7 ms|1.1 ms|1.7 ms|
|CASSANDRA-12151:Disabled AuditLog|14,422 op/s|0.7 ms|1.1 ms|1.6 ms|
|CASSANDRA-12151:FQL based AuditLog with Sync|13,372 op/s|0.7 ms|1.2 ms|1.7 ms|
|CASSANDRA-12151:FQL based AuditLog with Async|12,908 op/s|0.8 ms|1.2 ms|1.9 ms|
|CASSANDRA-12151:SLF4j based AuditLog|10,520 op/s|0.9 ms|1.6 ms|2.4 ms|
\\
{{cass-stress cmd: mixed n=1000000 -rate threads=10 -graph file=CASSANDRA_12151-benchmark.html}}
||MIXED - Test Suite||Throughput||Latency Mean||Latency 95th||Latency 99th||
|trunk|12,939 op/s [READ: 6,494 op/s, WRITE: 6,444 op/s]|0.7 ms [READ: 0.8 ms, WRITE: 0.7
ms]|1.2 ms [READ: 1.3 ms, WRITE: 1.2 ms]|1.7 ms [READ: 1.8 ms, WRITE: 1.7 ms]|
|CASSANDRA-12151: Disabled AuditLog|12,840 op/s [READ: 6,421 op/s, WRITE: 6,419 op/s]|0.8
ms [READ: 0.8 ms, WRITE: 0.7 ms]|1.2 ms [READ: 1.3 ms, WRITE: 1.2 ms]|1.8 ms [READ: 1.8 ms,
WRITE: 1.7 ms]|
|CASSANDRA-12151: FQL based AuditLog with Sync|10,932 op/s [READ: 5,452 op/s, WRITE: 5,481
op/s]|0.9 ms [READ: 1.0 ms, WRITE: 0.8 ms]|1.5 ms [READ: 1.6 ms, WRITE: 1.4 ms]|2.3 ms [READ:
2.4 ms, WRITE: 2.1 ms]|
|CASSANDRA-12151: FQL based AuditLog with Async|11,146 op/s [READ: 5,565 op/s, WRITE: 5,581
op/s]|0.9 ms [READ: 0.9 ms, WRITE: 0.8 ms]|1.5 ms [READ: 1.5 ms, WRITE: 1.4 ms]|2.2 ms [READ:
2.2 ms, WRITE: 2.1 ms]|
|CASSANDRA-12151: SLF4j based AuditLog|9,764 op/s [READ: 4,883 op/s, WRITE: 4,882 op/s]|1.0
ms [READ: 1.0 ms, WRITE: 1.0 ms]|1.7 ms [READ: 1.7 ms, WRITE: 1.6 ms]|2.5 ms [READ: 2.6 ms,
WRITE: 2.4 ms]|

\\

Looking at the results, with AuditLog feature disabled, there appears to be no measurable
difference in performance. FQL appears to have little or no overhead in WRITE only workloads,
and a minor overhead in MIXED workload. SLF4J appears to have minor regressions in both workloads
(with mixed slightly worse).

> Audit logging for database activity
> -----------------------------------
>
>                 Key: CASSANDRA-12151
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
>             Project: Cassandra
>          Issue Type: New Feature
>            Reporter: stefan setyadi
>            Assignee: Vinay Chella
>            Priority: Major
>             Fix For: 4.x
>
>         Attachments: 12151.txt, DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done on our server.
> It should show username, remote address, timestamp, action type, keyspace, column family,
and the query statement.
> it should also be able to log connection attempt and changes to the user/roles.
> I was thinking of making a new keyspace and insert an entry for every activity that occurs.
> Then It would be possible to query for specific activity or a query targeting a specific
keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org


Mime
View raw message