cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lyuben Todorov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-6572) Workload recording / playback
Date Mon, 17 Feb 2014 21:30:21 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13903514#comment-13903514
] 

Lyuben Todorov commented on CASSANDRA-6572:
-------------------------------------------

Just an update with progress so far, [branch here|https://github.com/lyubent/cassandra/tree/query_rec].
Currently query recording is enabled using JMX via StorageService#enableQueryRecording. This
will record every nth query and append it to the QueryLog. Currently the log is piggybacking
on the commit_log directory but the long-term plan is to either add a setting to cassandra.yaml
or modify {{enableQueryRecording}} to take another param that will be the directory for the
log. 

Queries are stored in an append only log as suggested, once the log reaches 4MB (this should
also be configurable, I'm thinking set a default and add a JMX function that can overload
said default). Once the limit is reached, the log is renamed (a timestamp is added to the
name) and a new log is created that will now store the new appends. 

As for the replaying, its still fairly basic, currently the entire query log file is read
in a single operation (smarter approaches than storing a large collection of query strings
are welcome), then the collection of queries is replayed sequentially. If there is too large
a gap between queries, a timeout takes effect to avoid stalling the replay (again this should
be configurable, currently the timeout is 10s). Replaying of the workload is invoked via JMX
right now, but *workload replayer* tool will be in charge of handling the replaying, and also
what cluster the logs get replayed to. 

> Workload recording / playback
> -----------------------------
>
>                 Key: CASSANDRA-6572
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6572
>             Project: Cassandra
>          Issue Type: New Feature
>          Components: Core, Tools
>            Reporter: Jonathan Ellis
>            Assignee: Lyuben Todorov
>             Fix For: 2.0.6
>
>
> "Write sample mode" gets us part way to testing new versions against a real world workload,
but we need an easy way to test the query side as well.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message