Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: domain of artur.kronenberg@openmarket.com
 designates 81.187.36.3 as permitted sender)
Message-ID: <53295B16.40605@openmarket.com>
Date: Wed, 19 Mar 2014 08:53:42 +0000
From: Artur Kronenberg <artur.kronenberg@openmarket.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:24.0) Gecko/20100101 Thunderbird/24.3.0
MIME-Version: 1.0
To: user@cassandra.apache.org
Subject: Re: How to extract information from commit log?
References: <f94be7b0ef15e9483239b605db875472@ufl.edu>
 <CAOP8aV_OeFbmJfVkeMRfv2pWTvwkz4Uk+BFiYftb2OaJeOR8QA@mail.gmail.com>
 <64e2c65edd65fda0c19efa183c43b295@ufl.edu>
In-Reply-To: <64e2c65edd65fda0c19efa183c43b295@ufl.edu>
Content-Type: multipart/alternative;
 boundary="------------020108040803090804010201"

This is a multi-part message in MIME format.
--------------020108040803090804010201
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

we did something similar. We did utilize some cassandra code though and 
wrote a custom commitlog reader that outputs our data into a readable form.

You can look here: 
http://grepcode.com/file/repo1.maven.org/maven2/org.apache.cassandra/cassandra-all/1.1.9/org/apache/cassandra/db/commitlog/CommitLogReplayer.java

This code is used to replay commitlogs when starting up cassandra. It 
has the ability to deserialize and transform the data into what you'll need.

-- artur


On 18/03/14 19:32, Han,Meng wrote:
>
> Hi Jonathan,
>
> Thank you for the timely reply. I am doing this experiment on a 
> continuous basis. To be more specific, I will issue a large amount of 
> read and write operations to a particular key in a short time 
> interval. I'd like to know the order that write operations happens at 
> each replica. TImestamps definitely help to determine order, but the 
> WRITETIME and SStable2Json both looks me only return the timestamps 
> when that key was updated the moment the WRITETIME/SStable2Json is 
> issued. It looks like a one time thing to me. Or put in another way, 
> if I want to get the write time for all write operations in that short 
> invertal to determine a total order for write on that replia I have to 
> constantly issue WRITETIME to this replica?  Correct me if I am wrong 
> here.
>
> Light me up pleeeeeeeeeease!
>
> On Tue, 18 Mar 2014 15:05:07 -0400, Jonathan Lacefield wrote:
>
>> Hello,
>>   Is this a one time investigative item or are you looking to set 
>> something up to do this continuously?  Don't recommend trying to read 
>> the commit log.
>>   You can always use the WRITETIME function in CQL or look within 
>> SSTables via the SStable2Json utility to see write times for 
>> particular versions of partitions.
>> Jonathan
>>
>> Jonathan Lacefield
>> Solutions Architect, DataStax
>> (404) 822 3487
>> <http://www.linkedin.com/in/jlacefield>
>> <http://www.datastax.com/what-we-offer/products-services/training/virtual-training>
>>
>>
>> On Tue, Mar 18, 2014 at 2:25 PM, Han,Meng <menghan@ufl.edu 
>> <mailto:menghan@ufl.edu>> wrote:
>>
>>     Hi Cassandra hackers!
>>
>>     I have a question regarding extracting useful information from
>>     commit log.
>>
>>     Since its a binary log, how should I extract information such as
>>     timestamp, values from it? Does anyone know any binary log reader
>>     that I can use directly to read commit log?
>>     If there is no such reader, could someone give me some advice hwo
>>     I can wrote such a reader?
>>
>>     Particularly, I want to know the order that write operations
>>     happens at each replica(cassandra server node) along with their
>>     timestamps, Does anyone know other methods how I can get this
>>     information without instrumenting Cassandra code?
>>
>>     Any help is appreciated!
>>
>>     Cheers,
>>     Meng
>>


--------------020108040803090804010201
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit

<html>
  <head>
    <meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <div class="moz-cite-prefix">Hi,<br>
      <br>
      we did something similar. We did utilize some cassandra code
      though and wrote a custom commitlog reader that outputs our data
      into a readable form. <br>
      <br>
      You can look here:
      <meta http-equiv="content-type" content="text/html; charset=UTF-8">
      <a
href="http://grepcode.com/file/repo1.maven.org/maven2/org.apache.cassandra/cassandra-all/1.1.9/org/apache/cassandra/db/commitlog/CommitLogReplayer.java">http://grepcode.com/file/repo1.maven.org/maven2/org.apache.cassandra/cassandra-all/1.1.9/org/apache/cassandra/db/commitlog/CommitLogReplayer.java</a><br>
      <br>
      This code is used to replay commitlogs when starting up cassandra.
      It has the ability to deserialize and transform the data into what
      you'll need.<br>
      <br>
      -- artur<br>
      <br>
      <br>
      On 18/03/14 19:32, Han,Meng wrote:<br>
    </div>
    <blockquote cite="mid:64e2c65edd65fda0c19efa183c43b295@ufl.edu"
      type="cite">
      <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
      <p>Hi Jonathan,</p>
      <p> </p>
      <p>Thank you for the timely reply. I am doing this experiment on a
        continuous basis. To be more specific, I will issue a large
        amount of read and write operations to a particular key in a
        short time interval. I'd like to know the order that write
        operations happens at each replica. TImestamps definitely help
        to determine order, but the WRITETIME and SStable2Json both
        looks me only return the timestamps when that key was updated
        the moment the WRITETIME/SStable2Json is issued. It looks like a
        one time thing to me. Or put in another way, if I want to get
        the write time for all write operations in that short invertal
        to determine a total order for write on that replia I have to
        constantly issue WRITETIME to this replica?  Correct me if I am
        wrong here.</p>
      <p>Light me up pleeeeeeeeeease!</p>
      <p> </p>
      <p> </p>
      <p>On Tue, 18 Mar 2014 15:05:07 -0400, Jonathan Lacefield wrote:</p>
      <blockquote type="cite" style="padding-left:5px;
        border-left:#1010ff 2px solid; margin-left:5px; width:100%"><!-- html ignored --><!-- head ignored --><!-- meta ignored -->
        <div dir="ltr">Hello,
          <div>  Is this a one time investigative item or are you
            looking to set something up to do this continuously?  Don't
            recommend trying to read the commit log.</div>
          <div>  You can always use the WRITETIME function in CQL or
            look within SSTables via the SStable2Json utility to see
            write times for particular versions of partitions.  </div>
          <div>Jonathan</div>
        </div>
        <div class="gmail_extra"><br clear="all">
          <div>
            <div dir="ltr">Jonathan Lacefield
              <div>Solutions Architect, DataStax</div>
              <div>(404) 822 3487</div>
              <div><a moz-do-not-send="true"
                  href="http://www.linkedin.com/in/jlacefield"><img
                    moz-do-not-send="true"
src="http://s.c.lnkd.licdn.com/scds/common/u/img/logos/logo_linkedin_92x22.png"
                    alt=""></a></div>
              <div><a moz-do-not-send="true"
href="http://www.datastax.com/what-we-offer/products-services/training/virtual-training"><img
                    moz-do-not-send="true"
src="http://www.datastax.com/wp-content/themes/datastax-2013/images/email-sig/email-sig-virtual-training.png"
                    alt=""></a></div>
            </div>
          </div>
          <br>
          <br>
          <div class="gmail_quote">On Tue, Mar 18, 2014 at 2:25 PM,
            Han,Meng <span>&lt;<a moz-do-not-send="true"
                href="mailto:menghan@ufl.edu">menghan@ufl.edu</a>&gt;</span>
            wrote:<br>
            <blockquote class="gmail_quote" style="margin: 0 0 0 .8ex;
              border-left: 1px #ccc solid; padding-left: 1ex;">Hi
              Cassandra hackers!<br>
              <br>
              I have a question regarding extracting useful information
              from commit log.<br>
              <br>
              Since its a binary log, how should I extract information
              such as timestamp, values from it? Does anyone know any
              binary log reader that I can use directly to read commit
              log?<br>
              If there is no such reader, could someone give me some
              advice hwo I can wrote such a reader?<br>
              <br>
              Particularly, I want to know the order that write
              operations happens at each replica(cassandra server node)
              along with their timestamps, Does anyone know other
              methods how I can get this information without
              instrumenting Cassandra code?<br>
              <br>
              Any help is appreciated!<br>
              <br>
              Cheers,<br>
              Meng</blockquote>
          </div>
        </div>
      </blockquote>
      <p> </p>
      <div> </div>
    </blockquote>
    <br>
  </body>
</html>

--------------020108040803090804010201--