Yes, it was. I was dumping data from Oracle into Cassandra= .

On Sun, Jul 4, 2010 at 11:11 AM, Andre= w Rollins <an= drew@localytics.com> wrote:

Is your IO under = heavy load? If it is, that may be the cause, otherwise I'm not sure wha= t causes significant lag. On Linux I like to use "iostat -tx 10" = to check IO.

- Andrew
=

On Sun, Jul 4, 2010 at 4:04 AM, Dav= id Boxenhorn <david@lookin2.com> wrote:

Thank you very much! I now understand things much better.<= br>
However, my configuration is as follows:

=A0 <CommitLogSyn= c>periodic</CommitLogSync>
=A0 <CommitLogSyncPeriodInMS>1= 0000</CommitLogSyncPeriodInMS>

So I should see my commit log change after 10,000 milliseconds =3D 10 s= econds? It seems to take much longer to show up.
<= br>
On Sun, Jul 4, 2010 at 10:52 AM, Andrew Rolli= ns <andrew@localytics.com> wrote:

By default C= assandra syncs the commit log to disk periodically, so if you are looking a= t file sizes, you won't see the most up to date numbers. This is just l= ike how if you tail a file that isn't flushing frequently, you might wa= it a little while before you see the updates.

In periodic mode, Cassandra=A0acknowledges=A0the write = to the client immediately (even before it is synced). You can run Cassandra= in batch mode instead, which basically means it writes in batches and it won't acknowledge the writes to the client until it has actually = synced. I'm still somewhat new to this, but that's my understanding= .

Have a look at=A0CommitLogSync in your storage-conf.xml for = more info about setting up syncing periods.

As an aside,= I'm not sure why the "ack immediately" or "ack after sy= nc" setting is piggybacked on the periodic vs batch setting. At first = glance it seems like concepts should be independent of one another.

- Andrew

On Sun, Jul 4, 2010 at 3:34 AM, David Boxenhorn <david@lookin2.= com> wrote:

As I understand it, when you write to Cassandra, you are a= ssured that, if successful, the new data has been written to a log file - s= o that if there is a crash your data is safe. Is this correct?

If t= he above is correct, there is something going=A0on that I don't underst= and. Are the log files to which the data is first written the ones that loo= k like /var/lib/cassandra/commitlog/CommitLog-1277998453387.log ? The reaso= n I ask is that when I write a lot of data, nothing seems to change in the = commitlog directory for a long time, then at some point the log files in th= is directory get updated. It looks to me like there's memory caching in= volved, and the new data is not being immediately written to disk. What is = going on?