cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ching-cheng (JIRA)" <j...@apache.org>
Subject [jira] Updated: (CASSANDRA-2183) memtable_flush_after_mins setting not working
Date Thu, 17 Feb 2011 16:54:24 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-2183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ching-cheng updated CASSANDRA-2183:
-----------------------------------

    Description: 
We have observed the behavior that memtable_flush_after_mins setting not working occasionally.
  After some testing and code digging, we finally figured out what going on.
The memtable_flush_after_mins won't work on certain condition with current implementation
in Cassandra.

In org.apache.cassandra.db.Table,  the scheduled flush task is setup by the following code
during construction.

------------------------------------------------------------------------------------------------------------------
int minCheckMs = Integer.MAX_VALUE;
       
for (ColumnFamilyStore cfs : columnFamilyStores.values())  
{
    minCheckMs = Math.min(minCheckMs, cfs.getMemtableFlushAfterMins() * 60 * 1000);
}

Runnable runnable = new Runnable()
{
   public void run()
   {
       for (ColumnFamilyStore cfs : columnFamilyStores.values())
       {
           cfs.forceFlushIfExpired();
       }
   }
};
flushTask = StorageService.scheduledTasks.scheduleWithFixedDelay(runnable, minCheckMs, minCheckMs,
TimeUnit.MILLISECONDS);
------------------------------------------------------------------------------------------------------------------------------

Now for our application, we will create a keyspacewithout without any columnfamily first.
 And only add needed columnfamily later depends on request.

However, when keyspacegot created (without any columnfamily ), the above code will actually
schedule a fixed delay flush check task with Integer.MAX_VALUE ms
since there is no columnfamily yet.

Later when you add columnfamily to this empty keyspace, the initCf() method in Table.java
doesn't check whether the scheduled flush check task interval need
to be updated or not.   To fix this, we'd need to restart the Cassandra after columnfamily
added into the keyspace. 

I would suggest that add additional logic in initCf() method to recreate a scheduled flush
check task if needed.


  was:
We have observed the behavior that memtable_flush_after_mins setting not working occasionally.
  After some testing and code digging, we finally figured out what going on.
The memtable_flush_after_mins won't work on certain condition with current implementation
in Cassandra.

In org.apache.cassandra.db.Table,  the scheduled flush task is setup by the following code
during construction.

------------------------------------------------------------------------------------------------------------------
int minCheckMs = Integer.MAX_VALUE;
       
for (ColumnFamilyStore cfs : columnFamilyStores.values())  
{
    minCheckMs = Math.min(minCheckMs, cfs.getMemtableFlushAfterMins() * 60 * 1000);
}

Runnable runnable = new Runnable()
{
   public void run()
   {
       for (ColumnFamilyStore cfs : columnFamilyStores.values())
       {
           cfs.forceFlushIfExpired();
       }
   }
};
flushTask = StorageService.scheduledTasks.scheduleWithFixedDelay(runnable, minCheckMs, minCheckMs,
TimeUnit.MILLISECONDS);
------------------------------------------------------------------------------------------------------------------------------

Now for our application, we will create a keyspacewithout any columnfamily first.  And only
add needed columnfamily later depends on request.

However, when keyspacegot created (without any columnfamily ), the above code will actually
schedule a fixed delay flush check task with Integer.MAX_VALUE ms
since there is no columnfamily yet.

Later when you add columnfamily to this empty keyspace, the initCf() method in Table.java
doesn't check whether the scheduled flush check task interval need
to be updated or not.   To fix this, we'd need to restart the Cassandra after columnfamily
added into the keyspace. 

I would suggest that add additional logic in initCf() method to recreate a scheduled flush
check task if needed.



> memtable_flush_after_mins setting not working
> ---------------------------------------------
>
>                 Key: CASSANDRA-2183
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2183
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7.0, 0.7.1
>            Reporter: Ching-cheng
>            Priority: Minor
>
> We have observed the behavior that memtable_flush_after_mins setting not working occasionally.
  After some testing and code digging, we finally figured out what going on.
> The memtable_flush_after_mins won't work on certain condition with current implementation
in Cassandra.
> In org.apache.cassandra.db.Table,  the scheduled flush task is setup by the following
code during construction.
> ------------------------------------------------------------------------------------------------------------------
> int minCheckMs = Integer.MAX_VALUE;
>        
> for (ColumnFamilyStore cfs : columnFamilyStores.values())  
> {
>     minCheckMs = Math.min(minCheckMs, cfs.getMemtableFlushAfterMins() * 60 * 1000);
> }
> Runnable runnable = new Runnable()
> {
>    public void run()
>    {
>        for (ColumnFamilyStore cfs : columnFamilyStores.values())
>        {
>            cfs.forceFlushIfExpired();
>        }
>    }
> };
> flushTask = StorageService.scheduledTasks.scheduleWithFixedDelay(runnable, minCheckMs,
minCheckMs, TimeUnit.MILLISECONDS);
> ------------------------------------------------------------------------------------------------------------------------------
> Now for our application, we will create a keyspacewithout without any columnfamily first.
 And only add needed columnfamily later depends on request.
> However, when keyspacegot created (without any columnfamily ), the above code will actually
schedule a fixed delay flush check task with Integer.MAX_VALUE ms
> since there is no columnfamily yet.
> Later when you add columnfamily to this empty keyspace, the initCf() method in Table.java
doesn't check whether the scheduled flush check task interval need
> to be updated or not.   To fix this, we'd need to restart the Cassandra after columnfamily
added into the keyspace. 
> I would suggest that add additional logic in initCf() method to recreate a scheduled
flush check task if needed.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message