Return-Path: X-Original-To: apmail-cassandra-commits-archive@www.apache.org Delivered-To: apmail-cassandra-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 134C211B78 for ; Wed, 20 Aug 2014 08:44:27 +0000 (UTC) Received: (qmail 42017 invoked by uid 500); 20 Aug 2014 08:44:26 -0000 Delivered-To: apmail-cassandra-commits-archive@cassandra.apache.org Received: (qmail 41977 invoked by uid 500); 20 Aug 2014 08:44:26 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 41965 invoked by uid 99); 20 Aug 2014 08:44:26 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 20 Aug 2014 08:44:26 +0000 Date: Wed, 20 Aug 2014 08:44:26 +0000 (UTC) From: "Jason Brown (JIRA)" To: commits@cassandra.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Comment Edited] (CASSANDRA-6809) Compressed Commit Log MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CASSANDRA-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14103626#comment-14103626 ] Jason Brown edited comment on CASSANDRA-6809 at 8/20/14 8:42 AM: ----------------------------------------------------------------- If we did try an async flush of the buffers in a sync thread, I think it would be instructive to actually measure that it is demonstratively beneficial, rather than assuming that it is. I'm not immediately sure how to measure that, but it's near 2am and coffee time doesn't begin for several more hours. was (Author: jasobrown): If we did try an async flush of the buffers in a sync thread, I think it would be instructive to actually measure that it is demonstratively beneficial, rather than assuming that it is. > Compressed Commit Log > --------------------- > > Key: CASSANDRA-6809 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6809 > Project: Cassandra > Issue Type: Improvement > Reporter: Benedict > Assignee: Branimir Lambov > Priority: Minor > Labels: performance > Fix For: 3.0 > > > It seems an unnecessary oversight that we don't compress the commit log. Doing so should improve throughput, but some care will need to be taken to ensure we use as much of a segment as possible. I propose decoupling the writing of the records from the segments. Basically write into a (queue of) DirectByteBuffer, and have the sync thread compress, say, ~64K chunks every X MB written to the CL (where X is ordinarily CLS size), and then pack as many of the compressed chunks into a CLS as possible. -- This message was sent by Atlassian JIRA (v6.2#6252)