Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Thu, 21 Nov 2013 22:02:36 +0000 (UTC)
From: "Benedict (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.12477773.1287518260714.10845.1385071356135@arcas>
In-Reply-To: <JIRA.12477773.1287518260714@arcas>
References: <JIRA.12477773.1287518260714@arcas>
Subject: [jira] [Updated] (CASSANDRA-1632) Thread workflow and cpu affinity
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/CASSANDRA-1632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Benedict updated CASSANDRA-1632:
--------------------------------

    Attachment: patch_v5a.diff

Slight tweak - only flushes if both our local queue and the backlog queue are empty. This is a slight behavioural change, but it should maybe allow us to batch more items in the buffer to improve compression.

> Thread workflow and cpu affinity
> --------------------------------
>
>                 Key: CASSANDRA-1632
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1632
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Chris Goffinet
>            Assignee: Jason Brown
>              Labels: performance
>             Fix For: 2.1
>
>         Attachments: 1632-v2.txt, 1632-v3.diff, 1632-v4.diff, 1632_batchRead-v1.diff, patch_v5.diff, patch_v5a.diff, threadAff_reads.txt, threadAff_writes.txt
>
>
> Here are some thoughts I wanted to write down, we need to run some serious benchmarks to see the benefits:
> 1) All thread pools for our stages use a shared queue per stage. For some stages we could move to a model where each thread has its own queue. This would reduce lock contention on the shared queue. This workload only suits the stages that have no variance, else you run into thread starvation. Some stages that this might work: ROW-MUTATION.
> 2) Set cpu affinity for each thread in each stage. If we can pin threads to specific cores, and control the workflow of a message from Thrift down to each stage, we should see improvements on reducing L1 cache misses. We would need to build a JNI extension (to set cpu affinity), as I could not find anywhere in JDK where it was exposed. 
> 3) Batching the delivery of requests across stage boundaries. Peter Schuller hasn't looked deep enough yet into the JDK, but he thinks there may be significant improvements to be had there. Especially in high-throughput situations. If on each consumption you were to consume everything in the queue, rather than implying a synchronization point in between each request.


--
This message was sent by Atlassian JIRA
(v6.1#6144)